In Action
One command. Full picture.
See how much more Arabic costs across 18 LLM tokenizers. Benchmark scores, diacritics analysis, dialect comparison, and more.
pip install artok
GPT-4.1, Claude 4.6, Gemini, DeepSeek, Llama 4, Qwen 3.5, Mistral, Grok, Cohere, AI21.
Arabic friendliness score 0-100 across 8 text categories.
How diacritics inflate tokens. Unique to Arabic.
MSA vs Egyptian vs Gulf vs Levantine vs Moroccan.
Rank by efficiency + cost + value. Medals for top 3.
Color each word by token cost. See where the tax hits.
Estimate monthly costs at scale with English comparison.
Savings from switching providers, adjusted for token ratios.
Arabic vs English vs French vs Chinese per tokenizer.
Pricing from GitHub, cached 24h. --update to refresh.
Share results as images. --export out.svg
Type Arabic, see counts live. --watch
artok --update.| Tokenizer | Provider | Input $/1M | Output $/1M |
|---|---|---|---|
| GPT-4.1 | OpenAI | $2.00 | $8.00 |
| GPT-4.1 mini | OpenAI | $0.40 | $1.60 |
| GPT-4.1 nano | OpenAI | $0.10 | $0.40 |
| GPT-4o | OpenAI | $2.50 | $10.00 |
| GPT-4o mini | OpenAI | $0.15 | $0.60 |
| Claude Opus 4.6 | Anthropic | $5.00 | $25.00 |
| Claude Sonnet 4.6 | Anthropic | $3.00 | $15.00 |
| Claude Haiku 4.5 | Anthropic | $1.00 | $5.00 |
| Llama 4 | Meta | $0.18 | $0.18 |
| Qwen 3.5 | Alibaba | $0.10 | $0.40 |
| Mistral Large 3 | Mistral | $0.50 | $1.50 |
| Mistral Small | Mistral | $0.10 | $0.30 |
| Gemini 2.5 Pro | $1.25 | $10.00 | |
| Gemini 3 Flash | $0.50 | $3.00 | |
| DeepSeek V3.2 | DeepSeek | $0.27 | $1.10 |
| Grok 2 | xAI | $2.00 | $10.00 |
| Command R+ | Cohere | $2.50 | $10.00 |
| Jamba 1.5 | AI21 | $0.20 | $0.40 |