naqlنقل
Inspect, convert, and validate ML models. Preserve your Arabic tokenizer.
8 formats supported · GPTQ/AWQ aware · Arabic tokenizer validation · lightweight
$
pip install naql
Chapter I
The Problem
Converting models breaks Arabic tokenizers.
You fine-tune a model with Arabic support. You convert it from HuggingFace to GGUF for deployment. The conversion silently drops Arabic tokens, shifts vocab indices, or corrupts diacritics. Your model now outputs gibberish for Arabic while English still works fine. naql catches this before it reaches production.
Chapter II
8 Formats
Read model headers directly. No heavy framework required.
- GGUF
llama.cpp quantized models — .gguf files - SafeTensors
HuggingFace safe format — .safetensors files - ONNX
Open Neural Network Exchange — .onnx files - MLX
Apple MLX framework — weights.npz + config.json - JANG
Adaptive mixed-precision MLX — "The GGUF for MLX" - PyTorch
Native checkpoints — .pt, .bin files - HuggingFace
Model directories with config, tokenizer, weights - GPTQ
GPU quantized models via AutoGPTQ — 4-bit inference - AWQ
Activation-aware weight quantization — high quality 4-bit
Chapter III
Arabic Check
Verify Arabic tokenizer preservation before and after conversion.
Coverage Scan
Counts Arabic tokens in the vocabulary. Reports percentage, character coverage, and script distribution.
naql arabic model/
Validation
Compares source and target tokenizers. Catches dropped tokens, shifted indices, and corrupted diacritics.
naql validate source/ target/
28 Base Letters
Verifies all 28 Arabic letters, tashkeel diacritics, Arabic digits, and common bigrams are present.
All 28 base letters covered
Bigram Coverage
Tests common Arabic bigrams against the tokenizer. Low coverage means over-tokenized Arabic text.
94% bigram coverage
Chapter IV
Conversion Matrix
What converts to what. naql generates the command and validates the output.
| From / To | GGUF | SafeTensors | ONNX | MLX | PyTorch | HF | GPTQ | AWQ |
|---|---|---|---|---|---|---|---|---|
| GGUF | - | Yes | - | Yes | - | Yes | - | - |
| SafeTensors | Yes | - | Yes | Yes | Yes | Yes | - | - |
| ONNX | - | Yes | - | - | Yes | - | - | - |
| MLX | Yes | Yes | - | - | - | Yes | - | - |
| PyTorch | Yes | Yes | Yes | Yes | - | Yes | - | - |
| HuggingFace | Yes | Yes | Yes | Yes | Yes | - | Yes | Yes |
| GPTQ | Yes | - | - | - | - | - | - | - |
| AWQ | Yes | - | - | - | - | - | - | - |
Chapter V
Commands
7 commands. Zero config.
- inspect
Read model format, size, quantization, layers, and vocab - arabic
Scan tokenizer vocab for Arabic token coverage - convert
Convert between formats with Arabic validation - validate
Compare source and target after conversion - diff
Compare two models side by side with Arabic token delta - formats
List supported formats and conversion paths - explain
Show how naql works — detection, conversion, validation
Chapter VI
Get Started
# Install
$ pip install naql
# Inspect a model
$ naql inspect model.gguf
Format: GGUF Quant: Q4_K_M Params: 2.5B Vocab: 151,936
# Check Arabic tokenizer
$ naql arabic model/
Arabic tokens: 4,217 (2.8%) Letters: 28/28 Verdict: GOOD
# Convert to MLX
$ naql convert model/ --to mlx
Converting HuggingFace -> MLX... done. Arabic check: PASS
# Validate the conversion
$ naql validate model/ model-mlx/
Vocab match: 151,936/151,936 Arabic tokens: preserved OK
Chapter VII
Demo
See it in action.
