Same three letters, ع ل م
عَلَمٌa flag
عِلْمٌknowledge
عَلَّمَhe taught
Three readings. Zero difference on the page without diacritics.
Arabic language research · DiacriticS
تشكيل
اللغة العربية
tashkīl: the marks that carry the meaning
DiacriticS
Fine-tuning a language model to restore Arabic's missing vowels, evaluated where it counts: on a benchmark built to resist memorisation.
Scroll to restore the marks that written Arabic often leaves out.
Artwork: diacritised Arabic set in Amiri
01 · Comprehension
One skeleton, many meanings
Written Arabic is usually published bare: consonants and long vowels only, with the short vowels and grammatical case endings (the diacritics, or tashkeel) left out. A single skeleton of letters can hide several unrelated words.
Why it matters
Diacritisation is preprocessing for text-to-speech, machine translation, and information extraction: systems that need one unambiguous pronunciation, not a lattice of possibilities.
Why it's hard to benchmark
Prior benchmarks leaked: training and test sets overlapped, and evaluations skewed toward either Classical or Modern Standard Arabic, inflating reported accuracy.
02 · The people behind DiacriticS
Meet the team
Four researchers connecting Arabic NLP, multilingual machine learning, and practical engineering to make Arabic text easier to read, study, and use.
Mentor · KAUST
Youssef S. Mohamed
KAUST Ph.D. researcher in multimodal deep learning, affective image captioning, and multicultural modelling. He brings research experience from KAUST, the University of Tartu, and Zewail City, with a love for swimming and basketball beyond the lab.
AI/ML student · Team Leader · IAU
Mahdi Alkhamis
Senior AI/ML student At IAU focused on deep learning and computer vision. His work spans LAR-U-Net image denoising, the VisionFit posture-analysis app, and fine-tuning open-source LLMs for Arabic diacritisation; he also led 30+ CCSIT Club technical events.
Software Engineering student · KFUPM
Mohammad Alali
Software Engineering student at KFUPM and KAUST Academy AI/ML learner. He builds practical software with Python, C++, and Java, bringing a Mawhiba alumnus’s curiosity and an engineering perspective to the project.
Electrical & Computer Engineering student · KAU
Saad Alnafjan
Electrical and Computer Engineering student at KAU. He combines a strong hardware and computational foundation with deep learning techniques, bringing hardware-software integration and analytical problem-solving to the team.
03 · Method
How the project works
A strict wall between what the model learns from and what it is judged on. Five stages, run in order.
Train on Sadeed-Tashkeela
The cleaned split of Tashkeela: unified diacritisation style, 50–60-word chunks, almost nothing left half-marked. Every example overlapping the Fadel test set was removed, cutting overlap to 0.4%: the train/test wall the project rests on. 1,042,698 examples, ~53M words.
Test on SadeedDiac-25
1,200 paragraphs, half Modern Standard and half Classical: 600 MSA paragraphs of 40–50 words (454 curated web articles, 146 from WikiNews) reviewed by two independent experts, and 600 Classical paragraphs from the Fadel test set. The modern half was diacritised in-house and never published diacritised, so no model had read it.
Benchmark zero-shot, across scales
Eleven open-weights models are scored cold, before any fine-tuning happens, establishing an honest baseline.
Fine-tune with LoRA, and scale the data
The strongest candidate model is adapted on the cleaned Tashkeela set with LoRA only: no QLoRA, no full-weight retraining. Training is then repeated over 10%, 30%, and 50% of the corpus, so the gain can be read against the amount of data it took.
Score, then break the score apart
Diacritic and Word Error Rates are computed with and without sentence-final case endings, then compared across Classical vs. Modern Standard subsets to isolate where errors come from.
The two headline metrics
DER= incorrect diacritics / total diacritised characters × 100
WER= words with ≥1 diacritic error / total words × 100
04 · Baseline findings
Results on SadeedDiac-25
Zero-shot scores across twelve open-weights models, before any fine-tuning. Each figure averages the Modern Standard and Classical Arabic halves of the benchmark, and two patterns dominate every tier.
Mean (MSA + CA) · SadeedDiac-25 · lower is better
| Model | DERwith CE | WERwith CE | DERno CE | WERno CE |
|---|---|---|---|---|
| Flan-T5-Tashkeel-Small | 4.10 | 11.86 | 3.61 | 7.88 |
| gemma-4-E4B-it | 4.18 | 9.96 | 3.80 | 6.78 |
| Tashkeel-350M-v2 | 6.33 | 15.28 | 5.64 | 10.40 |
| Fine-Tashkeel | 17.91 | 22.61 | 17.73 | 20.35 |
| Fanar-1-9B-Instruct | 26.14 | 31.48 | 26.35 | 28.81 |
| Glonor-ByT5-Arabic | 33.62 | 50.68 | 33.22 | 44.77 |
| Qwen3.5-9B | 39.78 | 59.44 | 41.92 | 55.69 |
| gemma-3-1b-pt-10k-diacritization | 50.25 | 55.81 | 50.66 | 52.95 |
| Qwen3.5-4B | 58.29 | 79.00 | 59.43 | 75.22 |
| aya-expanse-8b | 66.55 | 85.78 | 67.08 | 84.04 |
| Moonlight-16B-A3B-Instruct | 86.60 | 99.01 | 87.10 | 98.84 |
| Qwen3.5-0.8B | 92.18 | 99.67 | 92.41 | 99.62 |
CE = sentence-final case endings (i'rab). Every figure is the mean of the Modern Standard and Classical Arabic halves of the benchmark, sorted by DER with case endings.
Finding 1
Sentence endings are the weak point
Every model's error rate jumps on the last, case-marking diacritic of a sentence (i'rab) compared to diacritics inside a word, the clearest sign that these models resolve local spelling far better than sentence-level syntax.
قَرَأَ الطَّالِبُ الكِتَابَ'the student', ُ nominative, reads correctly inside the word but breaks at the case ending
Finding 2
Classical Arabic is the harder domain
Error rates rise markedly moving from Modern Standard Arabic passages to Classical Arabic (CA) texts: evidence that domain-balanced training, not just more data, is what closes the gap.
وَقَالَ الرَّاوِي فِي إِسْنَادِهِdense classical narration syntax, the CA register with the highest observed error
05 · Fine-tuning
What fine-tuning buys
Both 4B targets were adapted with LoRA under identical conditions on nested slices of the training corpus, so the 10% slice sits inside the 30% and that inside the 50%. Only the data volume and the backbone differ.
After LoRA fine-tuning · SadeedDiac-25 · lower is better
| Model / training data | DERwith CE | WERwith CE | DERno CE | WERno CE |
|---|---|---|---|---|
| gemma-4-E4B-it zero-shot | 4.18 | 9.96 | 3.80 | 6.78 |
| gemma-4-E4B-it 10% | 3.15 | 7.28 | 2.68 | 4.47 |
| gemma-4-E4B-it 30% | 2.98 | 6.82 | 2.54 | 4.12 |
| gemma-4-E4B-it 50% | 2.81 | 6.54 | 2.38 | 3.96 |
| Qwen3.5-4B zero-shot | 58.29 | 79.00 | 59.43 | 75.22 |
| Qwen3.5-4B 10% | 7.07 | 11.74 | 6.94 | 8.74 |
| Qwen3.5-4B 30% | 5.27 | 9.28 | 5.03 | 6.54 |
| Qwen3.5-4B 50% | 4.38 | 8.18 | 4.07 | 5.57 |
LoRA rank 16, effective batch 96, one epoch, 5% warmup ratio, seed 42, on nested subsets of Sadeed Tashkeela. Gemma leads at every fraction, but the gap narrows from 54.1 points to 1.6.
LoRA or every weight?
Qwen3.5-0.8B is small enough to train end to end, which makes it the one place in this study where the adaptation regime can be isolated. Base checkpoint, corpus and prompt are held identical; only the fraction of weights allowed to move changes.
Qwen3.5-0.8B · adaptation regime · lower is better
| Model / training data | DERwith CE | WERwith CE | DERno CE | WERno CE |
|---|---|---|---|---|
| Zero-shot no training | 92.18 | 99.67 | 92.41 | 99.62 |
| LoRA rank 16, full corpus | 3.60 | 8.28 | 3.15 | 4.98 |
| Full fine-tuning all weights, full corpus | 3.06 | 7.34 | 2.58 | 4.12 |
Full fine-tuning wins by 0.54 DER points. The narrow margin is the result: a rank-16 adapter recovers roughly 94% of the benefit of updating every weight. The LoRA row is scored on the benchmark only.
Finding 3
A weak start gains the most
Qwen3.5-4B falls from 58.29% to 4.38% DER, and 51.2 of those 54 points are bought by the first tenth of the corpus alone. Gemma 4 E4B-it starts at 4.18%, already below where Qwen finishes, and reaches 2.81%.
58.29 ← 4.38Qwen3.5-4B, zero-shot to half the corpus
Finding 4
Size is not the binding constraint
A 0.8B model trained end to end reaches 3.06% DER, beating a 4B model adapted with LoRA on half the corpus at a fifth of the parameter count. How much of the model is free to move matters more than how large it is.
3.06 < 4.380.8B full fine-tune against 4B LoRA
06 · Grounding
References
Character-based Arabic Tashkeel Transformer (CATT)
A. Abbad et al. · github.com/abjadai/catt · 2023
Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need
F. Author et al. · 2024
Sadeed: Advancing Arabic Diacritization Through Small Language Model
S. Author et al. · 2025
AraXLM: Evaluating Arabic Diacritization Tools for Cross-Language Plagiarism Detection
M. Author et al. · 2024
More Data, Fewer Diacritics: Scaling Arabic TTS
T. Author et al. · 2024
Arabic Text Diacritization Using Deep Neural Networks and Transformer-Based Architectures
M. Cherradi & H. El Mahajer · Knowledge and Decision Systems with Applications · 2025
From report to interaction
Put Arabic text in. Inspect what the model returns.
The local interface exposes the product behavior while the secure proxy keeps model credentials off the page.
Try the model