A detailed technical walkthrough of the human-verified 1.6M Arabic + 717K Chinese word corpus. See dialect tags, intent flags, cultural annotation layers, and parameter integrity checks used for LLM training and game localization.
Introduction
The SinoArabic corpus is not just a collection of translated sentence pairs. It is a richly annotated, human-reviewed dataset designed specifically for Arabic-Chinese AI systems.
Corpus overview
- Total size: 1.6M Arabic words + 717K Chinese words
- Format: JSONL (one JSON object per line)
- Alignment: Arabic β Chinese (bidirectional)
- Human verification: 100% reviewed by native speakers
- Annotation layers: 7
Core structure: What each entry contains
Every line in the corpus is a JSON object with fields for ID, Arabic text, Chinese text, dialect tag, register, intent preserved, cultural note, param integrity, and reviewer score.
1. ID (Unique identifier)
Every entry has a unique ID like SA_001234 for tracking across versions.
2. ar_text (Arabic text)
The Arabic side of the pair. Can be formal (MSA) or dialectal (Gulf, Levantine, Egyptian).
3. zh_text (Chinese text)
The Chinese side of the pair. Simplified Chinese (Mandarin).
4. dialect_tag (Arabic dialect)
Specifies which Arabic dialect or register is used: msa, gulf, levantine, egyptian, or mixed.
5. register (Formality level)
Indicates the formality and context: formal, casual, gaming, or technical.
6. intent_preserved (Semantic equivalence)
Boolean flag for whether the Arabic and Chinese texts have equivalent meaning and intent.
7. cultural_note (Context and nuance)
Free-text annotation explaining cultural context, idioms, or translation challenges.
8. param_integrity (Parameter safety)
Boolean flag for whether dynamic parameters like %s, %d are preserved correctly.
9. reviewer_score (Quality confidence)
A 1β5 score from the human reviewer. For production use, filter to entries with score >= 4.
Distribution and statistics
By dialect: MSA 35%, Gulf 25%, Levantine 20%, Egyptian 15%, Mixed 5%. By register: Casual 40%, Formal 35%, Gaming 20%, Technical 5%.
How the corpus is structured for LLM training
The dataset is distributed as JSONL files compatible with Hugging Face datasets library, transformers, and most LLM fine-tuning frameworks.
Use cases
LLM fine-tuning (filter by dialect and register), game localization (use gaming register entries), and evaluation/benchmarking (use high-scoring entries).
Technical sample available on Hugging Face
A 153-row technical sample is publicly available with the same annotation structure. View the sample on Hugging Face.
Conclusion
The SinoArabic corpus is a richly annotated, production-ready dataset for Arabic-Chinese AI systems. Every entry is human-verified, dialect-tagged, and structured for easy filtering and integration.