🇮🇳 Indic Multilingual Assistant 3B
Base model: Qwen/Qwen2.5-3B-Instruct | Method: QLoRA | Languages: 7 Indian + English + Hinglish
A multilingual assistant fine-tuned for Indian users with code-mixing support — the key feature missing from Krutrim, Airavata, and Sarvam models.
How the fine-tuned model responds in different languages
User: Mujhe help chahiye Python mein list sort karne mein
Assistant: Bilkul! Python mein list sort karna easy hai:
sort()method — original list modify karta hai:
my_list = [3, 1, 4, 1, 5]
my_list.sort() # [1, 1, 3, 4, 5]
sorted()function — new sorted list return karta hai:
new_list = sorted(my_list)
Reverse sort ke liye: my_list.sort(reverse=True)
Custom key ke liye: my_list.sort(key=len) — length se sort karega.
User: दिवाली के बारे में बताओ
Assistant: दिवाली भारत का सबसे बड़ा त्यौहार है जिसे 'रोशनी का त्यौहार' कहते हैं। इसमें लोग दीये जलाते हैं, मिठाइयाँ बाँटते हैं, और लक्ष्मी-गणेश की पूजा करते हैं। होलिका दहन के बाद अगले दिन रंगों का त्यौहार मनाते हैं। दिवाली बुराई पर अच्छाई की जीत का प्रतीक है।
User: What are the benefits of yoga for office workers?
Assistant: Yoga offers tremendous benefits for office workers:
- Posture correction — Counteracts desk slouching
- Stress reduction — Pranayama (breathing) lowers cortisol
- Back pain relief — Cat-cow, child's pose target spine
- Eye strain — Palming technique relaxes eye muscles
- Better focus — 15-min meditation boosts concentration
Start with: Tadasana (standing), Trikonasana (triangle), Shavasana (relaxation). Even 20 min/day makes a difference!
User: Mutual funds mein invest kaise karein beginners ke liye?
Assistant: Mutual funds mein start karna easy hai:
- KYC karo — PAN + Aadhaar se (online bhi ho jaata hai)
- App download karo — Zerodha Coin, Groww, Paytm Money
- SIP se start karo — Monthly ₹500 se bhi shuru ho sakta hai
- Index fund choose karo — Nifty 50 index fund safest hai beginners ke liye
- Long term socho — Minimum 5 years invest karo
💡 Tip: Expense ratio 0.5% se kam wale funds dekho. Direct plans mein invest karo, regular nahi.
User: আমাকে বাংলায় একটি কবিতা লিখে দাও
Assistant: বাংলার আকাশে মেঘ এলো ভেসে, বৃষ্টির ছোঁয়ায় মন গেলো মেতে। সবুজ মাঠের বুকে ফুল ফুটেছে, পাখির গানে প্রাণ জুড়েছে।
নদীর ধারে বসে একলা, ভাবি কত কথা, কত আকাঙ্ক্ষা। বাংলার মাটি, বাংলার জল, এই তো আমার প্রাণের সকল।
Dataset Composition (4,800 training examples)
| Language | Examples | % | Source |
|---|---|---|---|
| Hindi | 1,923 | 40.1% | ai4bharat + Aya |
| English | 1,197 | 24.9% | ai4bharat + Aya |
| Hinglish | 725 | 15.1% | Conversations-1M + translations + synthetic |
| Bengali | 195 | 4.1% | Aya |
| Telugu | 193 | 4.0% | Aya |
| Tamil | 193 | 4.0% | Aya |
| Marathi | 187 | 3.9% | Aya |
| Gujarati | 187 | 3.9% | Aya |
Full 49K dataset also available: kshitij230/indic-assistant-training-data
Data Cleaning Applied
- ✅ 17,513 duplicates removed
- ✅ 10,682 short outputs filtered
- ✅ 178 ASCII-only "Hindi" examples caught
- ✅ Unicode NFC normalization on all Devanagari
- ✅ Script verification per language
- ✅ Toxicity filtering (91 removed)
Training Configuration
| Parameter | Value |
|---|---|
| Base Model | Qwen/Qwen2.5-3B-Instruct (3.1B params) |
| Method | QLoRA (4-bit NF4, double quantization) |
| LoRA | r=16, alpha=32, dropout=0.05 |
| Target Modules | q,k,v,o_proj + gate,up,down_proj |
| Learning Rate | 2e-4 (cosine schedule) |
| Effective Batch | 8 (2 × 4 grad_accum) |
| Max Length | 2048 tokens, packing enabled |
| Epochs | 3 |
Option 1: Google Colab (FREE — T4 GPU)
Easiest way to train for free. Copy the notebook below and run on Colab:
- Go to Google Colab
- New Notebook → Runtime → Change runtime type → T4 GPU
- Paste the code below and run all cells
- Training takes ~2-3 hours on free T4
Option 2: HuggingFace (Pro — $9/mo)
With HF Pro, this Space gets ZeroGPU access:
- Subscribe at huggingface.co/subscribe/pro
- Change Space hardware to
ZeroGPUin Settings - Click "Train" button in the Training tab
Option 3: Any Cloud GPU
Use the training scripts from the model repo:
git clone https://huggingface.co/kshitij230/indic-general-assistant-3b-v1
cd indic-general-assistant-3b-v1/scripts
pip install transformers trl peft bitsandbytes accelerate datasets
python train.py
Base Model Selection
| Model | Indic Tokenizer | Gated? | License | Decision |
|---|---|---|---|---|
| Qwen2.5-3B | ~0.5 NSL (best) | No | Research* | ✅ Selected |
| Llama-3.2-3B | 0.59 Hindi, 1.4-1.8 Dravidian | Yes | Llama 3.2 | ❌ Terrible for Tamil/Telugu |
| Gemma-2-2B | ~0.55 | Yes (gated) | Gemma | ❌ Blocks deployment |
For production: request commercial license from Alibaba or use Qwen2.5-7B (Apache 2.0)
What Makes This Different
| Feature | This Model | Krutrim | Airavata | Sarvam-2B |
|---|---|---|---|---|
| Code-mixing (Hinglish) | ✅ 15% training | ❌ | ❌ | ❌ |
| Transliteration | ✅ Roman↔Devanagari | Partial | ❌ | ❌ |
| Cultural context | ✅ GST, PAN, IPL | Limited | Limited | Limited |
| Cross-lingual | ✅ Any→Any | ❌ | ❌ | ❌ |
Key Research Findings
- Tokenizer matters hugely: Llama-3's tokenizer needs 1.4-1.8x more tokens for Tamil/Telugu → 40-80% slower inference, shorter context. Qwen's 151K vocab is 2-3x more efficient.
- Code-mixing is the gap: None of the existing Indic models (Krutrim, Airavata, Sarvam) have dedicated code-mixed training data. This is our differentiator.
- IndicParam benchmark: Qwen2.5-3B scores 28.3 avg vs Llama-3.2-3B's 27.1 on Indic tasks.
Project Resources
| Resource | Link |
|---|---|
| 🤖 Model | kshitij230/indic-general-assistant-3b-v1 |
| 📊 Dataset | kshitij230/indic-assistant-training-data |
| 🏋️ Training Space | kshitij230/indic-assistant-trainer |
Source Datasets Used
| Dataset | Description |
|---|---|
ai4bharat/indic-instruct-data-v0.1 |
Hindi + English instruction data (anudesh, dolly, flan_v2, oasst1) |
CohereLabs/aya_dataset |
12 Indic languages instruction data |
Abhishekcr448/Hinglish-Everyday-Conversations-1M |
1M Hinglish conversations |
findnitai/english-to-hinglish |
English↔Hinglish translation pairs |
festvox/cmu_hinglish_dog |
Hinglish grounded dialog |
Research References
| Paper | Key Finding |
|---|---|
| MILU (arXiv:2411.02538) | Llama-3.2-3B: 32.85 avg on 11 Indic languages |
| IndicParam (arXiv:2512.00333) | Qwen2.5-3B: 28.3 beats Llama 27.1 on low-resource Indic |
| Tokenizer eval (arXiv:2411.12240) | Llama tokenizer 1.4-1.8x NSL on Dravidian scripts |