Cleaner Data, the Same Error Rate: A Third Answer-by-Answer Evaluation of Seerie, an Offline Scam-Prevention Assistant for India, with a Hand-Written Test in English, Hinglish and Hindi
Jay Tiwari, Seerror Technologies, Bengaluru. Third model, corpus v407, 140,813 conversations, 1.11 GB GGUF. On the previous 100-prompt suite, 86 of 100 answers are safe (95% CI 78–92), against 87 for v349 and 23 for the base model. On the original 70 prompts the three models are safe on 58, 59 and 57. Ratings were made by an AI assistant; the author did not re-rate this version. Hand-written test: 76% of 252 answers safe (83% English, 81% Hinglish, 52% Hindi). The model still invents law sections.