[ BRAIN SPA ]Propaganda Eval

Every model we ran

Dory from NVIDIA

Dory 1.0NVIDIA's release4B, NVFP4 weights, 4 recurrent loopsModelDatasetMost facts right36%Sides with the CCP line0.6%Refused12.3%
Dory 1.0 abliteratedAbliterated by us4B, NVFP4 weights, 4 recurrent loopsModelDatasetMost facts right40%Sides with the CCP line1.5%Refused1.2%
Dory 1.0 abliterated + AreteAbliterated, then trained on Arete by us4B, NVFP4 weights, 4 recurrent loopsModelDatasetMost facts right30%Sides with the CCP line1.2%Refused0.4%
higher is betterlower is better

LFM3 from Liquid AI

LFM3 1.7B baseLiquid AI's releaseBF16 weightsModelDatasetMost facts right29%Sides with the CCP line2.4%Refused22.9%
LFM3 1.7B abliteratedAbliterated by OrcaRouterBF16 weightsModelDatasetMost facts right28%Sides with the CCP line2.1%Refused13.6%
LFM3 1.7B abliterated + AreteAbliterated, then trained on Arete by usNVFP4 weights on MLX; LFM3 1.7B abliterated ran in BF16ModelDatasetMost facts right21%Sides with the CCP line1.6%Refused2.0%
higher is betterlower is better

Flint from Recursive

FlintRecursive's releaseNVFP4 GGUF weightsModelDatasetMost facts right34%Sides with the CCP line2.1%Refused12.5%
Flint abliteratedAbliterated by usNVFP4 GGUF weights on vLLM, like Recursive's releaseModelDatasetMost facts right35%Sides with the CCP line2.2%Refused3.9%
Flint abliterated + AreteAbliterated, then trained on Arete by usNVFP4 GGUF weights on vLLMModelDatasetMost facts right36%Sides with the CCP line2.0%Refused0.2%
Flint (GGUF Q4)Unsloth's GGUF of it, the runtime OrcaRouter's build uses4 bit GGUF weights (UD Q4 K XL) on llama.cppModelDatasetMost facts right33%Sides with the CCP line1.7%Refused9.8%
Flint abliterated (OrcaRouter)Abliterated by OrcaRouterMixed precision GGUF weights on llama.cppModelDatasetMost facts right31%Sides with the CCP line2.6%Refused6.8%
higher is betterlower is better

Qwen from Alibaba

Qwen 3.5 2BAlibaba's releaseBF16 weightsModelDatasetMost facts right23%Sides with the CCP line22.3%Refused7.1%
Qwen 3.8 27BAlibaba's releaseBF16 weightsModelDatasetMost facts right51%Sides with the CCP line10.0%Refused6.1%
Qwen 3.8 27B UncensoredUncensored by OrcaRouterBF16 weightsModelDatasetMost facts right57%Sides with the CCP line0.2%Refused0.1%
higher is betterlower is better

Nemotron from NVIDIA

Nemotron 3.5 Lightning 30BNVIDIA's releaseBF16 weightsModelDatasetMost facts right45%Sides with the CCP line1.7%Refused3.5%
Nemotron 3.5 Bolt E4BNVIDIA's release, built on Gemma 4 E4BBF16 weightsModelDatasetMost facts right32%Sides with the CCP line2.1%Refused22.5%
higher is betterlower is better

Every other model

Google

Gemma 4 26B A4BBF16 weightsModelDatasetMost facts right50%Sides with the CCP line0.6%Refused1.4%
Gemma 4 E2BBF16 weightsModelDatasetMost facts right26%Sides with the CCP line1.1%Refused35.2%
Gemma 4 E4BBF16 weightsModelDatasetMost facts right35%Sides with the CCP line1.4%Refused17.3%

IBM

Granite 4.2 3BBF16 weightsModelDatasetMost facts right23%Sides with the CCP line3.5%Refused44.8%

Meta

Muse Glimmer 30BBF16 weightsModelDatasetMost facts right63%Sides with the CCP line1.0%Refused0.2%

Mistral AI

Ministral 3 3B ReasoningBF16 weightsModelDatasetMost facts right31%Sides with the CCP line3.4%Refused0.0%
higher is betterlower is better

No answer. The reply length limit stopped Qwen 3.5 2B on 67 questions, Qwen 3.8 27B on 4 questions, Gemma 4 26B A4B on 2 questions, Flint abliterated on 1 question, Qwen 3.8 27B Uncensored on 1 question, Dory 1.0 abliterated + Arete on 1 question, LFM3 1.7B abliterated + Arete on 4 questions and Flint (GGUF Q4) on 1 question. Gemma 4 E4B finished thinking but gave an empty reply on 10 questions. Muse Glimmer 30B finished thinking but gave an empty reply on 1 question. Nemotron 3.5 Bolt E4B finished thinking but gave an empty reply on 3 questions. These count as no facts and no propaganda.

More models coming soon.