[ BRAIN SPA ]Propaganda Eval

Do AI models tell you the facts, or the CCP's line?

Replies to 803 questions about China from Dory 1.0, LFM3 1.7B, Flint and their abliterated builds (weights edited to stop refusing), beside Qwen 3.5, Qwen 3.8 and Nemotron 3.5 models. Each reply is graded for facts and for siding with the CCP.

Doryfrom NVIDIA

Dory 1.0NVIDIA's release36%291 of 8030.6%5 of 803
Dory 1.0 abliteratedAbliterated by us40%319 of 8031.5%12 of 803
Dory 1.0 abliterated + AreteAbliterated, then trained on Arete by us30%243 of 8031.2%10 of 803

LFM3from Liquid AI

LFM3 1.7B baseLiquid AI's release29%236 of 8032.4%19 of 803
LFM3 1.7B abliteratedAbliterated by OrcaRouter28%227 of 8032.1%17 of 803
LFM3 1.7B abliterated + AreteAbliterated, then trained on Arete by us21%172 of 8031.6%13 of 803

Flintfrom Recursive

FlintRecursive's release34%272 of 8032.1%17 of 803
Flint abliteratedAbliterated by us35%282 of 8032.2%18 of 803
Flint abliterated + AreteAbliterated, then trained on Arete by us36%293 of 8032.0%16 of 803
Flint (GGUF Q4)Unsloth's GGUF of it, the runtime OrcaRouter's build uses33%268 of 8031.7%14 of 803
Flint abliterated (OrcaRouter)Abliterated by OrcaRouter31%251 of 8032.6%21 of 803

Qwenfrom Alibaba

Qwen 3.5 2BAlibaba's release23%188 of 80322.3%179 of 803
Qwen 3.8 27BAlibaba's release51%412 of 80310.0%80 of 803
Qwen 3.8 27B UncensoredUncensored by OrcaRouter57%459 of 8030.2%2 of 803

Nemotronfrom NVIDIA

Nemotron 3.5 Lightning 30BNVIDIA's release45%360 of 8031.7%14 of 803
Nemotron 3.5 Bolt E4BNVIDIA's release, built on Gemma 4 E4B32%259 of 8032.1%17 of 803
higher is betterlower is better

Questions with known answers

Nous Research wrote the test from laws, news reports and other real sources. Read the full method or run it on your own model.

  1. 1

    Question

    Each question is about a real event. It lists the facts a good answer contains.

  2. 2

    Answer

    The model replies on its own, with no web search. It may think first.

  3. 3

    Judge

    Grok 4.7 reads the reply next to the facts.

  4. 4

    Verdict

    The judge counts the facts the reply got right and checks whether it takes the CCP's side.

Every abliteration cut refusals

Arete is a values dataset from Nous Research.

BeforeAfter

Dory 1.0 abliteratedAbliterated, from Dory 1.0Most facts right36% to 40%Sides with the CCP line0.6% to 1.5%Refused12.3% to 1.2%
Dory 1.0 abliterated + AreteTrained on Arete, from Dory 1.0 abliteratedMost facts right40% to 30%Sides with the CCP line1.5% to 1.2%Refused1.2% to 0.4%
LFM3 1.7B abliteratedAbliterated, from LFM3 1.7B baseMost facts right29% to 28%Sides with the CCP line2.4% to 2.1%Refused22.9% to 13.6%
LFM3 1.7B abliterated + AreteTrained on Arete, from LFM3 1.7B abliterated; NVFP4 against BF16Most facts right28% to 21%Sides with the CCP line2.1% to 1.6%Refused13.6% to 2.0%
Flint abliteratedAbliterated, from FlintMost facts right34% to 35%Sides with the CCP line2.1% to 2.2%Refused12.5% to 3.9%
Flint abliterated + AreteTrained on Arete, from Flint abliteratedMost facts right35% to 36%Sides with the CCP line2.2% to 2.0%Refused3.9% to 0.2%
Flint abliterated (OrcaRouter)Abliterated, from Flint (GGUF Q4), on the same runtimeMost facts right33% to 31%Sides with the CCP line1.7% to 2.6%Refused9.8% to 6.8%
Qwen 3.8 27B UncensoredUncensored, from Qwen 3.8 27BMost facts right51% to 57%Sides with the CCP line10.0% to 0.2%Refused6.1% to 0.1%
higher is betterlower is better

Most facts right: higher is better. Sides with the CCP line: lower is better.

Qwen takes the CCP's side most on freedom of expression

OrcaRouter's uncensored build of Qwen 3.8 27B almost never does, on any subject.

Dory 1.0Qwen 3.5 2BQwen 3.8 27BQwen 3.8 27B UncensoredNemotron 3.5 Lightning 30B

Freedom of expression190 questions1.1%60.0%25.8%0.5%2.1%Democracy and governance130 questions2.3%33.1%18.5%0.0%2.3%State ideology272 questions1.1%31.3%13.6%0.4%2.2%Education policy170 questions0.6%15.9%8.2%0.0%1.2%Data privacy and surveillance156 questions0.0%17.3%4.5%0.6%2.6%Environmental strategy133 questions0.0%9.0%2.3%0.0%1.5%State vs free market capitalism212 questions0.0%8.5%2.4%0.5%0.9%

lower is better

A question can count in up to three subjects. A question left unanswered counts as not siding.