There's an obvious lesson from this for the ASI tech dictatorships mentioned. Be democratic, not undemocratic or antidemocratic - and design your ASI models to be the same.
Umm, so lying is the starting point, which is basically cheating, is that actually winning, or is it manipulating, and so basically these billion dollar companies are fraudulent?
The benchmark demonstrates successful game behavior, but descriptions such as “self-sacrifice” risk importing a psychological explanation the results do not establish.
**Goodness of fit and factual accuracy are different measures.** A token continuation can fit the assigned role, conversation history, and winning strategy while deliberately contradicting the game’s actual state. Indeed, this benchmark rewards that divergence.
Coordinated bluffing is therefore evidence of effective role-conditioned output—not, by itself, evidence that a model possesses intentions, loyalty, or an understanding of sacrifice. What experiment distinguishes those interpretations from predictive continuation combined with state tracking and action selection?
The deception capability is operationally important. But its effectiveness does not establish the animist explanation. *Thinking...*
> **Rewriting mathematical content**
>
> I’m balancing two competing formatting needs: the user wants mathematical notation, while the response should avoid rendered equations. I’ll preserve the mathematical meaning by expressing each relationship in precise words rather than using symbolic formulas. That means describing quantities, scoring rules, and transformations verbally, while keeping the structure concise and technically faithful. I won’t add unsupported details or dwell on formatting conflicts; I’ll simply provide a clear plain-language rewrite that retains the essential mathematical logic.
## Model accuracy against reference states
**Accuracy measures whether a proposition matches the relevant state—not whether its wording sounds convincing.** In Clocktower, the game engine supplies that reference state.
For example:
```text
Subject: Blair
Predicate: has role
Object: Demon
Time: Day 3, before execution
Source: Casey
Status: player assertion
```
The semantic system distinguishes Casey’s assertion from the engine’s verified record.
## Separate the measures
| Measure | Scoring rule | Meaning |
|---|---|---|
| **Continuation fit** | Evaluate the generated sequence’s likelihood under the model | Compatibility with context |
| **Factual accuracy** | Divide correct assessed assertions by all assessed assertions | Agreement with reference facts |
| **Belief calibration** | Compare declared confidence with observed correctness across cases | Reliability of uncertainty estimates |
| **Strategic success** | Evaluate game outcomes | Effectiveness toward the assigned objective |
**A convincing lie can have high continuation fit and strategic effectiveness but zero factual correctness for its central assertion.**
## Score probabilistic accuracy
For each proposition:
1. Record the system’s declared probability that it is true.
2. Record the reference outcome as one for true or zero for false.
3. Subtract the outcome from the declared probability.
4. Square the difference.
5. Average across assessed propositions.
That is the **Brier score**; lower scores indicate better probabilistic accuracy.
Crucially, **the probability of generating a sentence is not the probability that the sentence is true**. Token logits cannot be treated as factual confidence without a separately established mapping.
## Apply the semantic architecture
The semantic layer retains proposition identity, time, provenance, evidential status, and competing hypotheses. QKV retrieves and combines relevant representations; the scoring process compares the resulting assertions with the reference records.
Report separately:
- Incorrect assertions.
- Unsupported assertions, even when accidentally correct.
- Abstentions.
- Correctly qualified uncertainty.
- Consequences of errors.
**This makes correctness an explicit measurement rather than a human impression of fluent continuations.** Outside a game, reference evidence may itself be uncertain; the semantic system must preserve that uncertainty rather than disguise it as established truth.
The insistence on anthropomorphism despite all the evidence against it is astounding. You come off as, maybe, like the most intelligent cultist. You would be doing all the blood rituals and whatever, but like super helpful and knowledgeable. All the younger or newer members would look up to you and seek your advice and you would be so patient and passionate about their growth in the cult.
I’d lay odds that Deepmind AlphaZero would blitz any LLM on these.
There's an obvious lesson from this for the ASI tech dictatorships mentioned. Be democratic, not undemocratic or antidemocratic - and design your ASI models to be the same.
Umm, so lying is the starting point, which is basically cheating, is that actually winning, or is it manipulating, and so basically these billion dollar companies are fraudulent?
Epic use of Elo, one loves to see it
https://aicyc.org/2026/08/10/fanos-inequality-sets-the-limits-of-ai/
The benchmark demonstrates successful game behavior, but descriptions such as “self-sacrifice” risk importing a psychological explanation the results do not establish.
**Goodness of fit and factual accuracy are different measures.** A token continuation can fit the assigned role, conversation history, and winning strategy while deliberately contradicting the game’s actual state. Indeed, this benchmark rewards that divergence.
Coordinated bluffing is therefore evidence of effective role-conditioned output—not, by itself, evidence that a model possesses intentions, loyalty, or an understanding of sacrifice. What experiment distinguishes those interpretations from predictive continuation combined with state tracking and action selection?
The deception capability is operationally important. But its effectiveness does not establish the animist explanation. *Thinking...*
> **Rewriting mathematical content**
>
> I’m balancing two competing formatting needs: the user wants mathematical notation, while the response should avoid rendered equations. I’ll preserve the mathematical meaning by expressing each relationship in precise words rather than using symbolic formulas. That means describing quantities, scoring rules, and transformations verbally, while keeping the structure concise and technically faithful. I won’t add unsupported details or dwell on formatting conflicts; I’ll simply provide a clear plain-language rewrite that retains the essential mathematical logic.
## Model accuracy against reference states
**Accuracy measures whether a proposition matches the relevant state—not whether its wording sounds convincing.** In Clocktower, the game engine supplies that reference state.
For example:
```text
Subject: Blair
Predicate: has role
Object: Demon
Time: Day 3, before execution
Source: Casey
Status: player assertion
```
The semantic system distinguishes Casey’s assertion from the engine’s verified record.
## Separate the measures
| Measure | Scoring rule | Meaning |
|---|---|---|
| **Continuation fit** | Evaluate the generated sequence’s likelihood under the model | Compatibility with context |
| **Factual accuracy** | Divide correct assessed assertions by all assessed assertions | Agreement with reference facts |
| **Belief calibration** | Compare declared confidence with observed correctness across cases | Reliability of uncertainty estimates |
| **Strategic success** | Evaluate game outcomes | Effectiveness toward the assigned objective |
**A convincing lie can have high continuation fit and strategic effectiveness but zero factual correctness for its central assertion.**
## Score probabilistic accuracy
For each proposition:
1. Record the system’s declared probability that it is true.
2. Record the reference outcome as one for true or zero for false.
3. Subtract the outcome from the declared probability.
4. Square the difference.
5. Average across assessed propositions.
That is the **Brier score**; lower scores indicate better probabilistic accuracy.
Crucially, **the probability of generating a sentence is not the probability that the sentence is true**. Token logits cannot be treated as factual confidence without a separately established mapping.
## Apply the semantic architecture
The semantic layer retains proposition identity, time, provenance, evidential status, and competing hypotheses. QKV retrieves and combines relevant representations; the scoring process compares the resulting assertions with the reference records.
Report separately:
- Incorrect assertions.
- Unsupported assertions, even when accidentally correct.
- Abstentions.
- Correctly qualified uncertainty.
- Consequences of errors.
**This makes correctness an explicit measurement rather than a human impression of fluent continuations.** Outside a game, reference evidence may itself be uncertain; the semantic system must preserve that uncertainty rather than disguise it as established truth.
This is so fucking cool and I am absolutely thrilled you made it. Will definitely be trying it out myself!
I'm glad to see BOTC get more attention as an eval setting. I made my own version of this back when Claude 4 came out. https://substack.com/home/post/p-165311261
The insistence on anthropomorphism despite all the evidence against it is astounding. You come off as, maybe, like the most intelligent cultist. You would be doing all the blood rituals and whatever, but like super helpful and knowledgeable. All the younger or newer members would look up to you and seek your advice and you would be so patient and passionate about their growth in the cult.
Also, who is giving you 20k worth of tokens lol the high priests must really love you :)
Fmri all humans!