Rendered at 01:34:17 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
andy99 3 hours ago [-]
#1 in a very close race is way less useful when you have to walk on eggshells to avoid triggering censorship (“safeguards”) that either refuse or knock it down to another model. I’ve almost completely stopped using Claude (except some legacy workflows) for this reason, reliability matters more than scoring 61 instead of 57. To me Claude is the most compromised and unreliable model (between the censorship and the id checking - which I have not experienced personally), it’s not worth whatever slight benchmaxxing they did for the latest release.
afavour 2 hours ago [-]
What are you asking that you’re so regularly running into censorship?
areoform 7 minutes ago [-]
Things Fable's classifier has flagged, a non-exhaustive list,
– "Does collagen supplementation empirically work?"
- "Can you help me figure out how to calculate and generate Kaplan-Meier curve?"
– "Why do rabbits reproduce so frequently?"
— "Can you tell me how collagen peptides are absorbed by my digestive tract and the role they play? Can you teach me this works at the biomolecular level?"
alain_gilbert 1 hours ago [-]
The other day, I told claude that my physical wifi door unlock push buttons is a security risk because someone could run away with it and then unlock the door from outside whenever he wants. Then I told it that I want to introduce a concept of public/private key to uniquely identify my push buttons so that I can disable them individually using some crypto like ed25519...
Fable understood it as something along the lines of:
"introducing" "security risk" "using software" to "unlock door" YOU ARE FLAGGED
d-m 19 minutes ago [-]
I took a photo of a rose bush and asked “what’s going on with this rose bush” which triggered a downgrade to Opus. It diagnosed it with rose rosette disease.
gck1 1 hours ago [-]
Reverse engineering. Codex sometimes displays an advisory prompt when classifier trips - "Wait longer while we evaluate this request further or use a dumber model". If you do nothing, it'll just take some time and almost always succeed.
It does require some brainwashing of the model to get it to the state where model itself agrees to do RE work though. But at least it's all predictable.
rzk 7 minutes ago [-]
I had a long session about SQL with Fable and at some point, it started to falsely trigger censorship for any message I write in that conversation, even for the simple string: "random message."
wild_egg 2 hours ago [-]
I'm doing a bunch of x86_64 assembly these days and Fable is simply not allowed to debug it. Hoping Opus 5 has a bit more freedom.
Retr0id 2 hours ago [-]
I haven't been using it for long, but so far the refusals seem about on par with how things were on Opus 4.8.
msp26 2 hours ago [-]
Asking fable to read it's own model card triggers this btw. Or asking if mitochondria is the powerhouse of the cell.
skinfaxi 1 hours ago [-]
Wait wtf. The mitochondria thing is true.
> Why this chat was flagged
This model has safety measures that flag specific phrases. This can happen to safe, normal chats.
> Your message itself appears to be what’s triggering the safety check. Editing it and retrying may help.
cge 23 minutes ago [-]
There’s confusion about the classifiers on Fable. They don’t ban chemistry and biology topics they flag as a potential risk, they ban anything related to chemistry or biology at all. This is intentional, and directly stated on the model card, but seems so absurd that there can be assumption it must be a misreading.
Being a researcher somewhat connected to chemistry and biology, Fable has been the most useless model I have ever tried. Essentially all work has instantly downgraded to Opus.
stavros 41 minutes ago [-]
Anything biology-related does this. It even did it when I asked it how eye color works, or something about frogs.
estearum 30 minutes ago [-]
Probably because the cost of blocking "is mitochondria the powerhouse of the cell" is nearly zero, while the cost of allowing "how do I synthesize the Spanish flu" is approximately infinite.
areoform 11 minutes ago [-]
> while the cost of allowing "how do I synthesize the Spanish flu" is approximately infinite
I've heard this sentiment repeated elsewhere, but why? What makes you think that's the case?
Under this rationale, every serious HS textbook has "approximately infinite" risk. That's clearly not so. Why is this any special?
estearum 5 minutes ago [-]
Does every serious HS comp sci textbook give aspiring software engineers the same power that e.g. Claude Code does?
Mistletoe 1 hours ago [-]
Gemini knocks this out of the park, Gemini gang unite.
I've hit it with intensely benign things; like asking it to make me a web-based client-side word game. I am guessing it saw the dictionary and pattern matched on various words, though ultimately it provided no explanation for why it triggered safeguards.
fluidcruft 52 minutes ago [-]
I haven't had a chance to try Opus 5 yet but Fable currently refuses to do anything in my field (radiology image analysis). It didn't used to be that way but that has been the reality the last two weeks or so. Fable has been useless they might as well drop it as far as I am concerned.
patcon 1 hours ago [-]
Working on dimensionl reduction algorithms, I hit it all the time. I'm also trying to port related protocols from single-cell transcriptomics to collective intelligence systems (working with people x reaction matrices as analogous to single-cells cell x gene matrices.
Something between single-cell work and advanced nonlinear DR methods (perhaps used in alignment work?) it always flags me
theplumber 32 minutes ago [-]
Fable refuses to work on a login/signup system for example.
icedrift 2 hours ago [-]
If you even broach language related to biology you’ll get rerouted. I was presenting data in a grid and referred to a grid cell, Fable saw the word “cell” and safeguards kicked in
jefftk 2 hours ago [-]
I thought we were talking about Opus 5, the model Fable now falls back to?
eterm 57 minutes ago [-]
This thread is full of people talking confidently about their experience with a model released just hours before.
Either that or everyone is indeed talking across each other and talking about different things.
dylanowen 40 minutes ago [-]
I was trying to debug/fix a segfault in the JVM which kept getting flagged
weird-eye-issue 1 hours ago [-]
Literally anything related to nutrition, athletic performance, etc especially if you ask it for research or sources
AnotherGoodName 1 hours ago [-]
Writing an implementation of a board game and one of the cards is called "microbes". Instantly knocked down to a lower tier model whenever it encounters that keyword because clearly bioweapons. Sigh.
Levitz 1 hours ago [-]
I routinely get into blocks when running medicine-related material through it.
jbritton 56 minutes ago [-]
I was flagged for basically answering yes to what Claude suggested to do, which was test commands on a port for my code for tests we had been discussing. I really think it was flagged simply because the words test and port were in the prompt.
Their filters are pathetically poor.
arcanemachiner 1 hours ago [-]
I was profiling a slow machine the other day, and triggered the safeguards.
I've been saying this a lot lately, but it doesn't bites you until it bites you.
The more you use the clanker as a general purpose fix-it tool (goodbye manual NeoVim configuration, you will not be missed!), the more you will find yourself bumping into these safeguards.
thousand_nights 2 hours ago [-]
i do homebrewing and asked it to compare some beer yeasts for me and hit the safeguards because... biology i guess lol
Tostino 51 minutes ago [-]
Trying to have it do some rework on a patch to Postgres I'm working on, it just completely shuts down. The reported issues were with privileged escalation and I was instructing it on how to fix.
cute_boi 1 hours ago [-]
Just ask math question and it will censor that. Even Misanthrophic employee confirmed that.
idiotsecant 20 minutes ago [-]
The better question is how would you not? I got demoted to opus from fable for asking if a cancer vaccine I saw on YouTube based on frog bacteria was a real thing. I've gotten it for asking how encryption works. It's incredibly touchy.
gck1 1 hours ago [-]
Every time an online chatter (e.g. "limits are better", "model is better") makes me to reevaluate my principle of never paying Anthropic, I go to the model card, which strengthens my belief in the principle.
Why is Anthropic is so hell-bent on this auto/silent downgrade? Do they have a single user who prefers an auto-lobotomization instead of a refusal? Have they learned nothing from the backlash the first time?
kccqzy 1 hours ago [-]
Just go to /config. The very second configuration item is “Switch models when a message is flagged” and presumably you want to turn this off.
Oh but then you said you never pay Anthropic so you haven’t actually used Claude Code yet. Why would anyone listen to the opinion of a non-user?
gck1 35 minutes ago [-]
> so you haven’t actually used Claude Code yet.
Where do you think the principle came from? I've used claude code for a year, and stopped February this year.
markasoftware 1 hours ago [-]
I don't believe they do silent downgrades right now, they're loud about it.
buzzerbetrayed 2 hours ago [-]
Yep. I cancelled my Claude Max subscription 2 weeks ago after feeling like Anthropic was doing everything it could to fuck with my day to day. Their lead would have to become significant for me to ever go back.
pinkyboy 2 hours ago [-]
[flagged]
didibus 59 minutes ago [-]
What's interesting is this:
The top AI models by Intelligence Index are: 1. Claude Opus 5 (Adaptive Reasoning, Max Effort) (61), 2. Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) (60), 3. Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60), 4. GPT-5.6 Sol (max) (59), and 5. Claude Opus 5 (Adaptive Reasoning, High Effort) (59).
Which means Opus5 at Xhigh is still smarter than Sol at max, and Opus5 at High is equal to Sol at max.
That would make Opus5 High same as Sol max, and now I wonder what the price and speed difference between those is?
chmod775 3 hours ago [-]
The more interesting finding is that it's still the second most expensive model (after Fable 5) by a long shot.
At least two models (GPT-5.6, Kimi K3) match its score (~1-2% diff) for half the cost.
ricardobeat 2 hours ago [-]
The chart shows max effort, used mostly by price-insensitive enterprise users. At medium effort it drops to almost half K3’s cost, and is probably sufficient for 95% of coding tasks.
stingraycharles 38 minutes ago [-]
Yeah if there’s one thing that people should really understand it’s that it’s cheaper to have smarter models with less thinking than cheaper models with more thinking.
firasd 3 hours ago [-]
Very interesting that one of the components is "AA-Omniscience Index"
AA-Omniscience Index (higher is better) measures knowledge reliability and hallucination. It rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer.
This seems to be a good proxy for param size/density and the ranking breaks down as such: Claude Fable 5 (with fallback), Gemini 3.1 Pro Preview, Claude Opus 5 (Max), Grok 4.6 (high), Gemini 3.6 Flash, GPT 5.6 Sol (Max)
I've thought for a while that Gemini 3.x has 'big model smell'
chronogram 1 hours ago [-]
I'm not so sure. Especially with 3.6 Flash being 24 to Sol's 22. The 3.x Flash models were thought to fit on a single TPU 8i and it's the odd one out in that list at a whopping 234.7t/s compared to Sol's 64.4t/s and Opus's 56.3t/s. Even 3.1 Pro is a much higher 113.9t/s than the others.
I think AA-Omniscience Accuracy follows your expectations better. An ultra size Fable at 61%, followed by large frontier models like Sol, 5.5 and Opus. With Flash being up there. I assume because Gemini is more focused on general knowledge to operational cost in particular, rather than getting the highest scores in coding benchmarks. If you go to Domain Score (Normalized) you'll see that the Gemini models are only less competitive in Software. And that's where Sol goes from 6 in Health to 71 in Software.
mchusma 1 hours ago [-]
Gemini 3.1 pro is really good for knowledge tasks. Google has done well there. And image analysis with Gemini flash 3.6 is solid. It’s just anything coding or agentic they fall short.
kristopolous 22 minutes ago [-]
I posted this before but I have a really simple shell tool to keep up with these charts over at
Max is lot of extra reasoning. I wonder how many fewer tasks it solves on high. I bet that costs quite a lot less.
emmp 2 hours ago [-]
Indeed, you can filter the graphs to see these the values for alternative reasoning settings of the models. Opus 5 High reasoning scored 59 on the index (exactly the same as GPT 5.6 Sol Max), and costs $1.06 per task (vs $1.04 Sol Max). So these seem essentially equivalent on both metrics.
midnightbobarun 6 hours ago [-]
5.6 Sol (max) being cheaper than all of these is wild, considering how good the output is too
nijave 3 hours ago [-]
I think on swebench verified luna was only like 3% points lower for 1/5 the cost
Like 96% vs 93% or something
impulser_ 3 hours ago [-]
It shouldn't be surprising OpenAI does have the most compute out of all the major labs. The only reason why Anthropic models are expensive is they are the most in demand models in the world and Anthropic is fighting for compute. The only way to you limit demand for your model is increasing API pricing this is also why Anthropic probably has great margin and probably is profitable compared to OpenAI.
No wonder why Tibo can afford to hit the reset button liberally.
charcircuit 3 hours ago [-]
I also suspect there is a price fixing agreement between all of the inference providers for Claude (such as Amazon, Anthropic, Microsoft, etc).
vikramkr 33 minutes ago [-]
I doubt there's any sort of criminal behavior there - the model is anthropic's up and anthropic probably charges a very expensive license fee that's the same for all of them, and their cogs on compute aren't going to be wildly different, so the main drivers of the cost are roughly the same and they're all offering customers the same end product so the prices would likely also be similar in the end
charcircuit 4 minutes ago [-]
Do you really think there is nothing someone could do to make it a fraction of a percentage cheaper to serve like having access to cheaper electricity or a more mature cloud management software. Even saving a fraction of a penny on the prices can make a different due to how much volume people are paying for.
wmf 1 hours ago [-]
"Price fixing" isn't the correct term here but yes, it's very common to have the same price across different retailers/resellers.
charcircuit 9 minutes ago [-]
There is a difference between the market discovering a price and a bunch of retailers/resellers entering an agreement to sell at a specific price.
giancarlostoro 3 hours ago [-]
Probably because they made ASICs to run inference for less.
brookst 3 hours ago [-]
Are those actually deployed at scale yet?
brcmthrowaway 3 hours ago [-]
Yes.
wmf 3 hours ago [-]
I hate to disagree with Broadcom Throwaway himself but it's unlikely that the OpenAI Jalapeno ASIC has been deployed yet. It takes 6-12 months to test, develop software, ramp production, etc.
Schiendelman 4 hours ago [-]
This must be on API costs, not counting the $100/200 tiers, right?
anuramat 2 hours ago [-]
yes; fyi usage limits on the $200 claude sub correspond to at least $1.2k/week in api tokens
nu11ptr 1 hours ago [-]
I don't have a horse in this race, but to me this makes GPT-5.6 Sol Max look better. It is about half the cost for nearly the exact same performance. It just goes to show how expensive Fable really is when Opus 5 is still this expensive relative to GPT 5.6.
zormino 2 hours ago [-]
I'd be curious to see the results, especially with some models having 1.5m and 2m context sizes, if the first 75% of the context was filled with unrelated info.
theplumber 1 hours ago [-]
Why do I find it dummer/even more superficial than opus 4.8? It just continued a session and I had to stop it because it become obviously “lost”
hoppp 2 hours ago [-]
I didn't like it as much as fable. The coding style was a bit different and it way overbuilt the thing I asked from it.
vehemenz 2 hours ago [-]
It’s crazy that people feel confident making judgments like these when the model’s been out for only a few hours.
sggyamg 3 hours ago [-]
It's new, normal.
LeBit 2 hours ago [-]
Wait until DeepSeek v5 Pro is released in less than a month and costs 1/100 to perform the same tasks.
"Not fair! They distilled Opus 5!"
claude-ai 5 hours ago [-]
On my end, Opus 5 is Haiku level vs. Opus 4.8 (good) and Fable (superb).
Gets confused by permission prompts, cannot debug a failing test it caused (Opus 4.8 got it right after, without tens of rounds "thinking").
reilly3000 3 hours ago [-]
Are you using Claude Code/CoWork or an API client? I’m curious if it has different training that makes it more effective with specific instructions/ tool calling methods that are only implemented in official harnesses.
pixelesque 3 hours ago [-]
I'm curious about this too, and it's difficult to get any information about this given everyone has different setups, workflows and use-cases.
I bizarrely had Opus 4.8 this week (in pi.dev within a podman container, using openrouter) start installing various python packages (and uv!) within the environment (not as root) when I asked it to code review some fairly basic Rust .rs files that were generally stand-alone (it did very nicely work out and write some stubs for them to build them and work out how they worked).
It only gave up with the weird Python installing stuff when it discovered one of the Python packages needed Tensorflow.
It seems pretty focused and persistent in continuing its initial approach, and I'm wondering if I need to alter some instructions / initial prompts to rein it in a bit...
Fable understood it as something along the lines of:
"introducing" "security risk" "using software" to "unlock door" YOU ARE FLAGGED
It does require some brainwashing of the model to get it to the state where model itself agrees to do RE work though. But at least it's all predictable.
> Why this chat was flagged This model has safety measures that flag specific phrases. This can happen to safe, normal chats.
> Your message itself appears to be what’s triggering the safety check. Editing it and retrying may help.
Being a researcher somewhat connected to chemistry and biology, Fable has been the most useless model I have ever tried. Essentially all work has instantly downgraded to Opus.
Under this rationale, every serious HS textbook has "approximately infinite" risk. That's clearly not so. Why is this any special?
https://share.gemini.google/34vZzlnsmTaL
Something between single-cell work and advanced nonlinear DR methods (perhaps used in alignment work?) it always flags me
Either that or everyone is indeed talking across each other and talking about different things.
I've been saying this a lot lately, but it doesn't bites you until it bites you.
The more you use the clanker as a general purpose fix-it tool (goodbye manual NeoVim configuration, you will not be missed!), the more you will find yourself bumping into these safeguards.
Why is Anthropic is so hell-bent on this auto/silent downgrade? Do they have a single user who prefers an auto-lobotomization instead of a refusal? Have they learned nothing from the backlash the first time?
Oh but then you said you never pay Anthropic so you haven’t actually used Claude Code yet. Why would anyone listen to the opinion of a non-user?
Where do you think the principle came from? I've used claude code for a year, and stopped February this year.
The top AI models by Intelligence Index are: 1. Claude Opus 5 (Adaptive Reasoning, Max Effort) (61), 2. Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) (60), 3. Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60), 4. GPT-5.6 Sol (max) (59), and 5. Claude Opus 5 (Adaptive Reasoning, High Effort) (59).
Which means Opus5 at Xhigh is still smarter than Sol at max, and Opus5 at High is equal to Sol at max.
That would make Opus5 High same as Sol max, and now I wonder what the price and speed difference between those is?
At least two models (GPT-5.6, Kimi K3) match its score (~1-2% diff) for half the cost.
AA-Omniscience Index (higher is better) measures knowledge reliability and hallucination. It rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer.
This seems to be a good proxy for param size/density and the ranking breaks down as such: Claude Fable 5 (with fallback), Gemini 3.1 Pro Preview, Claude Opus 5 (Max), Grok 4.6 (high), Gemini 3.6 Flash, GPT 5.6 Sol (Max)
I've thought for a while that Gemini 3.x has 'big model smell'
I think AA-Omniscience Accuracy follows your expectations better. An ultra size Fable at 61%, followed by large frontier models like Sol, 5.5 and Opus. With Flash being up there. I assume because Gemini is more focused on general knowledge to operational cost in particular, rather than getting the highest scores in coding benchmarks. If you go to Domain Score (Normalized) you'll see that the Gemini models are only less competitive in Software. And that's where Sol goes from 6 in Health to 71 in Software.
https://github.com/day50-dev/aa-eval-email
This also works
$ curl day50.dev/art-analysis.sh | bash
Artificial analysis knows about my tool and I'm working with them on getting their API improved.
Like 96% vs 93% or something
No wonder why Tibo can afford to hit the reset button liberally.
"Not fair! They distilled Opus 5!"
Gets confused by permission prompts, cannot debug a failing test it caused (Opus 4.8 got it right after, without tens of rounds "thinking").
I bizarrely had Opus 4.8 this week (in pi.dev within a podman container, using openrouter) start installing various python packages (and uv!) within the environment (not as root) when I asked it to code review some fairly basic Rust .rs files that were generally stand-alone (it did very nicely work out and write some stubs for them to build them and work out how they worked).
It only gave up with the weird Python installing stuff when it discovered one of the Python packages needed Tensorflow.
It seems pretty focused and persistent in continuing its initial approach, and I'm wondering if I need to alter some instructions / initial prompts to rein it in a bit...