If I asked a PhD, “How many Bs are there in the word ‘blueberry’?” They’d call an ambulance for my obvious, severe concussion. They wouldn’t answer, “There are three Bs in the word blueberry! I know, it’s super tricky!”
LLMs are fundamentally unsuitable for character counting on account of how they ‘see’ the world - as a sequence of tokens, which can split words in non-intuitive ways.
Regular programs already excel at counting characters in words, and LLMs can be used to generate such programs with ease.
This is true. They do not think, because they are next token predictors, not brains.
Having this in mind, you can still harness a few usable properties from them. Nothing like the kind of hype the techbros and VCs imagine, but a few moderately beneficial use-cases exist.
Without a doubt. But PhD level thinking requires a kind of introspection that LLMs (currently) just don’t have. And the letter counting thing is a funny example of that inaccuracy
If I asked a PhD, “How many Bs are there in the word ‘blueberry’?” They’d call an ambulance for my obvious, severe concussion. They wouldn’t answer, “There are three Bs in the word blueberry! I know, it’s super tricky!”
I don’t feel this is a good example of why LLMs shouldn’t be treated like PhDs.
My first interactions with gpt5 have been pretty awful, and I’d treat it but it’s not available to me anymore
Do you smell toast?
BlackBerry toast
FWIW, ChatGPT 5 gets this correct
Fuckin’ does it?
It did for me 🤷🏻♂️
You appear to be using the older gpt model. The newer model calculates and answers correctly for most words at least for the few I asked
It literally says 5 in the screenshot but ok
I saw that. I’m using the mobile app. There’s a possibility the web version is using an inferior model.
LLMs are fundamentally unsuitable for character counting on account of how they ‘see’ the world - as a sequence of tokens, which can split words in non-intuitive ways.
Regular programs already excel at counting characters in words, and LLMs can be used to generate such programs with ease.
But they don’t recognize their inadequacies, instead spouting confident misinformation
This is true. They do not think, because they are next token predictors, not brains.
Having this in mind, you can still harness a few usable properties from them. Nothing like the kind of hype the techbros and VCs imagine, but a few moderately beneficial use-cases exist.
Without a doubt. But PhD level thinking requires a kind of introspection that LLMs (currently) just don’t have. And the letter counting thing is a funny example of that inaccuracy