That’s because they don’t see the letters, but tokens instead. A token can be one letter, but is usually bigger. So what the llm sees might be something like
st
raw
be
r
r
y
When seeing it like that it’s more obvious why the llm’s are struggling with it
What’s the strawberry problem? Does it think it’s a berry? I wonder why
Ask an LLM how many Rs there are in strawberry
not a problem limited to llms, they perfectly replicate my stupidity ;)
For reference Bing chat is still confidently sure there are 2
I think the strawberry problem is to ask it how many R’s are in strawberry. Current AI gets it wrong almost every time.
That’s because they don’t see the letters, but tokens instead. A token can be one letter, but is usually bigger. So what the llm sees might be something like
When seeing it like that it’s more obvious why the llm’s are struggling with it