Recent searches
Search options
Apple did the research; LLMs cannot do formal reasoning. Results change by as much as 10% if something as basic as the names change.
https://garymarcus.substack.com/p/llms-dont-do-formal-reasoning-and
@ShadowJonathan Why would we judge LLMs on their ability to solve complex tasks? The interesting thing is if they can solve simple tasks well enough to be useful.
@beeoproblem@mastodon.gamedev.place@anderspuck @ShadowJonathan because they're being sold as if they can solve complex tasks
LLMs can use a prompt to generate text based off of a huge pile of content produced by other people. Sometimes that text is an exact copy of the original text. They may "solve" a problem if the solution is contained in their training data and your prompt is able to retrieve it.
They're a (very) improved version of a Markov chain. Not a problem solver of any sort