@Varpie

🪨 @Varpie@peculiar.florist

6mo

@resuna @petealexharris @munin Depends what you mean by "meaningful or misleading". Does the LLM understand what it means? No, it doesn't, it's a machine. But can you find a meaning to the output of that question and can it be helpful? Maybe. Does the LLM try to trick you into thinking that it is actually reasoning? Not consciously, because it doesn't have a conscience, it's a machine. But researchers have found that mimicking human reasoning and adding extra steps before giving the final output can improve results, so now we have "thinking" or "reasoning" models. Is that a loaded term that AI companies use to mislead users into thinking that LLMs can do actual reasoning? Probably. But was that the intention of the researchers that developed these techniques? I don't think so... but maybe?

1

0

🪨 @Varpie@peculiar.florist

6mo

@resuna @petealexharris @munin Have you ever heard of an example?

1

0

🪨 @Varpie@peculiar.florist

6mo

@resuna @petealexharris @munin Ok, this is going nowhere. You're wrong, "why did you do X" can provide more context and can be useful to then tweak the prompts and get better results, and saying that everyone asking that question to an LLM thinks that the LLM is capable of human reasoning is a very ignorant thing to say, and if you really want to continue defending that point, even given where we're at in the discussion, you're a lost cause. Or maybe I'm actually talking to an LLM, that would make sense since you're not capable of learning from your mistakes...

1

0

🪨 @Varpie@peculiar.florist

6mo

@resuna @petealexharris @munin Here are some articles explaining what "reasoning models" do, because clearly you need some education:
magazine.sebastianraschka.com/i/156484949/how-do-we-define-reasoning-model
www.ibm.com/think/topics/reasoning-model
newsletter.maartengrootendorst.com/i/153314921/what-are-reasoning-llms

I could post a lot more examples, but the TLDR (because I know you won't read them): "reasoning" models add intermediate "reasoning" steps that are just made to mimick human reasoning given the context, and that's the part we don't see ("under the hood") when AI tools spin (that and tool calling, which is another kind of training modern models have to return structured responses executing function calls).

1

0

🪨 @Varpie@peculiar.florist

6mo

@resuna @petealexharris @munin Yes it is. It's literally how it works. Just try whatever open weight small LLM model with "thinking" or "reasoning" or whatever they market it as, and try for yourself using Ollama or whatever tool that actually shows the full context and not just a spinner with "Thinking... Combobulating... Crafting...". "Thinking" "agentic" AI tools / models just add extra steps trained to simulate human reasoning, and the example I gave is actually fairly accurate to what you could see under the hood of an AI tool like Claude Code.

1

0

🪨 @Varpie@peculiar.florist

6mo

@resuna @petealexharris @munin Alright, let's give a more detailed example since clearly you don't get my point.

User: create a new table for students
(AI tool adds parts of existing database schema and code as context)
LLM "reasoning", not shown to the user but still part of the context because that's how "thinking" agents work: Ok, the user asked me to create a table for students. According to students.ts, the students contain a name, and a classroom. Ah, but according to the schema I received, the classroom table does not exist. Let me start by creating that.
(LLM sends request to create table to the AI tool, which fails because the table already exists)
LLM "reasoning", still not shown to the user but still relevant: Ah, I received an error. Ok, let's see... the table does not exist in the schema, so it is probably part of an old project and is no longer relevant. I should delete it and create the table again, so that we do not have irrelevant information in the database.
(LLM sends request to drop tables, the AI tool executes, user panics)
LLM answer: Ok, I have created the students table and the related classroom table.
User: You deleted my classrooms table... Why did you do that?

That would be the full context of the discussion, not just the last message. And the answer is likely to come from those "reasoning" steps that are hidden to the user. Which makes it a relevant question to ask.

1

0

🪨 @Varpie@peculiar.florist

6mo

@resuna @petealexharris @munin What happens if you ask an LLM to summarize a text into 4 bullet points, then in the next prompt ask it: "Remove the 2nd point"?
What happens if you ask an LLM to translate something, then ask it: "Do it again in [a different language]"?

Taken out of context, those questions are impossible to answer, so according to you, it will just give nothing relevant. But it doesn't, because every time you ask a follow-up question, it includes the context from the discussion. Which is what makes simple questions like "Why did you do that?" tasks that give statistically relevant output, not "fanfic about itself".

1

0

🪨 @Varpie@peculiar.florist

6mo

@resuna @petealexharris @munin You're assuming that there is no other context provided with the question, and that the training does not take into account that context. If I had to train for this specific question, I'd make sure to score positively answers that are relevant to the previous context. Which is what happens, and why it is a valid question to ask your LLM if you want some insight into the context that isn't shown in the UI but still in the discussion.

1

0

🪨 @Varpie@peculiar.florist

6mo

@munin @petealexharris Sure, I'll go touch some grass and talk to my therapist about this philosophical horseshit :meow_ok_fine:

0

🪨 @Varpie@peculiar.florist

6mo

@arstechnica Well... a 71.5% increase in price for the 50% increase in memory isn't quite "nearly double". But $500 for a 4GB upgrade is still absolutely insane.

0

🪨 @Varpie@peculiar.florist

6mo

@petealexharris I totally agree with you. And that is also a very different take from the beginning of the discussion, where Fi said that querying LLMs for "why" it does something is "thrice-divorced from reality" and "fucking delusional" and that people doing that should "touch some grass and get a fucking therapist"...

⭐1

1

0

1

🪨 @Varpie@peculiar.florist

6mo

@petealexharris @munin You misread me. Whether the model "understands" the question is a philosophical question. The non-philosophical question of whether it can give a useful answer is the relevant part, and my whole point is that pointing at the philosophical aspect to belittle people that look at the practical part, assuming that they don't understand it, is dumb.

2

0

🪨 @Varpie@peculiar.florist

6mo

@EUCommission That's great! Now please make sure to also tell them no to make "side-loading" (the loaded term for installing without the proprietary store) a nightmare, because without that, they can still just decide which AI providers (or anything else...) are available to install, which would make this measure useless.

⭐1

0

1

🪨 @Varpie@peculiar.florist

6mo

@petealexharris @munin "Why" is definitely a word from the training data, and "why did you do that?" is definitely also part of things asked a lot, that OpenAI and others have trained on, so my point still stands that it is a valid question to ask. Whether the model "understands" the question is just a philosophical question that is irrelevant for the fact that it is a useful question. Of course if you're using it in Prod and it deletes your DB and you think it understands and can improve itself, there are plenty of things you'd need to be corrected on, but saying that everyone asking that question is delusional is just wrong.

2

0

🪨 @Varpie@peculiar.florist

6mo

@petealexharris @munin When you ask an LLM "why is the sky blue?", it is statistically likely to give a correct answer. It still works the same way, computing probabilities of what the next token is, but the "why" has a semantically significant weight that influences the output, so it is an important keyword. It doesn't have to "understand" it, it just has to be trained in a way that makes it significant. You don't have to believe that it understand things to know that it is trained on human language and will behave correctly when fed human language.

1

0

🪨 @Varpie@peculiar.florist

6mo

@petealexharris @munin Where did I say they'd understand a better prompt?

1

0

🪨 @Varpie@peculiar.florist

6mo

@petealexharris @munin Clearly you've never tried it yourself...

2

0

🪨 @Varpie@peculiar.florist

6mo

@munin Well... As you mentioned, each generation "reads the prompt and any cache, if they exist, from prior session", and since they were trained on "explaining" their previous outputs to sound like a relevant discussion, asking why a model gave a specific output isn't as stupid as you make it out to be, as it can give some input on the "thinking" part of the previous output that is usually not directly visible. That can then be used to tweak prompts and add some guardrails (even though there is a fairly long list of examples of guardrails not being fully effective). Of course, the first problem is giving access to prod to an unreliable system...

⭐1

2

0

1

🪨 @Varpie@peculiar.florist

6mo

@SteamDB Holy shit finally the wait is over (at least partially)

0

🪨 @Varpie@peculiar.florist

6mo

@bagder Surely, that projection isn't correct because there is a big batch of things being found now and it will dwindle down after that... Can we really expect multiple breakthroughs in tooling and breakthroughs in classes of vulnerabities this year?

⭐2

1

0

2