It's not just llms though. Any form of machine learning or reinforcement learning typically turns into these long mathy sequences on large matricies. AI output is often the result of matrix eigenvalues or determinates or some such. LLMs were (initially) really surprisingly simple, preferring simpler expressions on much larger datasets.
And this doesn't begin to describe formal AI systems like SAT and SMT solvers like prolog that are a really intense study of "if-else" statements.
I suppose expert systems could reasonably be described this way but those are before my time and I've been in ai since 2013.
So then are you calling pathfinding AI? Are you saying you can’t make pathfinding with AI? My game’s AI uses a glorified Dijkstra’s algorithm, are you now calling it an AI or still not? If not, why did you use it as an argument?
I think it's more than "bundles of if else statements". You're pushing to an open set, counting the cost of each step, that sort of thing. Also, like you said for loops, though I can see why someone might count that as a kind of if statement.
There was this odd period of time growing up when I used labview, knew how to use if statements, but didn't know how to use data structures. And, believe it or not, I did try to make a path finding algorithm. Every deviation it could make from "go in a straight line to the exit" had to be its own code path, and a different code path for every re-route wasn't feasible, so I just ended up with something that could only pathfind around very simple obstructions. Basically, if "back up and go around" couldn't get the pathfinder unstuck, they were never going to be unstuck.
Even then AI was much more then just gigantic if else monster... Learning classifier systems for example are not if else monsters. Even a Boyer–Moore theorem prover ain't a if else monster...
Tho Cyc is pretty infamous for being one... And that is why it failed.
Edit:
Tho to some degree it could be said about any non adaptive physical Symbol system like ACR-T or SOAR that those have pretty much the same problem: their ontologirs are human defined.
However this is a known fundamental yet to be solved design flaw in the realm of symbolic AI: the knowledge acquisition bottleneck. Aka needing a Human to translate expert knowledge through interviews into Prolog or lisp structures. But what if there could be a way to automate that?
Actually, the simplest form of a perceptron neuron activation, the Heaviside step function, is essentially a binary if-else statement. The reason we moved away from that if-else structure is for trainability. Because the step function is discontinuous, it has no meaningful derivative, which makes it impossible to use gradient-based optimization like backpropagation.
This is not entirely true, e.g. ReLU also has no derivative at 0, and is widely used with backpropagation. In practice, we often just set ReLU'(0) = 0 and carry on.
ReLU-like functions that are actually differentiable everywhere (e.g GELU, SwiGLU etc) became popular only fairly recently (and honestly for reasons other than differentiability, see https://arxiv.org/pdf/1606.08415).
Well that depends on how you see it. This a methaphor we are talking about. In its simplest form both are conditional based outcomes. Be it binary, a gradient, or even multi dimensional.
if-else's are all conditionals, but not all conditionals are if-else's
And i can't imagine a gradient activation function being described as "behaves like if-else's" since they have so little in common. Though I suppose that is a matter of opinion.
49
u/SKRyanrr 14d ago
Tell me you know nothing about LLMs without telling me