LLMs, Requirements, and Results
How things stay the same as LLMs change everything
You’ve heard it already, AI is changing everything and anybody can write software. The barrier to entry is gone. This is both true and false– it depends what you mean. Almost as if strict binary thinking is risky, but I digress.
I’ve been writing software since 1989, and began using LLMs for software in the summer of 2023. Here are some reflections.
Let’s start with the truth. Software advances have always been about simplifying, componentizing, commoditizing. Building-sized computers now fit into a watch… and much smaller. Gone¹ are the days of assembly & low-level languages. While major, the LLM shift follows a long tradition of ‘compiling’ higher level languages into algorithm instructions.
Writing software is like a function,
There are aesthetics to the source code itself, just like the words we speak and write. But for this piece let’s just be pragmatic,
Watching LLMs transform words into code underlines a shared life experience working with computers: so much of coding truly is repetitive and predictable. After all, I just need…
to sum each group of numbers
to save data and display it later
to vertically align this web page element
Will humanity ever truly solve vertical alignment? Anyway, the most complex ideas are ‘simply’ compositions of smaller ideas. Many problems and their solutions are so common that the predicted code is indeed likely to be adequate.
So with LLMs, gone¹ are the days of hand-crafting yet another REST endpoint, yet another data entry confirmation modal, yet another nested for loop.
Still, the more one talks with computers, the more one inhabits the implicit gap between specification and result. Source code doesn’t fill in the blanks: it’s delightfully (and painfully) literal. The LLM fills in the blanks by design, chasing the most likely result. Yet nothing in life is free: this blank canvas wizardry also brings delight (and pain).
Unlike people, an LLM doesn’t process broader or deeper meaning, nor care that such meaning exists. The LLM doesn’t even have ‘cares’ per se, au contraire: its duty is to exercise the probability engine for a result.
And this brings us to what’s not true. The challenge has always been, what exactly do we mean? And do we mean the right thing? Consider again,
Those symbols do some real heavy lifting. One way or another, everything must be specified. If we don’t know what potential problems are, how do we know we specified correctly? Filling in the details is hard.
Specification and correctness aside, the computer will happily follow an inefficient algorithm for minutes or days. Or multiply our infrastructure costs by spinning up 10x nodes just like the worker manifest said.
Even if the days of writing source code are gone¹, we haven’t yet escaped the consequences of decisions we make, or decisions we skip (knowingly or not).
To reconcile the true and the false, let’s reduce writing software to a binary approximation:
the conversion of thought into computer processing, then observing results & iterating. Success occurs when the process produces a result matching expectations.
the engineering of correctness, efficiency, and operability. Success occurs when we can predict & manage system behavior.
So yes, LLMs have changed everything. It’s easier than ever to translate our thoughts into machine instructions. That doesn’t help to know if what we say is what we mean, or if we even know what we mean (or want). Those details matter depending on the goal, and proportionally to the scale and real-world stakes.
In that sense, LLMs haven’t changed anything. We’re still people using computers to shape digital realities. We still navigate the gap between known-knowns and unknown-unknowns. We still succeed when we understand & meet our needs… and we still suffer when we don’t.
The work has always been closing that gap of requirements and results. Gone¹ are the days of learning coding symbols to do so.
[1] Nothing is truly gone: when we seek to stretch the limits, we may find everything is present.
About Binary Alchemy
Binary Alchemy is the art of software engineering: part programming, part philosophy. Software melds the elegance of mathematics with the unyielding realities of zeroes and ones. Software is ethereal, less constrained than materials in its ability to metamorphose (or metastasize).
I wrote my first line of code in 1989. I have decades of experience building, operating, and re-building software spanning from MS-DOS to GCP, AWS, and Azure. ️⚙️ I’ve worked with startups and large enterprises, including Google, Microsoft, Square aka Block, Gusto, Mayo Clinic, and others.
I provide software & cloud architecture consulting services. Learn more: RedwoodConsulting.io