Outside-In and Inside-Out: Imperative, Inductive, and Generative Computing

Back to Basics: Imperative, Inductive and Generative Computing

What is “generative computing,” and is it different enough from other things to deserve a name? A reasonable place to start in defining generative computing is to contrast it to other kinds of computing. The everyday computing that we know and love arguably can be labeled as “imperative” computing¹. In imperative computing, you write a program composed of well-defined instructions (code) that tell the computer what to do, which is usually some variant of transforming input data into desired outputs. When you visit your bank’s website, a program retrieves data about your account from a database, and it systematically performs operations that calculate your balances and formats the results into HTML that renders in your web browser. Our digital world today is built on a foundation of imperative computing.

Imperative computing is great, because the computer does exactly what you tell it to. However, imperative computing is also terrible, because the computer does exactly what you tell it to. Getting code right is a famously hard problem², and enormous amounts of human energy are spent filing down sharp edges and rooting out unanticipated corner cases in software. While generative AI is already helping us write imperative programs³, we’re still very much in the mode where humans need to intervene and make sure everything makes sense, and there’s plenty of evidence that “vibe coding” introduces even more bugs than humans introduce on their own, at least for now. Using LLMs to write and fix code is interesting and powerful, but that’s not what we’re interested in exploring here.

Is there an alternative to imperative computing? AI models represent a different way of getting from inputs to outputs. Rather than telling the computer what operations to do, we instead collect up examples (of inputs and/or matching outputs), and we learn what needs to happen in between. We could reasonably call this style of computing “inductive computing,” since we’re inducing the operations from data, rather than defining them. This is useful, because there are plenty of things we’d like to do, but where we don’t know how to write down the steps to get from input to output in the usual imperative computing mode. For instance, humanity has tried and failed to write traditional imperative programs to accurately label the content of images, or that tell a positive product review from a negative one. However, we do know how to do this in inductive mode—we curate a bunch of examples, we label them, and we train a model (be it a deep learning model or any other machine learning model). The first wave of success in deep learning (in the pre-LLM era) cemented a place for inductive computing in our digital landscape. Inductive computing isn’t a silver bullet; it can be difficult to acquire and curate large enough sets of inputs and outputs that “cover” the range of inputs that we’d like to support, and training models is often equal parts art and science. There are many applications where the expense of curating and debugging a dataset isn’t worth the value that that one, use-case-specific model would yield. That said, inductive computing is a life saver in many domains, allowing computers to do things that we genuinely have no other way of doing.

So, effectively all of data-driven AI could be cast as inductive computing. So why do we need to invent another term, “generative computing?” LLMs certainly fit the definition of inductive computing, but we would argue they go a step further that warrants a new name. LLMs represent a consequential leap in the expressivity of inductive computing. LLMs don’t simply learn, say, a mapping from unstructured data to labels. Rather, they seem to be able to learn to perform sequences of actions, follow instructions, and even do arithmetic (albeit, usually poorly). This feeling that LLMs can be more “program-like” is palpable, though still not fully understood. Computational complexity analysis of transformers is still in early days, but work has already suggested that transformers and other generative models are more expressive than architectures that came before, and that they can implement Turing machine-like computation. In many ways, what we’re calling generative computing can be seen as a coming of age and triumph of the inductive paradigm.

Instructions and Data

The analogies between an LLM and a computer have been noted by many. One way to organize the world of computing is to boil it down to the bare essential elements: data, operations, and control flow. Operations consume and/or produce data, transforming it in ways that are useful to our larger objectives (e.g. add two numbers together, lookup an entry in a database, etc.). Control flow defines how go from one operation to the next, including constructs like if/then/else and loops.

Instruction-tuned LLMs can follow instructions, which have an obvious analogy to computer instructions/operations, just expressed in natural language. Those instructions can also operate on data—in the classic retrieval augmented generation (RAG) pattern, we look up potentially relevant documents from a database and include them in the prompt that we send to the LLM, along with instructions on what to do with them. So there’s a rather intuitive feeling that the LLM can be directed to perform operations on data, almost like applying a function to input data.

Finally, LLMs can certainly do at least a rudimentary form of control flow (minimally, sequencing a list of instructions in a do-this-then-that fashion), but it is worth noting that control flow is one of the key Achilles’ heels of LLMs today — they are not amazing at it. Even the strongest, biggest LLMs get lost easily in a complex flow of instructions in a prompt (it’s not uncommon to randomly drop or muddle instructions), and forget about trying to get them to loop through a list a fixed number of times without help—it’s just not going happen. Having larger, more extensively trained LLMs seems to help with control flow, but it’s safe to say that the control flow operations that a beginner learns in the first week of intro CS are still a struggle for LLMs.

Even leaving aside so-so control flow abilities, the computing model of LLMs is pretty odd when you look at it through the lens of computing. The memory of the system (as embodied in the kv cache) canonically can only grow until it runs out (i.e., the context window of the model fills up; management of kv cache is still fairly nascent), and there is limited distinction between instructions and data in the prompt. More vexingly, instructions can in principle be executed from anywhere in the memory at any time (this is how prompt injection attacks are possible), and all memory that is added in the course of the operation of the LLM is potentially tainted by content earlier in the context window (which opens up the possibility of side-channel attacks). Despite this, it’s a very interesting computing system, and we think that these issues can be addressed — we’ll cover this in a future post.

Our de facto programming model for LLMs is also somewhat strange when seen through the perspective of computing. Today, in large part owing to history, nearly every interaction with an LLM is organized around the metaphor of “chat.” Messages are sent into the LLM, the LLM produces a response, and the entire history of this back and forth is generally kept in an ever-growing history, whose size is limited by the context length that a model can handle. “Agent” interactions add other players to the party line, but it is still fundamentally a chat-like interaction.

It should be said that we’re already moving in the direction that I am describing — structured output modes in many LLMs, which attempt to constrain the LLM’s output to a format like JSON, are clearly acknowledging the need for the LLM and the system to meet each other in the middle, and the tendency toward increasingly elaborate input templates (starting with Hugging Face’s jinja2-based chat templates, and extending to tokenization libraries that now increasingly serve as companions to models).

Frameworks like Langchain also rose to prominence because they allowed a developer to somewhat more easily extract what they needed to get out of a model response to integrate it into the software that calls the LLM. There are also libraries like DSPy that take a more aggressive programming-first approach to LLM application development, very much in line with the spirit of generative computing. Our efforts are heavily influenced by these frameworks, but we think there are even bolder places we can go. Some of the best software libraries are “opinionated” (which generally has a positive connotation in the world of software) in that they focus on a limited set of key patterns and ideas, and free developers from uncertainty about choices that aren’t important (Ruby on Rails, the LAMP stack, and much of the Python programming language would all count as exemplars of “opinionated” designs). The Mellea library embodies our opinions on generative computing.