Blog Title

Karpathy's 3 Software Paradigms Expanded

2025-06-20T00:00:00Z

In June of 2025, Andrej Karpathy gave a talk at YC's AI Startup School titled "Software is Changing (Again)" where he introduced three major software paradigms:

To reiterate the categories:

Software 1.0 - code
Software 2.0 - weights
Software 3.0 - prompts

This is a thought provoking way to think about the recent paradigms of software creation. I wanted to expand on the Software 1.0 because that category encapsulates a number of paradigm shifts that have happened in the software space.

Software Paradigms Expanded

	Paradigm	Start (≈)	Description
0	Hard-wired logic	1930s	Programs built by physically wiring circuits; fast execution, no flexibility
1a	Low-level symbolic code	1946	Programs written in numeric opcodes or assembly; error-prone but foundational
1b	Compiled procedural	1957	Code written in high-level languages like Fortran and C; compiled to machine code
1c	Managed and dynamic	1984	Code runs in VMs or interpreters; safe, high-level languages like Java, Python
2	Neural networks	2012	Software learned from data via training; great for vision/language, opaque logic
3	LLMs, reasoning, agents	2022	Natural language prompts create code or plans; tools and reasoning extend capabilities

Software 0: There was no general notion of "software" as a distinct entity or activity from hardware. The initial theory of software would be developed in 1935 by Alan Turing. The term "software" was coined by John Tukey in 1958 which is well after we started creating "software" (programming). And it should be mentioned that the first software was created by Ada Lovelace in 1843 for Babbage's Analytical Engine.
Software 1a: Software exists but is basically a direct mapping to the machine code instructions that the processor executes. No conceptual abstraction from the steps the computer is performing.
Software 1b: Higher level languages that are translated or compiled to machine code and then executed. Introduces higher-level abstractions like functions and data types, letting programmers describe algorithms without managing individual instructions or memory directly.
Software 1c: Software languages and tools now abstract the runtime environment itself using managed memory, dynamic typing, reflection, and interpreters. Code is no longer a static set of instructions but a flexible, evolving system that can inspect, modify, and extend itself while running.

Software Paradigms Expanded (Again)

The above table is a useful, but I also want to see finer granularity. So let's expand all three paradigm groups:

	Paradigm	Start (≈)	Key innovations	Typical tech / tools	What “programming” looks like	Human role	Notes & trade-offs
0.0	Hard-wired logic	1930s	Logic encoded in wiring & relays	IBM accounting machines, ENIAC plug-boards	Re-patch cables, flip switches, resolder boards	Electrical technician	Fast once wired; change = physical labour; no “stored” software
1.0	Machine-code	1946	Stored-program computer + numeric op-codes	IBM 701 punched cards, EDSAC paper tape	Hand-encode op-codes; feed decks/tape	Coder / calculator	First real software; opaque & error-prone; debugging = re-punch
1.1	Assembly language	1950	Mnemonics, labels, macros	Autocoder, MASM, NASM	One line per instruction; think in registers & jumps	System programmer	Easier than raw op-codes, still non-portable & low-level
1.2	Compiled procedural	1957	Compilers translate algebraic languages	Fortran, Algol, C, Pascal, COBOL	Describe algorithms; compiler handles optimisation	Software engineer	Portable & efficient; manual memory, explicit control flow
1.3	Symbolic / functional	1958	REPL, garbage collection, code-as-data macros	Lisp, Scheme, early Smalltalk	Type at a prompt; meta-program; treat code as data	Researcher / rapid-prototyper	Pioneered GC & FP; concept-rich, slow on early HW
1.4	Managed object-oriented	1984	Virtual machines, JITs, static typing + GC	Java/JVM, C#, CLR	Class hierarchies; runtime hides memory	Application developer	Safety & tooling at scale; some perf overhead
1.5	Dynamic scripting	1991	Interpreters + batteries-included libs & REPLs	Python, JavaScript, Ruby, Perl, Bash	Write glue code quickly; duck-typing	Scripter / web dev	Rapid iteration; late error detection; slower runtime
2.0	Deep perception nets	2012	GPU-accelerated gradient descent for vision	CNNs, ImageNet, TensorFlow 1.x	Define architecture + dataset; training learns weights	ML engineer	Excels at perception; data/compute hungry; opaque logic
2.1	Sequence & attention models	2014	RNNs, LSTMs, attention for language & time	seq2seq, Bahdanau attention, early Torch	Encode/dec sequence; training handles long context	NLP engineer	Better for language; still brittle, limited context
2.2	Self-supervised models	2017	Transformers & massive pre-training	Transformer, BERT, GPT-2/3 pre-2022	Pre-train on web data; fine-tune downstream	Foundation-model engineer	Broad knowledge; zero-shot transfer; huge training cost
3.0	LLM chat	2022	Conversational text & code gen, reasoning	ChatGPT, Claude, Gemini, Deepseek	Conversation, copy/paste code & errors	Prompt engineer	Great for synthesis & translation; needs validation
3.1	LLM agents	2025	Autonomous tool use & multi-step planning	Devin, Claude Code, Codex, Github Copilot	Write instructions, review results, iterate	Context engineer	Delegates work; must sandbox & supervise for safety

The numbering has gotten a bit unwieldly, but I decided to keep it this way so that it keeps a direct connection to Karpathy's numbering.

A few thoughts on this table:

Software 2.*: These paradigms were foundational for what was to come but for a number of reasons these had much narrower adoption in the software industry than what came before and what has come after.
Software 3.1: Agentic LLMs feels like an entirely different mode of developing software that is worthy of it's own paradigm.
I originally had a separate "LLM reasoning" row but I realized this was really just an improvement to the "LLM Chat" paradigm because it doesn't change the fundamental mode of programming or the human role so I combined it into "LLM Chat".

Note

I worked with ChatGPT (o3 and 4o) to populate and refine the tables above (and double checked and refined using wikipedia). The rest of the text was written by hand with some brainstorming and review by ChatGPT 4o.

A new LLM Coding Agent in 5 incremental steps and less than 80 lines of python

2025-06-12T00:00:00Z

Updates:

2025-06-16: add Github Copilot authentication command to prerequisites
2025-06-20: example of output from Github Copilot device auth flow

In this post I will show you how to create a working LLM coding agent in 5 incremental steps. We will use Simon Willison's llm library and John Daly's llm-github-copilot plugin. The plugin gives us access to LLM models via Github Copilot which means all you need to get started is a github account (no LLM API sign-up/credit card required).

This is very similar to a previous previous post An LLM Coding Agent in 6 incremental steps and about 140 lines of python. The key difference is the use of llm instead of LiteLLM. The resulting agent is much more concise and hopefully easier to understand.

Step 0: Prerequisites (uv, llm, etc)

You will either need a Github account (to access Github Copilot) or you will need LLM API keys with some billing credits. I recommend starting with the Github Copilot.

Create a directory for your agent:

$ mkdir my-agent
$ cd my-agent

Install uv (if you have not already) and then setup a python virtual environment:

$ curl -LsSf https://astral.sh/uv/install.sh | sh
$ uv venv
$ source .venv/bin/activate

Install versions of llm and the llm-github-copilot plugin that have sufficient tool calling support.

$ uv pip install "git+https://github.com/simonw/llm.git"
$ uv pip install "git+https://github.com/jmdaly/llm-github-copilot.git@refs/pull/18/head#egg=llm_github_copilot"

Authenticate with Github Copilot. You will be shown an 8 character code that you need to fill in at the provided github URL in order to authorize this client. Once you do this, API access token refreshes will happen automatically.

$ llm github_copilot auth login
Starting GitHub Copilot login process to obtain an access_token & API key...
Fetching API key...

Please visit https://github.com/login/device and enter code 1234-ABCD to authenticate GitHub Copilot.

Waiting for authorization... (attempt 1/12)
...
Authentication successful!

We are now ready to start the fun!

Notes:

The full code for each of the steps can be found at github.com/kanaka/llm-agent.
In the examples that follow, user input that is typed into the agent is shown with a plum color.

Step 1: Single API call

First, we are going to make a single API prompt call to our chosen LLM model and simply print the result. For this step, we will pass the first argument on the command line as the prompt that we want to send the LLM model.

#!/usr/bin/env python3

import llm
import sys
DEFAULT_MODEL = "github_copilot/o3-mini"

model = llm.get_model(DEFAULT_MODEL)
response = model.prompt(sys.argv[1])
print(response.text())

Now run your new "agent":

$ chmod +x agent.py

$ ./agent.py "In a single sentence, define the word 'agent'"
An agent is someone or something that acts on behalf of another person or entity, or that produces a specific effect.

We make a single call LLM model and provide it with a single "user" message. We then print the response message from the LLM.

Notes:

With the Github Copilot free plan, you get 50 "premium requests" per month. The code above uses "github_copilot/o3-mini" because it is currently the least expensive model with excellent coding ability (at 0.33 premium requests per call). The "github_copilot/gemini-2.0-flash-001" model is slightly cheaper (at 0.25 per call) but does not have full tool calling support via Github Copilot. If you have paid plan then you can switch the model to github_copilot/gpt-4.1 to get unlimited completion calls.
Refer to llm prompt API documentation

Step 2: Chat loop

Our current implementation makes a single query, prints the response, and then exits. We don't have a way to "chat" with the model. LLM models via their direct APIs are essentially stateless. The client has to provide all the state/context for each call. The llm library provides a conversation object that will accumlate this state for us.

Let's construct a conversation instance and then wrap its prompt calls in a loop. The loop prompts for a user prompt, calls the conversation.prompt API with the user prompt, and then prints the response.

#!/usr/bin/env python3

import llm
DEFAULT_MODEL = "github_copilot/o3-mini"

# Conversation setup

model = llm.get_model(DEFAULT_MODEL)
conversation = model.conversation()

while True:
    try:
        user_input = input("user> ")
    except EOFError as e:
        break

    response = conversation.prompt(user_input)
    print(f"assistant> {response.text()}")

Run our new version of the "agent" (user input is shown with a plum color):

$ ./agent.py
user> List five funny names for a Linux computer (without any explanation):
assistant> TuxTastic
KernelKicker
BashfulBot
PenguinPirate
SudoSultan
user> In a short sentence, explain why the fifth one is funny.
assistant> SudoSultan is funny because it blends the superuser command "sudo" with a regal title, humorously elevating its authority.

The second user query asks about the assistant's first response. A correct answer confirms that the context (previous user and assistant messages) is being accumulated correctly, otherwise the assistant would be unable to answer the followup question.

Step 3: Tool calling

Our "agent" does not have much agency; it can't do anything unless it can convince the user to act on it's behalf. If I ask "Show me what's in file ./README.md.", the assistant will likely answer that it can't do that, but will give suggestions to the user for how to view the file themselves.

Let's take a step towards greater agency by defining a tool that the assistant can invoke. Define a function read_file that the assistant can invoke and add it to a tools list:

# Tool definitions

def read_file(path: str):
    "Read the file at path. Returns a map {'content':content}"
    return {"content": open(path).read()}

tools = [read_file]

Let's also define some debug functions for concisely showing tool calls and responses and then pass the tools

def trunc(s, max=80):
    return s[:max-4] + '...' if len(s) >= max else s

def before_call(tool, tool_call):
    print(trunc(f"tool call> {tool.name}({tool_call.arguments})"))

def after_call(tool, tool_call, tool_result):
    print(trunc(f"tool result> {tool_result.output}"))

We need to include the tools list and debug functions when we instantiate our conversation instance:

model = llm.get_model(DEFAULT_MODEL)
conversation = model.conversation(
    tools=tools,
    before_call=before_call,
    after_call=after_call,
)

Finally, in our loop, we use the conversation.chain method instead of the prompt method.

    response = conversation.chain(user_input)

The chain method will will automatically detect when the assistant returns a response containing a tool call. It will then run that tool and send the result of the tool to the assistant. All tool calls are resolved before the chain method returns with the final assistant response.

Run our new version of the "agent":

$ ./agent.py
user> List the imports in ./agent.py
tool call> read_file({'path': './agent.py'})
tool result> {"content": "#!/usr/bin/env python3\n\nimport llm\nDEFAULT_MODE...
assistant> The file contains a single import:

import llm

Notes:

Refer to llm conversation API documentation

Step 4: File editing tools

Our agent has the ability to read files, now let's make the final small change that will turn it into a true coding agent. All that is need is a few more functions that give it the ability to list files, edit files, and create files.

import subprocess

def ls_dir(path):
    """Runs `ls -la path` to list files in the current directory.
    Returns a map {'stdout':stdout,'stderr':stderr,'returncode':code}"""
    cp = subprocess.run(["ls", "-la", path], capture_output=True, text=True)
    res = {k: getattr(cp, k) for k in ('stdout', 'stderr', 'returncode')}
    return res

def edit_file(path, match, replace):
    """Edit the file at 'path' replacing the first occurence of
    'match' string with 'replace' string. 'match' and 'replace' are
    raw strings and should not have escaped newlines, backslashes, etc.
    Returns an empty map on success"""
    orig = open(path).read()
    new = orig.replace(match, replace, 1)
    if new == orig: raise Exception("match string not found")
    open(path, "w").write(new)
    return {}

def create_file(path, content):
    """Create (or replace) file at 'path' with 'content'.
    'content' is a raw string and does not need extra escaping.
    Returns an empty map on success"""
    open(path, "w").write(content)
    return {}

tools = [read_file, ls_dir, edit_file, create_file]

Now let's use the new editing ability to make changes to a file:

$ echo 'hello NAME!' > greet.txt

$ ./agent.py
user> Change the greeting in the text file to use my first name.
assistant> I need a bit more information before proceeding. Could you please tell me the file name (or path) of the text file you want to change, and also provide your first name that should be used in the greeting?
user> My name is Joel. You find the file.
tool call> ls_dir({'path': '.'})
tool result> {"stdout": "total 20\ndrwxrwxr-x  3 joelmartin joelmartin 4096 ...
tool call> read_file({'path': 'greet.txt'})
tool result> {"content": "hello NAME!\n"}
tool call> edit_file({'match': 'NAME', 'path': 'greet.txt', 'replace': 'Joel'})
tool result> {}
assistant> I've updated greet.txt so that the greeting now reads "hello Joel!" Let me know if you need any further changes.

Note that sometimes the model will not ask the clarifying questions and will use the result of ls_dir to guess the first name based on directory/file ownership.

Step 5: Final changes (and a sprinkle of inception)

Those four simple tools/abilities (list files, read a file, edit a file, and create a file) are enough to give our agent the ability to do coding tasks. So let's use our agent itself to make some final improvements to its own code.

$ ./agent.py
user> Copy agent.py to agent2.py where we will make changes. Use correct python indenting for all future changes.
tool call> read_file({'path': 'agent.py'})
tool result> {"content": "#!/usr/bin/env python3\n\nimport llm\nDEFAULT_MODE...
tool call> create_file({'content': '#!/usr/bin/env python3\n\nimport llm\nDE...
tool result> {}
assistant> agent2.py has been created with the same content as agent.py. Let me know what changes you'd like to make next.
user> Add a friendly startup message showing the model.
tool call> edit_file({'match': ')\n\nwhile True:', 'path': 'agent2.py', 'rep...
tool result> {}
assistant> I've inserted a friendly startup message that displays the model in agent2.py. Let me know if you need any other modifications.
user> For the three prints and the input call, colorize the part before the '>' using ASCII escape codes (red, green, blue, purple).
tool call> edit_file({'match': 'input("user> "', 'path': 'agent2.py', 'repla...
tool result> {}
tool call> edit_file({'match': 'tool call>', 'path': 'agent2.py', 'replace':...
tool result> {}
tool call> edit_file({'match': 'tool result>', 'path': 'agent2.py', 'replace...
tool result> {}
tool call> edit_file({'match': 'assistant>', 'path': 'agent2.py', 'replace':...
tool result> {}
assistant> I've updated agent2.py to colorize the parts before the '>' using the specified ASCII escape codes:
- The input prompt now colors "user" in red,
- The "tool call>" prefix is colored in green,
- The "tool result>" prefix is colored in blue,
- The "assistant>" prefix is colored in purple.

Let me know if there are any further changes you'd like to apply!
user> Before calling a tool, confirm with the user. If confirmed continue as before, otherwise raise a llm.CancelToolCall exception with a "User Refused" message.
tool call> edit_file({'match': 'print(trunc(f"\\033[32mtool call\\033[0m> {t...
tool result> {}
assistant> I've updated the agent2.py before_call function to confirm with the user before calling a tool. The tool call will now prompt the user with "Proceed with tool call? (y/n)" and raise a llm.CancelToolCall exception if the user does not confirm.

Let me know if additional modifications are needed!

Now review the code changes and then test our agent modified agent:

$ diff -urp agent.py agent2.py
...

$ chmod +x agent2.py

$ ./agent2.py
Agent started with model: github_copilot/o3-mini
user> What is in ./greet.txt?
tool call> read_file({'path': './greet.txt'})
Proceed with tool call? (y/n) y
tool result> {"content": "hello Joel!\n"}
assistant> The file contains: "hello Joel!" followed by a newline.

Notes:

The full code for each of the steps can be found at github.com/kanaka/llm-agent.

Step 6: You take it from here!

You now have a basic working coding agent. The next steps are up to you. Here are some ideas:

Add streaming output so that you can see the progress of the model as it is generating longer responses.
Make the tool confirmation more sophisticated and allow specific actions or whole tools to be run without confirmation the next time.
Add proper python argument parsing. Allow the model name, system prompt, tool confirmation mode, etc to be specified as command line. Allow the initial query to optionally be specified on the command line (like step 1).
Give it the ability to run tests.

An LLM Coding Agent in 6 incremental steps and about 140 lines of python

2025-05-01T00:00:00Z

[Refer to LLM Agent in Five Steps for a shorter/simpler version of this tutorial that leverages the llm python module instead of LiteLLM]

Updates:

2025-05-06: more detail around tool calling and parameter inspection
2025-06-20: simplify litellm dependency install.

In this post I will show you how to create a working LLM coding agent in 6 incremental steps. We will use the python LiteLLM library and use Github Copilot which means all you need is a github account (in other words, no LLM API sign-up is needed to get started). The full code for each step shown below can be found at github.com/kanaka/litellm-agent.

Step 0: Prerequisites (uv, LiteLLM, etc)

You will either need a Github account (to access Github Copilot) or you will need LLM API keys with some billing credits. I recommend starting with the Github Copilot.

Create a directory for your agent:

$ mkdir my-agent
$ cd my-agent

Install uv (if you have not already) and then setup a virtual environment:

$ curl -LsSf https://astral.sh/uv/install.sh | sh
$ uv venv
$ source .venv/bin/activate

Install a branch of the LiteLLM project (into the venv) that has support for Github Copilot models.

$ uv pip install git+https://github.com/BerriAI/litellm.git@litellm_dev_03_05_2025_contributor_prs

Note: in the examples that follow, user input that is typed into the agent is shown with a plum color.

We are now ready to start the fun!

Step 1: Single API call

First, we are going to make a single API "completion" call to our chosen LLM model and simply print the result. For this step, we will pass the first argument on the command line as the query that we want to send the LLM model.

#!/usr/bin/env python3

from litellm import completion
import sys

model = "github_copilot/o3-mini"
extra_headers = {"editor-version": "vscode/1.85.1"}

response = completion(
    model=model,
    extra_headers=extra_headers,
    messages=[{"content": sys.argv[1], "role":"user"}],
)

print(response.choices[0].message.content)

Now run your new "agent":

$ chmod +x agent.py

$ ./agent.py "In a single sentence, define the word 'agent'"
An agent is someone or something that acts on behalf of another person or entity, or that produces a specific effect.

We make a single call LLM model and provide it with a single "user" message. We then extract the response message from the LLM and print it. The LiteLLM completion function is based on OpenAPI defined API. While other providers have similar APIs, they do have differences. LiteLLM provides a common API that has the same input and output format across all the providers.

Notes:

The first time you run your agent with a Github Copilot model, you will be shown an 8 character code that you need to fill in at the provided github URL in order to authorize this client. Once you do this, API access token refreshes will happen automatically.
With the Github Copilot free plan, you get 50 "premium requests" per month. The code above uses "github_copilot/o3-mini" because it is currently the least expensive model with excellent coding ability (at 0.33 premium requests per call). The "github_copilot/gemini-2.0-flash-001" model is slightly cheaper (at 0.25 per call) but my experience is that it has trouble using tools correctly. If you have paid plan then you can switch the model to github_copilot/gpt-4 to get unlimited completion calls.
The "editor-version" HTTP header is needed for now in order to use the "Github Copilot" functionality in the LiteLLM branch.

Step 2: Chat loop

Our current implementation makes a single query, prints the response, and then exits. We don't have a way to "chat" with the model. LLM models via the API are essentially stateless. The client program has to provide all the state/context for each call.

Let's wrap the completion call and output in a loop. Prior to the loop we initialize a messages list (context) with our initial system prompt. The loop prompts for a user query, adds this to the context, calls the completion API, and then adds the assistant (LLM model) response to the messages list in addition to printing it out.

messages = [{"content": "You are a coding agent", "role":"system"}]

while True:
    try:
        user_input = input("user> ")
    except EOFError as e:
        break
    messages.append({"content": user_input, "role":"user"})

    response = completion(
        model=model,
        extra_headers=extra_headers,
        messages=messages,
    )

    resp_message = response.choices[0].message
    messages.append(resp_message.model_dump())
    
    print(f"assistant> {resp_message.content}")

Run our new version of the "agent" (user input is shown with a plum color):

$ ./agent.py
user> List five funny names for a Linux computer (without any explanation):
assistant> TuxTastic
KernelKicker
BashfulBot
PenguinPirate
SudoSultan
user> In a short sentence, explain why the fifth one is funny.
assistant> SudoSultan is funny because it blends the superuser command "sudo" with a regal title, humorously elevating its authority.

The second user query asks about the assistant's first response. A correct answer confirms that the context (messages list) is being accumulated correctly, otherwise the assistant would not be able to answer the followup.

Step 3: Tool calling

Let's take a step towards greater agency by defining a tool that the assistant can invoke. First, create an API data structure that defines tools in a format the assistant understands. The following defines a single tool read_file that the assistant can invoke:

import json
tools = [
    {
        "type": "function",
        "function": {
            "name": "read_file",
            "description": "Read the file at path. Returns a map {'content':content}",
            "parameters": {
                "type": "object",
                "properties": {
                    "path": {"type": "string"},
                },
                "required": ["path"]
            },
        },
    }
]

We pass this new tools structure in the completion call:

    response = completion(
        model=model,
        extra_headers=extra_headers,
        messages=messages,
        tools=tools,
    )

Let's add a trunc function for abbreviating long output messages that we'll use in the next step:

def trunc(s, max=80):
    return s[:max-4] + '...' if len(s) >= max else s

If the assistant decides that it wants to call a tool, then the response message will contain a populated tool_calls field. If this field is set and the tool name is read_file, then we parse the arguments (which are encoded as a JSON string), then we read the file referred to in the path argument, and finally we append a tool call response to the context (messages list). The response message contains the original unique ID for this tool call and has a "content" field that contains the JSON encoded result of the tool call. Replace the assistant output message line as follows:

    tool_calls = resp_message.tool_calls
    if not tool_calls:
        print(f"assistant> {resp_message.content}")
        continue

    for tc in tool_calls:
        fn = tc['function']
        if fn.name == "read_file":
            fn_args = json.loads(fn.arguments)
            print(trunc(f"calling read_file({fn_args})"))
            fn_result = {"content": open(fn_args['path']).read()}
            res_str = json.dumps(fn_result)
            print(trunc(f"result: {res_str}"))
            messages.append({
                "role": "tool",
                "tool_call_id": tc['id'],
                "name": fn.name,
                "content": res_str,
            })
        else:
            raise Exception(f"Unknown tool call: {fn.name}")

There is one additional change that we need to make to the top of our loop. If the last message from the assistant is a tool call, then we should skip the user input/prompt and just send the tool call result that we added:

    if messages[-1]["role"] != "tool":
        try:
            user_input = input("user> ")
        except EOFError as e:
            break
        messages.append({"content": user_input, "role":"user"})

Run our new version of the "agent":

$ ./agent.py
user> What is the first line of file ./agent.py?
calling read_file({'path': './agent.py'})
result: {"content": "#!/usr/bin/env python3\n\nfrom litellm import completio...
assistant> The first line of the file is: #!/usr/bin/env python3

Step 4: Generic tools interface

The current implementation uses a hard-coded definition for the read_file tool that is passed to the completion API. Instead of creating a new definition for each tool that we want to support, we can use python's introspection to generate this definition from the python functions themselves. The inspect.signature function returns detailed information about each parameter of a python function. We use this information to construct the map of parameter map needed for the completion call.

Define a function that that takes a map of tools names to functions and returns the tools value needed for the completion API:

import inspect

def get_tools_param(tools_map):
    typemap = {int: "integer", float: "number", bool: "boolean"}
    tools = []
    for name, fn in tools_map.items():
        props, req = {}, []
        for p in inspect.signature(fn).parameters.values():
            if p.kind.name.startswith("VAR"):  # skip *args/**kw
                continue
            props[p.name] = {"type": typemap.get(p.annotation, "string")}
            if p.default is p.empty:
                req.append(p.name)
        tools.append({
            "type": "function",
            "function": {
                "name": name,
                "description": (fn.__doc__ or "").strip(),
                "parameters": {
                    "type": "object",
                    "properties": props,
                    **({"required": req} if req else {})
                }
            }
        })
    return tools

Then define the tool functions with a sufficient docstring and result format that makes it clear to the assistant how the tools should be used:

def read_file(path):
    """Read the file at path.
    Returns a map {'content':content} (raw file string in content)"""
    return {"content": open(path).read()}

import subprocess

def ls_dir(path):
    """Runs `ls -la path` to list files in the current directory.
    Returns a map {'stdout':stdout,'stderr':stderr,'returncode':code}"""
    cp = subprocess.run(["ls", "-la", path], capture_output=True, text=True)
    res = {k: getattr(cp, k) for k in ('stdout', 'stderr', 'returncode')}
    return res

Remove the global tools variable and add a new tools map (after the tool functions) that enumerates the available tools including the new ls_dir tool/function.

TOOLS_MAP = {
    "read_file": read_file,
    "ls_dir": ls_dir,
}

Prior to our main loop use the get_tools_param function to initialize the tools value that is passed to the completion call:

tools = get_tools_param(TOOLS_MAP)

Finally, adjust the tool calling loop to use a more generic mechanism for calling the tools defined in the TOOLS_MAP:

    for tc in tool_calls:
        fn = tc['function']
        if fn.name in TOOLS_MAP:
            fn_args = json.loads(fn.arguments)
            print(trunc(f"calling {fn.name}({fn_args})"))
            fn_result = TOOLS_MAP[fn.name](**fn_args)
            ...

Let's try it out:

$ ./agent.py
user> Summarize the functionality of the python file in this dir in one sentence.
calling ls_dir({'path': '.'})
result: {"stdout": "total 16\ndrwxrwxr-x@  7 joelmartin  staff   224 May  2 ...
calling read_file({'path': './agent.py'})
result: {"content": "#!/usr/bin/env python3\n\nfrom litellm import completio...
assistant> This Python script implements an interactive command-line coding agent that leverages a language model to process user inputs and execute defined filesystem tools (such as reading files and listing directory contents) based on dynamically generated tool calls.

Step 5: File editing tools

Our agent has the ability to list and read files, now let's make the final improvement that will actually turn this into a coding agent.

Define the functions that will allow the agent to edit and create new files and add them to the TOOLS_MAP:

def edit_file(path, match, replace):
    """Edit the file at 'path' replacing the first occurence of
    'match' string with 'replace' string. 'match' and 'replace' are
    raw strings and should not have escaped newlines, backslashes, etc.
    Returns an empty map on success"""
    orig = open(path).read()
    new = orig.replace(match, replace, 1)
    if new == orig: raise Exception("match string not found")
    open(path, "w").write(new)
    return {}

def create_file(path, content):
    """Create (or replace) file at 'path' with 'content'.
    'content' is a raw string and does not need extra escaping.
    Returns an empty map on success"""
    open(path, "w").write(content)
    return {}

TOOLS_MAP = {
    "read_file": read_file,
    "ls_dir": ls_dir,
    "edit_file": edit_file,
    "create_file": create_file,
}

Those new tools enable a large amount of agentic coding ability. Let's improve the error handling and format of the tool calling messages:

            print(trunc(f"tool call> {fn.name}({fn_args})"))
            try:
                fn_result = TOOLS_MAP[fn.name](**fn_args)
            except Exception as e:
                fn_result = {"error": str(e)}
            res_str = json.dumps(fn_result)
            print(trunc(f"tool result> {res_str}"))
            message.append({
                ...

Now let's use the new editing ability to make changes to a file:

$ echo 'hello NAME!' > greet.txt

$ ./agent.py
user> Change the greeting in the text file to use my first name.
assistant> Could you please clarify two things for me?
1. What is the path or name of the text file you're referring to?
2. What is your first name that you would like to use in the greeting?
user> My name is Joel. You find the file.
tool call> ls_dir({'path': '.'})
tool result> {"stdout": "total 16\ndrwxrwxr-x@  7 joelmartin  staff   224 Ma...
tool call> read_file({'path': 'greet.txt'})
tool result> {"content": "hello NAME!\n"}
tool call> edit_file({'match': 'NAME', 'replace': 'Joel', 'path': 'greet.txt'})
tool result> {}
assistant> The greeting in greet.txt has been updated to "hello Joel!"

Note that sometimes the model will not ask the clarifying questions and will use the result of ls_dir to guess the first name based on directory/file ownership.

Step 6: Final touch ups

Those four simple tools/abilities (list files, read a file, create a file, and edit a file) are enough to give our agent the ability to do coding tasks. So let's use our agent itself to make some final improvements to its own code.

$ ./agent.py
user> Copy agent.py to agent2.py where we will make changes. Start by adding a friendly startup message (with model).
tool call> read_file({'path': 'agent.py'})
tool result> {"content": "#!/usr/bin/env python3\n\nfrom litellm import comp...
tool call> create_file({'content': '#!/usr/bin/env python3\n\nfrom litellm i...
tool result> {}
assistant> I've created agent2.py with a friendly startup message that prints the model being used.
user> For the three prints and the input call, colorize the part before the '>' using ASCII escape codes (red, green, blue, purple).
tool call> create_file({'content': '#!/usr/bin/env python3\n\nfrom litellm i...
tool result> {}
assistant> I've updated agent2.py to colorize the prompt and prints. Red is now used for the user input, green for tool call, blue for tool result, and purple for assistant's plain output.
user> Before calling a tool, confirm with the user. If confirmed continue as before, otherwise add a response map to the messages with a "User Refused" error.
tool call> create_file({'content': '#!/usr/bin/env python3\n\nfrom litellm i...
tool result> {}
assistant> I've updated agent2.py to prompt the user for confirmation before calling any tool. If you enter anything other than 'y', the tool call will be skipped and an error message ("User Refused") will be added to the messages.

Now test the agent driven code changes:

$ chmod +x agent2.py

$ ./agent2.py
Welcome! Starting agent with model: github_copilot/o3-mini
user> What is in ./greet.txt?
Confirm calling tool 'read_file' with arguments {'path': './greet.txt'}? (y/N)> y
tool call> read_file({'path': './greet.txt'})
tool result> {"content": "hello Joel!\n"}
assistant> The content of "./greet.txt" is:
hello Joel!

The full code for each of the above steps can be found at github.com/kanaka/litellm-agent.

Step 7: You take it from here!

You now have a basic working coding agent. The next steps are up to you. Here are some ideas:

Add streaming output so that you can see the progress of the model as it is generating longer responses.
Make the tool confirmation more sophisticated and allow specific actions or whole tools to be run without confirmation the next time.
Add proper python argument parsing. Allow the model name, system prompt, tool confirmation mode, etc to be specified as command line. Allow the initial query to optionally be specified on the command line (like step 1).
Add the ability to continue a previous session (save/pickle the context and reload it if requested).
Give it the ability to run tests.
Get it working reliably with other models. For example, github_copilot/gemini-2.0-flash-001 is the cheapest model but it has trouble using the tools the way they are currently defined.

The Most Important Parts of HTML5

2011-08-06T00:00:00Z

or Why <video> and <audio> are Boring

or The New Web Platform

or An Introduction to HTML5

[This page was originally published at http://blog.n01se.net/?p=375. That site no longer exists. On 2025-05-06 I resurrected the content here for posterity. I have update many of the links if I was able to find a modern versions of the same (or similar) content. Otherwise I have linked to Internet Archive Wayback Machine versions of those pages. The original version of this page (along with the terrible background) in the Wayback Machine is here.]

A Little Perspective

The Birth of the Web

20 years ago today (Aug 6th, 1991), Tim Berners-Lee released the World Wide Web on the world while working at CERN. Actually what he released was a program called "WorldWideWeb" which was eventually renamed "Nexus" to clarify the distinction between the concept of the World Wide Web and the browser itself.

The initial browser could render documents written in HyperText Markup Language (HTML). The first HTML supported some limited formatting and hyperlinks (we just call them links now) to other documents. Here is a screenshot from 2 years later when color and inline images were added:

WorldWideWeb Browser Running on a NeXT System

The Rise of the Web

In the next 10 years "the Web" exploded in terms of innovation, standardization (Footnote 1), number of users, browsers (installs and variations), web servers (installs and variations), and economic impact. With the release of Mosaic (at NCSA) in 1993, with its seamless integration of graphics and text, the World Wide Web quickly grew to become the dominant use of the Internet and the driving force behind Internet adoption. In fact, for most computer users "the Web" and "the Internet" have become synonymous.

The Fall of the Web

At the turn of the century, Web innovation slowed due to the emergence of Microsoft's Internet Explorer as a near monopoly in the web browser market. Once Microsoft achieved a controlling share of the web browser market they lost interest in driving or cooperating on new Web standards and technologies because this might threaten their profitable Windows platform.

The Rise of the Web (Again)

In the last three years there has been a new explosion of Web innovation. The Web has been released from its cage by three main trends: the rise of Mozilla Firefox (a spiritual descendant of the Mosaic browser), the rise of Google Chrome, and the rise of mobile devices.

Web Browser Market Share (Wikimedia)

A core part of human nature is to give names to everything including to abstract concepts. The new energy and innovation surrounding the Web platform is no exception; it needs a name. Which brings us to "HTML5" ...

"HTML5"

Technically, HTML5 is a specification from the World Wide Web Consortium (W3C) (Footnote 2). Many pedants will claim this is the only correct usage. For the rest of us, HTML5 is a useful term to describe the rapid changes that are currently happening to the Web platform. This is what I will mean when I use the term "HTML5". I will refer to the formal specification as "W3C HTML5".

W3C HTML5

Although the W3C HTML5 specification is not going to be officially complete until 2014, the specification finished last call review earlier this week (Aug 3rd) so there are unlikely to be any radical changes in the next three years before finalization.

W3C HTML5 is Boring

The W3C HTML5 is very important at one level, but it is also pretty boring. It is basically a formal description of the state-of-the art in the Web platform from 3 years ago. The actual content of the specification is pretty mundane (even apart from the dry and technical nature of specification documents). The most interesting API in the W3C HTML5 is the Canvas 2D Context and that is defined in a separate document.

W3C HTML5 is Important

However, the W3C HTML5 is important because it makes official all the good ideas that have been learned over the years and it attempts to remove most of the things that are now considered mistakes. It also brings a great deal of consistency and completeness to the various DOM APIs and HTML elements. And probably most importantly, it has brought the various browser makers into agreement. This means web developers that develop against what is defined in the W3C HTML5 specification should have one application that works well on all recent browsers versions without the need for browser specific kludges.

W3C HTML5 vs HTML5

If I were to sum up the differences between the W3C HTML5 specification and the larger concept of HTML5 it would be this:

The W3C HTML5 promotes many of the existing second class elements of the Web such as video, audio, animations, smart forms, etc into first class elements.
HTML5 (the common usage) takes those new elements and adds power and functionality to them that was not previously possible. HTML5 also creates a whole new set of first class elements out of technologies that were not part of the Web in the first place such as hardware device access, binary data, file system access, multiprocessing, etc.

Or another way of summing up the relationship between the two:

The W3C HTML5 specification serves as the foundation and framework upon which all the interesting HTML5 developments are happening.

The Most Important Parts of HTML5

Now we come to my purpose for this article: to list and describe the HTML5 APIs, standards, and technologies that are most important (and most interesting). I have tried to imagine the Web as it will exist five years from now and from that vantage point determine what parts of HTML5 were most crucial in bringing us to that imagined future.

The following list is ordered from most to least important. Obviously this is just my opinion. I do web application development (noVNC, websockify) and participate in HTML5 working groups and discussions, but the future is always more interesting (and less) than any experts can predict.

So without further ado...

1. Faster Javascript Engines

The Web as a application platform is built on this more than anything else. The new Javascript engines are the warp drive for the Web. Without the warp drive, Star Trek is a story that takes place on earth (or at best in one solar system). Without the massive increase in Javascript performance we would still be talking about web pages and not web applications.

2. WebSockets

This moves the browser solidly into the space of highly interactive networked applications. After fast Javascript engines, low-latency networking has the largest potential for allowing the Web to conquer new application domains.

3. Binary Data Types (Typed Arrays and Blobs)

Javascript started its life as a way to do validation of textual form data. However, many of the first class elements introduced in HTML5 contain, receive and/or output binary data and so native binary data types in Javascript has become a necessity. Developers have used various hacks to encode binary data in old Javascript data types for many years. But using these hacks is a significant barrier and the full power of HTML5 will not be unleashed without native binary data support.

4. Web Audio API

This is not the <audio> tag but rather the APIs/proposals for allowing low-latency, direct audio manipulation from Javascript. The <audio> tag (which is part of W3C HTML5) allows an audio file to be embedded directly in a web page and it provides a playback and synchronization API.

The Web Audio API proposals allows for direct creation and manipulation of audio waveforms and also address issues of high-latency playback that exist in current <audio> tag implementations. These proposals are still very young and the final solutions may merge with the <audio> tag, but the issues addressed by the Web Audio API proposals will be in future browsers in one form or another.

5. Canvas 2D Context

Direct pixel manipulation. Everybody agrees it is important so there is not much I will add. I put it below the Web Audio API because much of what can be done with Canvas 2D Content can be done with other methods (SVG, WebGL, CSS3).

6. CSS3 and WOFF

Cascading Style Sheets 3 and the Web Open Font Format together bring the full power of design, typography, layout, and visual transformation to the web. Also, with CSS3 (in particular the Flexible Box Model), web applications will finally have a simple and powerful way of doing user interface layout without the element positioning mess that is necessary with CSS2.

7. Local Storage, Offline Applications and the File APIs

There are many application domains that are just not feasible without fast, local and persistent storage (at least not until everyone has cheap Gigabit Internet connections). Some form of local storage is also necessary for web applications to be usable when there is no Internet connection available. There are a number of APIs/standards being developed in this area but they are all addressing different aspects of the same fundamental limitation of pre-HTML5 browsers.

8. Web Workers

Moore's Law is dead, long live Moore's Law! The year-after-year exponential increase in CPU frequency due to Moore's Law ended several years ago. But Moore's Law was actually a statement about transistor cost/density and this has not changed, it simply has a new face: processor cores per square inch of chip. In a few years, even your mobile phone will have more processor cores than you have fingers.

New software models are required to take full advantage of the new multi-core reality of Moore's Law. Fortunately, even though Javascript has always been a single threaded language it was also designed from the beginning to an event driven language. This means that while multiple lines of Javascript in the same web application cannot be running simultaneously, the browser can be doing multiple things at once on behalf of that Javascript code that is running.

Being event driven only goes so far. The Web Workers specification was created to allow a single web application to have multiple threads of Javascript running simultaneously. To avoid the massive complexity that usually comes with multi-threaded programming (locks, special data structures, etc), Web Workers are independent Javascript contexts and they can only interact with each other and with the main Javascript thread using event driven message passing.

9. SVG 1.1/2.0

The SVG (Scalable Vector Graphics) format has been around for a long time and some browsers have been able to embed SVG images into web pages. SVG is finally starting to be adopted by all browser makers in a form that adds powerful APIs and that allows full access and manipulation to the contained elements (i.e. a truly first class element).

SVG is actually a difficult one to place in the list. Many (perhaps even most) of the uses of the Canvas 2D Context are actually more appropriate to SVG and in many ways SVG is far more powerful. However, SVG has had an uneven history and I fear that it has accumulated some unjustified mental baggage that will prevent it from being as fundamental and important as it otherwise would be. I will be happy to be proved wrong if it turns out to be more important than I have rated it.

10. WebGL

This is the Canvas 3D Context and it is basically a hardware accelerated OpenGL API for the Web. Like SVG, this is potentially a very important piece of HTML5. But I say potentially because Microsoft has been somewhat dismissive of WebGL (possibly since it is defined in terms of OpenGL rather than their own DirectX API) and so there is uncertainty about whether this will ever make it into Internet Explorer. If there were less uncertainty I would place this higher in the list because it brings the Web to the doorstep of so many new application domains (including 3D games).

11. All the Rest

The ultimate vision of many who are pushing forward the Web platform is to make the Web platform as powerful, capable and comprehensive as native applications. The first 10 items each open up the Web platform to large new application domains that have historically only been possible with native applications. But they leave many gaps that must be filled before we reach a future where the question asked by developers is no longer "Can I build this as a web application?" but rather "Do I want to build this as a web application?"

There are numerous proposals that are being worked on to address the gaps in web application functionality. Several of them would probably be higher on the list if they were further along or had less uncertainty about whether they will be universally adopted by all browser makers. Here are just a few of the proposals/APIs that are attempting to fill the gaps:

WebRTC/Stream API: Peer-to-peer video conferencing.
Geolocation: Where in the World am I?
Orientation: Which way is up?
Crypto: Encrypt/decrypt efficiently in Javascript
WebCL: The Web version of OpenCL. Who wouldn't want to use the GPU directly for computation from Javascript?
WebNotifications: Tell me what's happening, but gently.
Web Intents: Associate data types with default actions and pass data back and forth between web applications. Your favorite web application for editing images will inevitably be different then the default one for that web based slideshow application. Shout with me: "OLE!"
Page Visibility: Imagine how much energy the world would be save if those animations and movies stopped rendering when you aren't looking at them.
requestAnimationFrame: with setTimeout you get 100 FPS or you get 2 FPS (often within the same second). Now you can get a consistent 30 FPS. And there was much rejoicing.
Microdata: unambiguous parsing of embedded machine-readable data.
Etc
Etc (Footnote 3)

Mac vs PC vs Browser (from xkcd.com)

What about <video> and <audio>?

The <video> and <audio> tags are probably the HTML5 features that have caused the most excitement (and angst) on the web regarding HTML5. In fact, some of you may have scanned ahead looking for them in the list and were surprised they were not #1 and #2. These two elements are part of W3C HTML5 and while they do make first class elements out of what has traditionally been second class (e.g. done with Flash), they are still really just a better way of doing what has already been done. The <video> and <audio> tags don't significantly expand the scope of what web applications are capable of so that is one reason why they are not on the list. Another problem with these tags is that the list of supported media formats is inconsistent across browsers. Until this is resolved, their adoption will be hampered. (Footnote 4)

Footnotes

These were originally inline in the article and they seemed to interrupt the flow so I moved them here. And even without these notes the article is still too long.

Footnote 1

Here are some notable standards that developed around the Web platform in the first few years after the first web browser was released by Tim Berners-Lee:

1995 - HTML 2.0 is published as the first official HTML standard.
1995 - HTML make the first baby step toward becoming dynamic and interactive when JavaScript is created by Brendan Eich and added to Netscape Navigator 2.
1996 - The CSS1 (Cascading Style Sheets) specification is completed and parts of it appear in Internet Explorer 3. Part of the reason for CSS is to separate web page appearance from content and functionality. This follows as well established principle in computer science of separating concerns.
1998 - The DOM Level 1 (Document Object Model) specification, the API of HTML elements on a page, is published by the W3C.

Footnote 2

I should note another important group (with a long and forgettable name) that is involved in HTML standards is the Web Hypertext Application Technology Working Group (WHATWG). Many (perhaps most) of the people at WHATWG also participate in work at the W3C. The WHATWG can be thought as a more dynamic and less restricted version of the W3C.

The WHATWG maintains two overarching standards documents. The first is HTML. Many of the ideas that are in the W3C HTML5 were first documented here. This document can be thought of HTML5+. The second standard document is called Web Apps 1.0. This contains the content of the HTML document and adds many of the APIs that are considered important for the Web to become a full fledged application platform.

The WHATWG Web Apps 1.0 document is much closer to what people generally mean when they use the term "HTML5", however, even that does not encapsulate the whole meaning. There are many other APIs and proposals that are part of the new energy of the Web that are or will make their way into most browsers (either via the standards process or not).

Footnote 3

Etc. I am not aware of a web proposal/API yet with those three letters. The question is will this article will make it more or less likely that there will be one soon?

Footnote 4

By combining fast Javascript, WebSockets, Canvas, Web Audio, and binary data types you can have video and audio without the <video>/<audio> tags and without any plugins. The frame rate and resolution might be lower than desired but it is possible.

Resources

Dive Into HTML5
HTML5Rocks
Excellent Presentation on HTML5 from HTML5Rocks
When can I use... Compatibility tables for support of HTML5, CSS3, SVG and more in desktop and mobile browsers.
HTML5 Readiness - Flower chart of HTML5 & CSS3 features in major browser versions
The HTML5 Test - How well does your browser support HTML5?
The Web Platform: Browser technologies ()
HTML5 Tweets
W3C Blog
WHATWG Blog
Google Chrome Blog
Mozilla Firefox Blog
Opera Developer Blog
Microsoft Internet Explorer Blog
"An Open Web" book
"20 Things I Learned About Browsers and The Web" Google comic
"What is HTML5"

Blog Title

Karpathy's 3 Software Paradigms Expanded

Software Paradigms Expanded

Software Paradigms Expanded (Again)

Note

A new LLM Coding Agent in 5 incremental steps and less than 80 lines of python

Step 0: Prerequisites (uv, llm, etc)

Step 1: Single API call

Step 2: Chat loop

Step 3: Tool calling

Step 4: File editing tools

Step 5: Final changes (and a sprinkle of inception)

Step 6: You take it from here!

An LLM Coding Agent in 6 incremental steps and about 140 lines of python

Step 0: Prerequisites (uv, LiteLLM, etc)

Step 1: Single API call

Step 2: Chat loop

Step 3: Tool calling

Step 4: Generic tools interface

Step 5: File editing tools

Step 6: Final touch ups

Step 7: You take it from here!

The Most Important Parts of HTML5

or Why <video> and <audio> are Boring

or The New Web Platform

or An Introduction to HTML5

A Little Perspective

The Birth of the Web

﻿﻿The Rise of the Web

The Fall of the Web

The Rise of the Web (Again)

"HTML5"

W3C HTML5

W3C HTML5 is Boring

W3C HTML5 is Important

W3C HTML5 vs HTML5

The Most Important Parts of HTML5

1. Faster Javascript Engines

2. WebSockets

3. Binary Data Types (Typed Arrays and Blobs)

4. Web Audio API

5. Canvas 2D Context

6. CSS3 and WOFF

7. Local Storage, Offline Applications and the File APIs

8. Web Workers

9. SVG 1.1/2.0

10. WebGL

11. All the Rest

What about <video> and <audio>?

Footnotes

Footnote 1

Footnote 2

Footnote 3

Footnote 4

Resources

Links

The Rise of the Web