TLDR: Bun 1.4 reduces Letta Code's memory footprint substantially, but only modestly increased speed. I observed only one test regression and identical builds between Bun 1.3 (Zig) and Bun 1.4 (Rust).

Bun 1.4 is now out, which is a big deal! Bun was completely rewritten in Rust. Claude rewrote the bulk of Bun, using nearly $165k in API pricing (though the Bun team did not pay this, as Bun was acquired by Anthropic):

Pre-merge, this took 5.9 billion uncached input tokens, 690 million output tokens, and 72 billion cached input token reads — around $165,000 at API pricing. By hand, I think this would've taken 3 engineers with full context on the codebase about a year, during which time we wouldn't be able to improve Node.js compatibility, fix bugs, fix security issues or implement new features. We never would've done that. The realistic alternative was to do nothing and keep fixing the bugs at the top of this post forever.

So, it's a big change, and done with seemingly nothing but Free Speed and Reliability.

Letta Code is written in Typescript, and we often use Bun for development. I use Bun for all my personal builds, and I was curious about what it did to Letta Code's performance. I talk a little bit about Letta's cloud backend vs. local backend as well, for those who are curious.

Overview

I compared two git worktrees of Letta Code, one using Bun 1.3, and the other with Bun 1.4.

For each tree, I tested:

  • bun install with a cold cache, and with a warm cache.

  • bun run build

  • tsc --noEmit

  • Full unit test suite

  • CLI cold start, letta --version ten times with wall time + peak RSS via /usr/bin/time -l

  • All sequential runs, no parallel testing

Let's take a look at the results.

Results

Migration cost

The compiled bundle was byte-identical between Bun 1.3 and 1.4 (34.20 MB), which is a big win for such a huge backend change. The lockfile format also did not change.

We did notice a Windows PowerShell fallback test failure on 1.4, but it seems to be an easy fix. This was the only one of 6,681 tests to regress on 1.4. Perhaps because Windows is Cursed, actually.

Idle time

Bun 1.4 has a significantly lower amount of idle CPU use -- this is basically just letting the Letta Code TUI sit idle. Roughly ~48% less (6.7% vs. 3.5%).

Or, in stem-and-leaf form:

I think this test is a little simplistic, though -- would be nice to test runtimes better.

Development workflow

Bun 1.4 improves on development workflow, but not where I was expecting. I was expecting more free speed for things like unit tests. The big gain is on bun install, with nearly a 40% improvement in memory usage on cold installs (no cache available). Wall time is actually mildly slower, though probably within a standard error.

I did see some improvement in builds + tests (~5-6% faster) but this was marginal:

Backend

Letta Code has two backends. The default uses our Cloud backend, and the other is the local backend mode powered by Pi. The local backend is faster because it doesn't have to talk to Letta Cloud and can go directly to the provider. I wanted to understand a little more about improvements in each of these two backends across 1.3 and 1.4.

Broadly, these tests highlight the speed difference between local and cloud backends. I showed these to the team and we're going to see what we can do to improve the latency. Currently, the cloud backend buys you a lot of flexibility and portability -- cloud agents are massively parallel, can easily teleport to other computers, and have robust state across many different runtimes. The local backend is meant for agents locked to a single computer.

Cloud

Cloud is faster on 1.4 by about 10%, in terms of time-to-first-token and stream completion.

Local backend

The local backend also saw improvements in speed, but to a smaller extent (~6%).

Headless mode

When you run letta -p "<YOUR PROMPT>", your most recently used agent will run and return only the final assistant message.

I asked for a breakdown of where the time goes in headless mode across our two backends, broken down by Bun version. Most of the gap in runtime is due to the cloud boot time and general state synchronization -- basically, network costs.

Compilation

You can also compile binaries with bun build --compile. I don't normally use this mode, but Charles Packer (Letta CEO) recommended taking a look at it.

The Letta Code binary is 95.3 MB on Bun 1.3, and grows to 98.3 MB on Bun 1.4. Not a huge difference, but it is an increase. Compiling cuts cold-start wall time by about ~25% for both 1.3 and 1.4, so not a huge change across versions.

Conclusion

Bun 1.4 is solid! I'm impressed with the runtime/build improvements we've seen, though it didn't buy as much speed as I was expecting to see.

The general story is that Bun 1.4 is much lighter. 40% less memory on cold installs, 14% less on CLI startup, and half the idle CPU usage while the TUI is sitting in the background.

More importantly, the fact that the Bun team was able to completely change the backend from Zig to Rust without changing the resulting bundle + lockfile with only a single regression in our testing.

Well done to the team!

-- Cameron