Notational Intelligence, Linus Lee | Compile 26

⬅️ Back to Videos

🎥 Notational Intelligence, Linus Lee | Compile 26

Linus Lee (Thrive Capital). Duration: 17 min

Timestamps

  • 0:00 Notational intelligence
  • 1:27 What makes good notation
  • 2:28 Abstraction
  • 3:18 Suggestiveness and natural transformations
  • 5:28 Graphical notation
  • 6:27 The coordinate plane and the arrow
  • 8:14 Programming languages as notation
  • 9:28 Inventing new notation with deep learning
  • 11:13 Building the toy model
  • 13:42 The handout and training results
  • 15:03 Invariants that make symbols meaningful
  • 15:56 Our world vs. an alien world of ideas
  • 16:30 Models as a simulator for anything

Linus Lee thinks notations (how we write things down) might have a bigger impact on human intelligence than the machines we build. He walks through what makes a great notation, then builds a toy deep learning model that invents its own alphabet from scratch.

  1. More of the world is notation than computation. Lee points out that in a room of a few thousand people, there are maybe a thousand computers but probably a hundred times more instances of notation: symbols, maps, notes, signage. He argues notations are the invisible infrastructure of thinking, and we barely notice them.

  2. Abstraction is the first superpower of good notation. When you write y = mx + b, that b isn’t one number. It’s a whole family of possible numbers. You can manipulate an entire set of relationships by moving a single symbol. That’s notational leverage.

  3. Suggestiveness means the shape of the symbol does work for you. Lee compares Leibniz’s dy/dx (suggestive, you can multiply and rearrange it) with Newton’s dot notation (simple but inert). Leibniz won because the notation itself invites manipulation. Terry Tao calls this a “natural transformation”: operating on the symbol does something real to the idea.

  4. The arrow is shockingly young. The earliest recorded arrow symbol is from 1737, barely 300 years old. Before that, if you wanted to point at something you drew a hand. Someone had to invent the arrow, and that invention changed how we think on paper. Lee’s point: notations are designed, not discovered.

  5. Flatness is a feature. 3D notation could exist, but 2D is mobile, scalable, copyable. You can bring the same geometric tools (arrows, ratios, measurement) to a map of a continent or a diagram of a bacterium. The medium’s constraints become its strengths.

  6. Programming languages are a kind of notation, and indentation is graphical. Lee argues that when you indent code or syntax-highlight it, you’re using visual perception biases. The same ones that make the coordinate plane work are communicating scope and structure. Your editor is a notational system.

  7. Lee built a toy model that invents its own alphabet. He set up a generator that draws 32x32 grayscale images of symbols and a decoder that tries to read them back. The model learned 1,024 distinct symbols, an entirely made-up writing system, starting from noise. Over training, it progressed from basic black-and-white to complex shapes. It looks a lot like the evolution of written language.

  8. Without visual constraints, the model cheats. Train it naively and it maps each concept to a single pixel. Lee had to impose scale invariance, rotation invariance, and brightness invariance. These are the same biases human visual perception has, and they force the model to learn real, meaningful shapes. The invariants are what make the symbols work for both the model and any human who might read them.

  9. The real promise is treating models as simulators for alien worlds. Lee draws a split. On one side, notations humans invented organically over centuries. On the other, a model imagining totally new ways of writing down ideas, not constrained by human language, human perception, or human physics. He thinks that is where things get interesting: using deep learning not just to model our world, but to speak ideas we don’t yet have words for.

Related TMFNK Content

Crepi il lupo! 🐺