Karpathy's New AI Trick: Ramble Your Idea, Then Build the World

A practical guide to voice-rambling an idea, shaping it into a scene graph, and letting an AI coding agent build and improve an interactive Three.js world.

Karpathy's New AI Trick: Ramble Your Idea, Then Build the World

TL;DR

  • AI can transform text into procedural 3D experiences.
  • Claude Opus 5 generated a Three.js world from a single paragraph of The Lord of the Rings.
  • The AI-generated world included low-poly objects, animated characters, and cameras.
  • Procedural worlds created by AI are presented as a beginning for further interaction and development, unlike static generated images.
  • A guide is available on turning voice rambles into world briefs for AI development.
  • The process involves building interactive scenes with Three.js, structuring with scene graphs, and using AI agents to test their own work.