Karpathy made me believe in maths
Growing up I knew that learning advanced math made no sense.
But now, simple math like back propagation… and this matters… no need to spend 5 years learning crazy proofs…can be used to make a robot move.
Yes, there is a lot more going on under the hood.
But getting started and reaching the frontier is possible in less than 6 months. And if you really grind or have a cs/math/technical background it’s possible in ~2 months.
And this is cool! For two reasons:
Math has a meaningful frontier again
You can get there in ~2 months
The question then is it actually valuable?
And the consensus answer here is yes.
It looks like: RL Loop → Real Robot → Data Capture on Robot → Defendable Data Moat
And if I had to bet the first people over the wall will be running on top of Isaac Sim & Warp. But Tesla with a custom hardware and software stack, top tier engineering talent, and deep pools of capital may also win.
But I like warp, specifically, because you need deformable dynamics to get object grasping right.
The basic heuristic I use is how useful can I be if I limit myself to using two deformable objects, aka thumb and forefinger grasping. And the answer is I can do a lot:
- Fold clothes, open/close doors, some (but not all) meal prep
So, the approach of building an entire articulated hand seems like overkill and solving the wrong problem.
The real unsolved challenge is a control problem.
But for now, math is cool again. And I’ll write more about other stuff (deformable objects, robotic control) in another post.