Screen History MCP Server
This weekend I hacked together a simple tool to track what I've worked on.
If you want just want to try it, the repo is here.
The flow is: 1 screenshot / minute => OCR => ChromaDB => MCP server
Frontend is tested in Claude and allows me to ask questions like:
Yes, Rewind, does a great job.
But I was hoping for:
- something simple and free
- an excuse to explore MCP
A few notes on what I learned:
1. MCP Means I don't have to build a frontend
This is actually the third time I've tried to create this software. Two years ago I built a web app that did processing in the cloud and a year later I built a Mac Desktop app. Both took about a week and neither one worked very well.
Now the data and the formulation of how the data is provided to the frontend is what matters. The heavy lifting of U.I. interface design and conversation tool calling / retry logic / etc. is left to larger companies to deal with.
This changes product development roadmaps -- READ NO MORE FRONTEND -- and leads to a consolidation of software surface where people go to do work. Now, you need one good frontend client and a bunch of useful backend MCP servers.
2. Programming is fun again
About halfway through this project I realized I needed a node server instead of a python server. This change would have been half a day and massively demotivating. Now, it's a simple prompt in cursor and done in five minutes.
The amount of time I spend redoing work I've done is directly proportional to how much I want to stop.
3. Shifting work from me doing the work to me discribing the work
Task Lists in .md format help Cursor Agent do work. And then I can do something else or...
4. Context Switching Becomes More Important
Now, I can queue up 15 minutes of work by writing a long task doc and then switch to writing up another long queue of work for another project. In an age where everyone gets free junior software engineers, now the bottleneck is human computational units.
Thanks for reading. Code is here.
- Joe