Searching for a smaller Linux distribution I came across Damn Small Linux and installed that. Well, installed is the wrong word as I first ran it from the LiveCD.
Read more here
The down side, and this is something I am still fighting with, is that it is not a true Debian distribution. It has its own packaging and distribution system. DSL's author had to make significant changes to the Debian core to make a workable distribution that fit on a business-card CD. DSL is intended to be a system for recovering a broken box.