I had a look around at the state of the art in Linuxes that boot from USB and I come across a little gem: DSL USB Embedded.
Read more here
DSL USB Embedded under Vista with IE7 to prove it is Windows. It boots quick quickly too (less than a minute) which is much better than Ubuntu/Knoppix live CDs.
The USB key can also be set up to boot a pc straight into linux for some light Windows password resetting and it can have some persistant memory allocated to it should you not wish to cover your tracks.