Your HD in the old computer may still be functional. Itd be good to check.
I have the "luxury" of a small network here (one for everybody in the family!), so have a few spare parts, cannibalized machines, etc. hanging around. Plus knowing how they work allows me to recover from breakdowns, pretty quickly. Used Thinkpad T420 from ebay (200-250 bucks) is a snappy enough machine, and as other pointed out, if the screen or keyboard on it are not suitable, just hook it up to external monitor and keyboard.
I'd go the laptop route, stingy on power, portable if the urge hits, and can run through desktop like interface (keyboard, mouse, display) on the cheap.