The hardest part about using a nuclear reactor for space travel is making it light enough to launch, yet robust enough to survive an unplanned 'launch event.'
The New Horizons vehicle that went to Pluto uses (unironically) a plutonium power source for electricity but not for propulsion.