If you can accelerate continuously, then you have to decelerate at some point -— starting at approximately the half-way point between the start and your destination. Ideally, you can just turn yourself around and point the rocket in the opposite direction.
Unless you can decelerate using some other means, like aero-braking.
Readers of Tintin books would recognize that exact scenario from the Explorers on the Moon volume.