Literally the lower bound is zero.
With friction approaching zero and speed approaching zero, you can get from point A to point B with energy expendeture approaching zero. All the rest is just details.
Assuming A and B are at exactly the same altitude, of course.
< }B^)
That doesn't even play in the vacuum of space. An object at rest has inertia. You must apply a force across a distance (energy) to move it. Remember that first day of physics class? F = mA. E = F X D. Remember that other silly law...an object in motion tends to remain in motion? Yes, you have to apply energy to stop the object at point B.
With friction approaching zero and speed approaching zero, you can get from point A to point B with energy expendeture approaching zero. All the rest is just details.
Only in fantasy land. It's the details that expose the misunderstanding. You shouldn't omit them.