Yes, except for the true cost of something is the cost of replacement, which has already jumped as soon as the price of oil jumps.
... so the price would come down right away on the downward side - cost of replacement applied. And all of those people waiting to fill their tanks would go to the station with the reduced price, just as you have said.