Delay was strategic error. As was failure in getting to Baku oil fields, even if the soviets destroyed it on way out as this would have denied red army of oil.
I totally agree: If he had captured the oil supply first the game would have been over for the Russians.