He got to her house to pick her up at 4pm. She wasn’t ready until 5pm. Their reservation for 5pm at the nicer restaurant (Aria?) was missed because she wasn’t ready on time. That’s probably what he meant…
He should have left her after waiting 30 minutes. That was his first clue.