Or listening to voicemail.
Yeah, that's what I thought: listening to voicemails..if he was pressing buttons and then listening, that's probably what he was doing.
Also, since he's a delivery guy, some company's correspond with their delivery guys via phone, so it could have been company related.
Still "rude", but probably not unusual.