Never really permanently deleted. If you use an ISP (comcast) or cloud (yahoo) they could retrieve them in the case of a criminal investigation or something. But would never retrieve/restore if you asked. In case of a corporate environment (exchange server) they are backed up and sent offsite. So if you delete it locally, there is a copy saved somewhere. Plus the recipient of your email would have a copy on their client.
And the on the server the client’s email went through.
This is a horribly easy lie to bust.