Did you have a contract for the work you did? That kind of stuff is normally put into the contract. I do a lot of work for the military and our contracts specifically say that the military owns the rights and source materials that we produce.
If you contracted for the paper copies of the flyers alone, you aren’t obligated to provide your source files unless it’s specifically stipulated in your contract for service.
This can get really tricky. It depends on who owns the rights to the original photos that you used. It is also dependent on the specific language in the contract and your state and local laws.
This was an excellent learning experience for me, both from a design standpoint and a nuts 'n bolts standpoint. I really enjoyed it. I hope this latest request of his is not going to lead to bad blood.