That would still be illegal. It's not only illegal for felons to *own* firearms, it's illegal for them to "possess" them, which includes borrowing one and just holding it (or taking it hunting).
Don't laugh, I heard Gordon Liddy say something to this effect when questioned about his post felon target shooting.
I haven't heard him admit to target shooting (since that too would be illegal for him to do), but I did hear him joke about how "I don't have any guns, but my wife has a lot of them, and she keeps them in the beside table on my side of the bed..."
What kind of felony does G-Lid have on his record?
Clinton, as he was leaving office, pardoned an Arkansas friend who was a felon - so that he could go deer hunting with his rifle.