That was Warren and REM working as Hindu Love Gods. It’s an excellent blues / rock album from 1990. I think it’s hard to find now, but definitely pick it up or look for it online, because it’s some of his best stuff - mostly covers, but all arranged brilliantly and a very fun album.
Best known of which appears to be “Raspberry Beret”.