Sadly, that is indeed how many view Christianity--as a big NO to all the fun, usually sexual, but not only. Many have lost their faith due to this "egoism." It makes me think of that first egoist who uttered non serviam so long ago.
I see joy in many people who are religious and even in those who have suffered tragedies in their life.