OK, so I tried it last night based on your post and realized that Matt is truly a nice guy and it's easy to feel sorry for him. The others were pretty funny but at the end when Earl went and Matt took it so hard, well, the fun went and it was hard to watch.
Afterwards they replayed the second episode and I think I would have enjoyed seeing that one first.