My son ref'd 5 yr old soccer. The parents were really angry when he chose not to call off sides on both teams. The kids had no idea what side they were playing on. They just wanted to chase the butterflies!
Swimming was a great sport for our kids. Both went on to play waterpolo in highschool, and my daughter played waterpolo and did crew in college.
Truely a family oriented sport.