Actually, my wife and I are almost 50 with no children. The other people we know who are moving are generally in our same general demographic situation. But, overall, you might be right. People are leaving for a variety of reasons, though.
I'm in my late 20s and the people I know who have left are in their late 20s through mid 30s. The difference in our perspectives probably reflects the limited segment of the population I know; of course I'm going to see more people my age and not be aware of people who are older.