i see, survival of the smallest. So, according to evolution, people today find small brains more attractive, and hence tend to have more small-brained offspring with attractive small-brained mates. That's interesting. Or is it that having a smaller brain gives you an advantage in, say, Wall street, or the advertising industry, or maybe plumbing or accounting? Indeed a small brain would be of great advantage to the lucky ones who have it... though at the moment I can't think of why... perhaps my brain isn't small enough and I'm due for extinction. But it doesn't really have to be an advantage, does it? We can just say that it is, and that's good enough for evolution. Or we can say that it isn't even an advantage, and that's good enough for evolution.
What I said was that a smaller brain is not a disadvantage for survival in a society where most survival functions are performed at a level above the individual. This has nothing to do with selection of mates or anything else. For example, consider a group of nomadic tribesmen who need to devote 80% of their cognitive resources to survival (hunting, avoiding predators, finding shelter, making warm clothing, etc.), and their group is small enough that everyone must contribute to survival. In that group, children with smaller brains--for whom the 80% of the brain dedicated to survival is only equivalent to someone else's 40%--would not survive.
As humans have become more socialized, those survival functions became specialized to certain members of the group. For example, I don't raise and kill my own meat; I go to a butcher who kills the animals which, in turn, are raised by ranchers. Same thing with clothes: even if someone learns how to make clothes, she doesn't start the process by growing the cotton in her backyard--she buys the fabric already made. And so on. Since survival functions are now performed by specialized members of society, and each of those specialized members is responsible for only a tiny part of a survival function, it is no longer necessary for each person to maintain the brain power to survive individually or in small groups. Hence, average individual brain size has become smaller with no effect on species survival.
The other side of this coin is that larger brains have somewhat of a survival disadvantage. An infant with a large brain might have trouble exiting the birth canal--not a problem with today's medicine, but as little as a century or so ago was a death sentence for both mother and child. It takes more energy to maintain a larger brain, which can be a problem when food is scarce. With selective pressures to maintain a large brain eased because of relegation of survival functions to a large group rather than a handful of individuals, those with slightly smaller brains have a survival advantage.