I couldn't disagree more. Vocabulary is a sign of *reading*. Mostly in English, but anyone who knows Latin or another Western language understands enough to answer SAT questions.
The only "culture and upbringing" that affects this ability is the kind of "culture" that says it is unnecessary to read. And the people who want to drop the SATs are the ones who are supporting that "culture".