I believe that is part of it. There just aren't that many buildings in the area that could withstand those winds. I recall one picture beforehand of a church used as an evacuation center, with many families & children inside. The side walls were mostly large windows. It was enough to make one weep, hours before the storm hit...
Also, IIRC, PAGASA issued warnings late, in the opinion of many meteorologists who were monitoring the storm, and the upgrades to "Signal Level 4" (the highest level of warning), and the storm surge warnings were even later. PAGASA didn't even mention the storm until it was in their "area of responsibility", and then, with it being a relatively fast moving storm, time was short.
I was monitoring the storm and my wife and I were frantically trying to get word out to relatives and friends in the area, via her Facebook account, e-mails, etc.
Another problem is that there is no word in Tagalog, Visayan, or the local dialects, that I or my (Filipina - Visayan) wife know of, for "storm surge". Usually typhoons in the Philippines do not create large storm surges, so people don't really understand what might happen. On a meteorological forum I was monitoring (I post there occasionally), one US (I think) member was discussing the approaching storm with a Filipino member, and the Filipino said he was reasonably sure he'd be ok, as his house was fairly solid, and not in a (rain-caused) flood prone area. But he didn't understand the threat from the storm surge until the US member said "like a tsunami". Then the Filipino freaked out, but that was only a few hours before the Typhoon hit, and the Filipino and his family had nowhere better to flee to. We have not heard from him since, but hopefully that is just a downed communications issue.