That should be the only criterion.
Yes, it may be difficult to lease the upper floors. Fine. Leave them empty and finance the added construction cost with donations or a federal handout.
Personally, I would pay to see the towers back and two stories higher. Every time I see an old movie located in New York and the camera shows the towers, it strengthens my resolve to rebuild.