His emotional address heralded a five-day debate on terrorism which should hear from 145 speakers.
He told delegates: "This vicious attack places in jeopardy the whole purpose of the United Nations.
"Terrorists deny the dignity of human life. Terrorism pray particularly on cultures and communities that practice openness, tolerance.
"Their targeting of innocent civilians mocks the efforts of those who seek to live together in peace as neighbours and defies the very notion of being a neighbour.
"Look at that destruction, that massive, senseless, cruel loss of human life and then I ask you to look in your hearts, and recognise that there is no room for neutrality on the issue of terrorism. You're either with civilization or with terrorists."
The debate is expected to focus primarily on a new international treaty to co-ordinate the campaign against terrorism.
The meetings come three days after the 15-nation Security Council adopted a sweeping resolution requiring all nations to crack down on terrorists and groups that help them.
Kofi Annan, UN Secretary-General said: "The task now is to build on that wave of human solidarity - to ensure that the momentum is not lost, to develop a broad, comprehensive and above all sustained strategy to combat terrorism and eradicate it from our world.
" This important meeting has a critical role to play in this. It must not be merely symbolic.
"It must signal the beginning of immediate, practical and far-reaching changes in the way this organisation and its member States act against terrorism."
ITN News, the broadcast, compared Giuliani to Churchill.
Sharpton has appeared nowhere in the British media.
Regards, Ivan