To your average American, English, Irish, Aussie, Kiwi, South African, it all sounds alike. Since I find linguistics to be interesting, I can tell the difference. Not always, but usually. I have trouble with the South Africans though. There are plenty of Brits that say 'rine in spine." :*)
There is no doubt. I was quite deep into reading cultural experience websites before 9/11, and I have come across a few websites of New Zealanders who had lived or studied and/or travelled to America. One thing that upset them is that they found Americans think they are British by listening to their accents and can't tell they are from New Zealand. (At schools when English teachers teach about NZ English, they would have taught people find NZ and Australian English similar, so all New Zealanders expect people to confuse them with Australians)
And my colleagues at work are amazed by this statement as well, which I find...astonishing.