I probably wont go out of my way visit Manhattan, but if I was there on business or was passing through and could grab a few extra days to stay, I would take it in.
Your point about crime is not untrue, but if you look at per capita states, you are more likely to suffer at the hands of violence in St.Louis or Detroit, and there are fewer reasons to visit those cities than NY.
This is my attitude.
I will never make it a point to visit (I was sent there in 2006 for work - only time I’ve been). But if there was some truly desirable event I wanted to attend there I would go.
I did enjoy my visit seeing some of the sites that are so familiar from movies and TV.
Even got to see filming of a Late Night w/David Letterman. Back when he was still funny and not a grouchy old curmudgeon.