I did some contract work out at the naval base in Bayonne before the first Gulf War. Seeing all the tanks and heavy armor lined up and getting loaded onto ships day after day after day was pretty amazing. And New Jersey has lots of beautiful places - and I imagine Bayonne does too. However - Bayonne from what I saw was very much a factory and shipping port city.
Bayonne is changing rapidly. There's been a lot of redevelopment in Jersey City, mostly along the waterfront, recently and Bayonne is beginning to look like a lot of Jersey City used to. My husband grew up in a very nice enclave at the southern tip, and since we've been away the change is shocking even to me. The military terminal is now becoming a location shoot for movies/tv. Hubby tells me that early in the 20th century there were scores of bars along the east side that sailors would frequent. It's also amazing how people who grew up in Bayonne have such a deep affection for it. When I hear jokes about it, it's usually (but not always) from people who have no idea.
If all you saw of Bayonne was the area around the MOT I can understnad you impression. The residential section is very different. Broadway (the shopping district) has seen better days I have to admit, but that is common in the age of the mega malls. With the sudden rise of property values here in the last 5 years the area has really impoved.