I worked with various types of primates as a zookeeper.
Some of the macaques were aggressive. But other monkeys were quite friendly.
There was one large male chimpanzee that no one trusted. We once had to enter his area, and we had two men with 30-30s aimed at the tranquilized chimp in case he woke up.
The monkeys at the Taj had the run of the place. It is India, after all, with a monkey god in the Hindu pantheon. The one time I did look directly at one for more than a couple seconds the reaction was “What you lookin’ at! You lookin’ at me? You want a piece of me?”. I don’t doubt at all your experience with primates in a zoo. Merely saying that I’ve seen temple (in this case a tomb) monkeys in India behave like a biker gang on meth. And that when aroused they can do some serious harm.