Funny, no one had a problem with the Kennedys
However, John started to crack down on the Mafia. Some believe this had something to do with the assassination of JFK.
Certainly, Lee Harvey Oswald came from the land of Carlos Marcellos (Mafia king of New Orleans).
There were some who said the Mafia took a dim view of people who used the Mafia and them betrayed the Mafia.
I heard that Joseph Kennedy (senior) had his stroke when he heard of John cracking down on the Mafia.