I suspect the novelty simply wore off.
Cuba is a typical communist/socialist country. Not much to see, eat, hear, or experience. A tourist can have the same experience regarding European colonialism in the Carribean by visiting a dozen other countries or territories. Some of the others are even successful and civilized with modern conveniences!
Kennedy blew it. Just think what Cuba could have been today.