Castro fears the fate of Rumania's dictator, Nicholae Ceausescu. Crowds began to spontaneously denounce him, and he was dead several DAYS later.
The key is the Cuban military. Ceausescu's fate was sealed when the army turned against him. If Castro is to be overthrown, there must be a rebellion in the Army. I don't see any signs that it will occur.