I've been using PGP for over a decade (actually GnuPG in recent years). Great stuff. 4096-bit keys. Go ahead and break it...
But if you send emails back and forth encrypted in PGP, the existence of your communications will be known, even if the content is not.
The FBI could get a search warrant, go to your house, copy your keystore, and install a keylogger on your computer. This would enable them to obtain your private key when you typed in the passphrase to your keystore.
Of course, there are ways to guard against this. How paranoid are you? Is your computer case secured by a lock? Do you store your keyboard in a safe when you are away?