They used to grow them in La Conchita, Calif., which is on the coast just east of Carpinteria and about 10 miles west of Ventura. The owner of the banana plantation there boasted that it was the only one in the continental US. The climate at that location was apparently just right.
They grew several varieties of bananas, and they were the tastiest bananas I have ever eaten. Unfortunately, they went out of business a few years ago--probably couldn't compete with the banana republics down south.
Wiki has a good description -
https://en.wikipedia.org/wiki/Banana_production_in_the_United_States
The key sentence -
If independent cultivation was counted towards a nations total of banana production, its estimated that the United States would rank at about the 4th or 5th country for banana production in the world.