Farmers don't get a pass on having to compete. If one farmer can grow a crop more cost effectively than another, that farmer will prevail at the market...assuming each creates a product of equivalent quality and value. The less skilled, less efficient farmers will go out of business and be replaced by those who can do a better job.
The government sticks its nose and our tax money into agriculture all too often. Part of that is a consequence of our tremendous capacity to produce food that we export around the world. Just 50 years ago we expected seasonal variation in food availability. Today, there is cooperation between northern and southern hemispheres that provide a nearly continuous supply of foods that were once only available in the summer.
Frankly I buy from the farmer who sells his good from the corner of the local swap meat or farmer's market...
It's your money to spend as you see fit. It's a good thing that you don't have any desire for bananas or other tropical fruits. The local farmer isn't growing those items.
You make some valid points, although I think the small business owner farmer should be able to compete it today's market. Organic foods should be available and affordable rather than Mass produced chemical driven meats and vegetables but that is not the case.
IMHO that is why we have an obese problem, it's so inexpensive to eat unhealthy processed foods, yet to eat healthy is not affordable to many households.