I have a buddy who’s an avid cyclist (I have to confess I was the world’s slowest Cat 3 for 15 years as well), who traveled to France in the 80’s and drove up to the Arenberg Forest section of the P-R course. He got his bike out of the car, took one look at the pave (a dry beautiful day) and put it back in.
I worked my way up over time through combined Cat4 and Masters 40+ 4/5. I was never a threat in road races with serious climbs because of my size, but I was always competitive in Cat4 Crits and ITT. My mates enjoyed that I could hammer in the wind.
I kind of think Cat3 is just a quick step for Pro1/2 riders to get in a better mix of competition and only guys over 35 stay there for awhile. In one of my early Cat3 circuit races I was completely spit out and abandoned so I didn't have to ride past the finish stand all alone. The humiliation sucked. I couldn't imagine not being able to suck wheels in a 60 rider bunch with no real climbing.
But there was a sliver lining. Our club had been working on bring more people into competition and we had some good younger riders who were capable of giving it a go. I was able to down grade to Cat5 and ride with them as a "coach on course". Those races were honestly the best times I had in USCF.