My family moved up here when my father retired from the military, and for a good portion of time after that, I never, ever went to the Cape.
I heard it was crowded, pain in the ass, touristy, not worth it, etc.
When I went to the Cape Cod National Seashore in late September one year, I walked up the beach and didn’t see another person for hours. It blew me away. I didn’t know there was anything like that left on the Eastern Seaboard.
My wife and I now go to the Cape, but we do so in late October/early November, or late March/early April.
Great place off season.
When our kids were little (maybe 20 years ago), we did a family trip to the cape in mid June as soon as school got out. It was wonderful. Not many people on the beaches. Now that they are all grown, we do as you do - vacations after Labor Day. Everything empties out. We were in the Sawtooth Mountains and Ketchum, ID a couple weeks back and the town was nearly empty as were the hiking trails and lakes. Snow on the nearby peaks was spectacular.