That's pretty much my strategy for now. The turn signals were a safety issue, and the raised handlebars were for comfort, so those'll be the last things I restore to original.
BSA handlebars were a nightmare. My Thunderbolt was 100 percent stock, and after a 3/4 hour cruise I could barely feel my hands or wrists.
You're dead-on right about it being like commanding a jackhammer. I did summer construction work including jackhammers, and that's the best analogy to the BSA. IIRC it would abate as you reached 70 or 75 MPH, but then you were just begging for a speeding ticket.