Besides the cap and ball ammo, the government also required that the weapon shoot at distances that exceeded the design. Ultimately, they increased the rifling and in concert with the powder change, that increased the cyclic rate, bore pressures, and powder residue. These bean counters and politicians killed more of our guys than any VC. The weapon that was fielded was completely different and the reliabity that sold it to the testers was gone.
There's a good link about the gunpowder debacle at The Gun Zone.
Cap and ball ammo? Odd choice for a 20th-century military rifle.
The problems are even more extensive than the examples you provided: too many separate teams involved with the specifications for equipment design (even multiple commands, services and agencies) for instance why would the Air Force and the army have to combine their requirements? Completely different missions.
Another problem is that civilians control the design and testing process. Wonderful, talented folks but no real idea about the environment and operational situation servicemen will have to face. Hence, the AR-15/M-16 works like a charm in their tests but fails miserably when our lives depended on it.
Here were issues we faced:
1. Because of the filthy-burning ammo, the M-16 tore case heads off and then double-fed.
2.There is no physical access to the chamber on the M-16, so only a field-strip and an assembled cleaning rod could clear the weapon.
3. The safeties stuck in the "Off" position on some weapons and could only be moved by using the butt of the bayonet or other objects (badly made detent).
4. The initial design of the sights was primitive and not intuitive (using a bullet tip to adjust them? Winding a front sight post up to move the strike of the bullet down?), causing many or most rounds to go somewhere other than intended. Saved a lot of VC.
5.The 55-grain round was not particularly effective on human beings. I saw people shot, dust flying off them, sometimes more than once and not go down. I didn't see the "catastrophic" effects they advertised.
6. The weapon was fragile: the stock shattered if you hit somebody with it. I actually saw a Marine shatter his stock, hitting a VC over the head with it.
I was lucky and smart: I turned in my M-16 to our armorer after I saw what was happening and retrieved my old M-14 and kept it for the remainder of my time in Vietnam. When I was finally wounded, my Lieutenant came up to me and said "Rick, I'm sorry you're hurt but can I have your rifle?".
Combat is not the place to test new stuff, particularly a very bad environment like Vietnam. I would love to have had the R&D types and the weapons manufacturers and the sales folks out with us for a week.