Because of the enormous amount of pressure.
Not the way you are thinking. Pressure readings early on during this disaster were showing 8,300 PSI at the base of the BOP. This location is before the restrictions of the BOP rams and bent riser on top of the BOP. When they attempted the top kill they created approx. 11,000 PSI backpressure which would subject the casing to the same. That casing was rated up to 10,300 PSI with some reserve.
So, they know they exceeded the pressure rating of the casing in that attempt so therefore they know they would exceed it by more with an attempted full shut-in.
Now, by cutting off the riser above the BOP and increasing the cross sectional area of the leak they also reduced the back pressure at the BOP. They know if they were to shut the well in, say with a valve on top of the BOP, that they would again exceed the pressure capability of the possibly damaged casing.
Look at it like your gardern hose with an adjustable valve on the end. Your water pressure with it off is 40 PSI, with is full open it is 20 PSI. It will vary according to the setting of the valve.