I crushed all 24, then built a test one or two.
Cantilevered castles sometimes stand up to large rocks but not bombs.
And sometimes large thick walls stand up to bombs but not big rocks.
Depends on the angle.
One of those you have to drop the rock into the hole onto the target.
I got up to Castle 13 or so, then away for the evening.