Was this a typo, or did you mean to say the 12 ga wouldn't work?
In my youth, I shot at a range with all manner of crap left on it - Dishwashers, Tv', etc. I remember hitting a cooking pot left by someone at 30 yards with a simple 2 3/4" hollowpoint lead slug. I figured it would pancake and dent the pot, as the pot was a much harder alloy and about 3/4"-1 inch thick on the bottom - but the slug blew clean through. With slugs today being saboted, and made of unique alloys that are harder than lead, and especially with the specialty rounds you can get, I would think a modern 3" or 3 1/2" saboted slug would be as effective as anything for head shots, if placed right. Not that I know anything about bear.
I remember outdoor life doing a piece on a guy who offed a Grizzly point blank with a .22LR pistol to the noggin. On the necropsy, out of twelve rounds or so, a whole bunch had found brain.
If the 12 ga doesn't work reliably, do you know why? Anatomical issue?
But the study I am referring to, determined that it was indeed the anatomy and the nature of the bear’s body structure that resists the 12 gage shotgun with slugs.
A Brown Bear's frontal muscle structure is layer upon layer of dense ligaments, sinew and muscle fiber, as well heavy bone. 12 gage slugs & Buckshot failed all penetration tests on actual bear carcases and skulls repeatedly.
The skull on a bear is also quite thick and is layered in much the same way as their frontal chest area. The 12 gage slug only dented the skull and did not fully penetrate it.
The 12 gage greatly lacks velocity and bullet performance to deal with Brown Bear sized game. Slugs are not jacketed and quickly flatten out on impact, which causes it to rapidly dissipate energy and mass. (Which is below standard in the first place.)
Common sense does not work here. Only physics. The low sectional density in relation to the diameter of the 12 gage slug is the key factor. It's sectional density is slightly only above 110, which is considered substandard for effective penetration in strong bodied animals like Brown Bear. They also deliver low impact energy of 1400 pounds and operate at low velocities of around 1200 fps.
Heavy rifle calibers like the 375 H&H, at the velocities they generate, translate into concentrated energy, plus controlled expansion in relation to a sectional density above 350 and stronger bullet design. Which is exactly why they will bring an animal the size and strength of a Brown Bear down in it's tracks. All that energy gets transferred deep into the animal's vital areas.
Muzzle velocity of a 350 grain bullet easily reaches 2400 fps, and delivers 5800 Foot pounds of bone cracking energy, and carries a sectional density of over 350.
I hope that explains it a bit better.