It's the "to whatever degree" that is important. His concern with Pearl's defenses are rather minor and really on the level that would be expected from a commander of a fleet. It is not a primary issue, and that's the point. The primary issue is the inability to place the fleet on a war footing due to the state of the facilities at Pearl Harbor (training, equipping, ect). The secondary concerns are cost and morale.
No disagreement here.
In their October 1940 meeting, Richardson advised FDR that his Pacific Fleet was too weak to be an effective deterrent, and the Japanese knew it.
He then asked if FDR intended to go to war with Japan, to which Roosevelt replied: that depends on where the Japanese attack.
There was then no evidence of an imminent Japanese attack, but the possibilities would naturally concern any military leaders.
But in Roosevelt's mind, some such attack was absolutely necessary to put America at war.
That's the only point I'm trying to make, for now. ;-)