The Media and Don't Ask, Don't Tell Part 2

Critique of the original report | 12/06/ 2010 | Irish Catholic

[IrishCatholic]

While I have been here since 2003. This is my first post. Wouldn't you know it was a vanity?

The following post is my review of the DADT report available on the Defense Department web site. The review is quite lengthy so it is broken up into four parts. Those parts are listed in the first comment section of each post. I wrote the review late at night and after work, so please forgive any errors. It may seem to be all about the DADT report, but what it is really about is how the press portrayed the report, and how the report is now viewed in the eyes of millions of Americans.

[IrishCatholic]

Full text of part 2 is included below.

[IrishCatholic]

Part 2. The Findings from the Survey Report.

We aren’t quite ready to delve into the report just yet, even though we are two thousand words into the indictment of the press coverage of the report. The first thing to do is look at the numbers. After all, the press touted the troops were overwhelmingly comfortable with people suffering from same sex attraction serving alongside; let’s look at the numbers that support that confidant sound bite:

According to Appendix A (A1), there are 3,247,934 active service members in the active duty and reserves. The survey was sent out to 399,856 members, and the sample total for the report was 115,052 responders. However, only 109,973 completed the full survey. Percentage wise that is:
• Surveyed 12.3% of the total service members.
• Sample used 3.5%
• Actually completed survey 3.3%

So while a statistician might say this is all in a day’s work, the layman might say that only ten out of a hundred was asked to fill out the survey, and only three out of a hundred did. On what one would assume is a socially important topic, why would the military have such a poor rate of response? If you listen to Gates and Mullen it is about the standard response rate and nothing out of the ordinary. You would think that is something the press might consider as a follow up to see if they are telling the truth or are lying through their politically polished teeth.

What is available, if one wants to ask, is the fact that each survey had a unique number attached to it. Which means that if you filled out the survey, even though they say it was an ‘anonymous’ survey, they have your number and know which was your survey and how you filled it out. Name or number, it doesn’t make a difference. They knew who filled out the questions. The soldiers aren’t fools. No one who is in a life and death occupation can afford to be naive. Any trooper or NCO who got his survey or was going to fill one out online noted this little fact. Then, reading the questions and how they were constructed, (We’ll get to that later.), knew full well that his answers could come back to destroy his career later. No wonder over 70% chose to chuck the survey in the trash. You don’t play with your career when you think the fix is in. How eager was the press to talk to some of the people that took the survey? Any interest in whether it was filled out honestly?

Also found in Appendix D (D-62) is this little nugget of information on who is taking the survey. 34% of the survey participants classify themselves in the “warfare community.” 65.6% classify themselves as “Not warfare community.” The non combat troops were weighted almost two to one against combat troops in their participation in the survey. Once again the nabobs and pundits can toss that off as standard operating procedures. Compared to the total size of the military those troops actively engaged in combat might show that even 34% is oversampling in statistician land. The only problem is that it is in these combat units the repeal of Don’t ask, don’t tell might hit the hardest. After all, it is one thing to be living off base in New Jersey and having a gay coworker, and it might be quite another to have one sharing a bunker with you in Afghanistan for months at a time.

Here is the introduction of the survey given to the military. It already shows that Don’t ask, don’t tell is on the way out. The question is, how badly is it going to hurt the military?

Dear Service Member,
You and about 400,000 other active duty and reserve personnel have been selected at random to participate in an important confidential survey that will help shape the future of our military.

As you know, in his State of the Union address this year, President Obama called upon Congress to repeal the law commonly known as “Don’t Ask, Don’t Tell.” Following that, I directed the Department of Defense to consider how to best implement a repeal of this law should that occur. This survey is part of that effort. Your responses to this survey will help us assess the impact of a change in the Don’t Ask, Don’t Tell law and policy on military readiness, effectiveness, and unit cohesion. We need you to participate. The survey is confidential and will enable you to be straightforward and candid in your responses.

Your Responses to this survey are voluntary and strictly confidential. Only our survey support contractor staff will have access to the surveys, and no data will be disclosed to anyone that could be used to identify individual participants.

Thanks for your time and for your great service to our Nation.

Robert Gates
Secretary of Defense

Oh, let me add a post script to that intro. According to Admiral Mullen, Chairman of the Joint Chiefs of Staff, if you don’t like what’s coming who cares about your great service. Get the hell out.

Just to bring everyone back on point. This critique isn’t really about the report. It was written with an outcome in mind as was the survey and they worked backwards to get the answer they wanted. What the problem is, is the press didn’t bother to examine whether this was so or not. The same fourth estate that was all over the Bush Administration on every point of intelligence failure in wartime, couldn’t be bothered to read on the train ride home what might be a major disruption across the entire military for years to come. This critique is about what the press missed because they were lazy, incompetent, or they wanted to miss the fact this survey is crap because they share the same promotion of same sex disorder as does the President, Secretary Gates, and Admiral Mullen.

Ignoring the spin simply by glancing at the figures and charts in part one of the report should have showed that Gates and Mullen were shoveling manure. Think about it in the following way. The baseline on all the military readiness, morale, cohesion, effectiveness, etc. starts with zero change. This repeal is the effort to change that status quo. Now it can either have a positive effect on the military or it can have a negative effect on the military. A neutral effect basically means the troops will carry on regardless if it sucks and they hate it. It is this area that the planners of the survey knew they had to rely to fudge the numbers to make this move by the President anything other than one more liberal disaster. A major one at that. But let’s just consider the baseline of no change in the law. If that is what happens then everything goes along like it has. It is the repeal that changes everything. How did the survey figures in part one of the report show how 3.5% of the military feels?

Page 2 Figure 1 Expected Impact of DADT Repeal Across the Main Subject Areas

Unit Cohesion 19% Positive impact , 27% Negative impact
Effectiveness- Combat Deployed 12% Positive impact, 35% Negative impact
Effectiveness- Non Combat Deployed 17% Positive impact, 20% Negative impact
Readiness 7% Positive impact, 22% Negative impact
Morale 5% Positive impact, 28% Negative impact, and 11% Don’t know or won’t say
Retention 4% Positive impact, 24% Negative impact, and 11% Don’t know or won’t say

In every single category in the first table the negatives far outweigh the positives for repeal. This means by page two of the report it was plainly obvious that the repeal of DADT would have a clearly negative impact on the military according to its members. Yet that was blaringly missing from any mainstream reporting the day the report was released. No one so much as thought to call anyone on the numbers.

Page 11 Figure 2 Expected Impact of DADT Repeal Across Main Subject Areas of Spouse Survey

Family Readiness 1% Positive impact, 8% Negative impact
Attendance at informal military social events 1% Positive impact, 18% Negative impact
Attendance at Deployment support activities 1% Positive impact, 13% Negative impact
Attendance at family support programs 1% Positive impact, 15% Negative impact
Retention 3% Positive impact, 12% Negative impact, 11% Don’t know or won’t say
Referral 5% Positive impact, 18% Negative impact, 10% Don’t know or won’t say

It seems that in the spouse survey the repeal was no more acceptable or popular than it was with the members themselves. The negative impact of the repeal far outweighed any positive impact for the repeal. What was so hidden from the press? In both tables in the initial report the soldiers most emphatically did not portray they were comfortable with the repeal of DADT, even with an introduction to the survey telling them it was coming anyway and that their survey was anonymous, but here is your own personal secret code to enter to do the survey. Oh, and we mailed or emailed you the survey so we know who you are and what your number is.

Now the text of the first report is actually quite positive. They do leave enough caveats to say that although I have been calling them on it, they did say the negative outweighed the positives:
“ A majority of Service members perceive that the effect of a repeal of DADT will be neutral—that is, it will have either “no effect” or will affect their immediate unit “equally as positively as negatively.” A smaller, but still substantial, group said that repeal will affect their unit “very negatively/ negatively,” and an even smaller group said that repeal will affect their immediate unit “very positively/positively.” This pattern of responses holds true across all the major areas of interest, including unit cohesion, unit effectiveness (both for those who have been deployed to a combat zone and those who have not), personal and unit readiness, and personal morale.”

But they don’t really mean it. They proclaim that the vast majority of the members are, in fact neutral on the subject saying it is equally negative as positive. How interesting is that? How can something so volatile be both good and bad at the same time? Well, it’s all how you structure the questions and survey. There is nothing more ambiguous than giving an option of having something both equally good and bad. It takes a great section of the survey from the objective to the subjective and destroys what little value the survey had to begin with.

Skipping down to the bottom of page 19 of the report, is the interesting, and I’m sure standard scientific practice of “weighting” the results of the questions. This is where they, well, massage the data to compensate for what they term oversampling, response rate, and “demographic structure.” “Unweighted” results just give you the numbers who responded. Which numbers would you rather use? If you were the press and the numbers didn’t match up, wouldn’t you want to know how the statistician weighted the numbers? That is why percentages given in the report are, to me, are misleading. Only using the raw numbers can an accurate picture result. And, there are some questions in the survey where the majority of the respondents didn’t answer. This takes the already miniscule sample and shrinks it even more.

Now while the press is busy with everything other than their job. Three of the four Chiefs of Staff of the services have come out against the repeal of DADT. Army Chief of Staff General George Casey Jr. is against it. General James Amos, Commandant of the Marine Corps is against it. Air Force Chief of Staff General Norton Schwartz is against it. As we mentioned Admiral Mullen is already flying the rainbow flag from the mast so that takes care of the Navy, and finally Coast Guard Commandant Admiral Robert Papp Jr. is for the repeal. That final service, the Coast Guard, is actually under the Department of Homeland Security, not the Pentagon unless the department is transferred there in time of war. Operating as it is currently, it is odd that they were included in the Pentagon survey. But, I guess the people pushing the survey and the repeal needed to stack the deck all they could. However, the Coast Guard members are as unhappy about the prospect of repeal as is all the other services.

So as a non statistician, let’s look at the numbers in Part 3 and see if the questions and numbers match the hype. If you look at where the questions are going, you can see the attempt to influence the thinking of the person taking the survey. It is no wonder that the majority looked at it and walked away.