pajama brigade software proposal

[drangundsturm]

Just brainstorming here. As we all have seen, the "pajama brigade" has proved to be able to expose major media errors and obvious bias. But, what if the freepers were even more organized? Could software take this fact-checking beyond the blog?

Consider the kind of software used by Project Gutenberg. They scan a public domain book and have software that can feed out the OCR results one page at a time, along with an image of the scanned page. Thousands of volunteers proof read the OCR output and submit corrections. If a person only has time to proof one single page, no problem. Each page is proofed by multiple people to gaurd against poor proofing or malicious people who might purposely introduce errors. The software tracks all of it.

Consider a web site, (pajamabrigade.com? or maybe just a new part of freerepublic.com) that does something similar, but instead of proof reading text, that site organizes volunteers to fact check media stories in a more formal and organized way. Some brainstorming ideas:

• A moderator could post a hot news story and highlight facts within that story that need checking. The facts to be checked are doled out by software, randomly, to volunteer fact-checkers. Volunteers check each fact, then document their methods of checking and conclusions (how they checked, what the veracity conclusion they came to). Each fact is handed out to multiple fact checkers to ensure accuracy. The moderator then scores how well each fact checker did (or maybe this is automatic, how well each fact checker's conclusions matched the majority who checked the fact in question). More reliable and thorough fact checkers earn a higher rating, and their conclusions become weighted more heavily when making determinations. People who proove unreliable are eventually excluded from the system. Maybe the moderator rates each fact as to how difficult it might be to check and people who check harder facts get more 'points', something like that.

• The site could have cool stats like how many facts were checked today (or this week, this month, etc.) how many factual errors were found, how many retractions were published by red-faced news media, etc. etc. The site could score media publications on how many errors were found in their stories. It could be very cool. (Hey, why not give fact-checkers one extra point if they certify that they were in their pajamas at the time they checked the fact ;)

• There could be forums and tutorials on "how to check facts", lists of fact checking resources, FAQs, etc., to help novice fact checkers to come up to speed. There could be lists of volunteer experts registered in the system for different topics requiring expert opinions. (The credentials of these experts could also be fact checked by the very same system.) Imagine a journalism student getting a real-world education this way. This would be an invaluable resource in and of itself.

• The eventual goal could be to fact-check every single major media story within 72 hours using an army of thousands of volunteers. Some volunteers may only decide to check one fact per week, others may check several per day. No long term commitment required, just like the gutenberg system for proofing text.

All that would be needed would be some perl scripts to organize the system, and a database to track the results. A simple prototype would take a few clever programmers a week to put together, and the sophistication of the system could grow over time until it probably surpasses my wildest dreams.

A system like this could be the next logical step in revolutionizing media reporting. It would be revolutionary in the same way that Linux revolutionized software, with armies of organized volunteers competing neck and neck with billion dollar corporate giants.

[drangundsturm]

Submitted for your approval.

[Aunt Polgara]

Don't forget, the well-dressed pajama brigade isn't complete without the required bunny slippers!

[narby]

It'd probably work. Technically of course it would, but the trick would be to organize the volunteers.

It may be that Freeper types are like Senators. As hard to herd as cats.

Let's see what JimRob thinks. Might not fit within FR, but then again, how else would the job get done?

[tang-soo]

With Apologies to Randy Newman

Pajama People

Pajama People got no reason
Pajama People got no reason
Pajama People got no reason
To live

They got little stubby fingers - little blurry eyes
They protest all day - postin' great big lies

They got snotty noses and coffee-stained teeth
They wear week-old underwear on their nasty little seat.

Well, I don't want no Pajama People
Don't want no Pajama People
Don't want no Pajama People `Round here

Pajama People are not the same as you and I (A Fool Such As I)
Good folk are liberals 'til the day they die (It's A Wonderful World)

Pajama People got nobody
Pajama People got nobody
Pajama People got nobody
To love

They got little beady eyes - and a furrowed brow
You gotta cuff their ears - just to say hello

They got little protest horns
That go beep, beep, beep

They're always saying
"Cheney's our veep, veep, veep!"

They got nasty little toes
hangin' from jammys below
We gotta make sure - it's their time to go.

Well, I don't want no Pajama People
Don't want no Pajama People
Don't want no Pajama People
'Round here

[HAL9000]

It's an interesting concept. Fortunately, I don't know perl.

[Jack of all Trades]

Interesting idea: distributed fact checking ala SETI At Home. You would need to reduce or remove the need for the moderator to vet the incoming data. In SETI at Home, the software runs a standard algorithm, and the results are accepted as true. In your system there would need to be an automatic feedback mechanism to weight the validity of the accepted data.

[militantmama]

Is that not essentially what free republic and bloggers already do?

[drangundsturm]

It may be that Freeper types are like Senators. As hard to herd as cats.

The system could use all the same techniques to encourage participation as sites like ebay and projects like gutenberg. Fact-checkers are rewarded by earning points for each fact they successfully check. On reaching a high enough point total, they obtain a title, such as "Senior Verification Analyst". Fact checkers will actually compete to see who can do a better job and obtain higher ratings. The site can list the top scorers for the month, year, etc., and list when people get "promoted" to the next level. Special awards can be conferred for breaking a major story.

At a certain level the person qualifies to be a moderator, with a title like "Bureau Chief", "Senior Bureau Chief", etc. Believe me, this can be a powerful motivator. Programmers now routinely put linux and gnu project experience on their resumes, and employers pay serious attention to that kind of experience. College kids majoring in journalism would flock to this kind of opportunity to get experience. Others could contribute as subject matter experts who consult with fact-checkers on a when-needed basis.

[drangundsturm]

Is that not essentially what free republic and bloggers already do?

It is, except I am proposing a much more organized approach that could be more thorough, and could gather statistics about accuracy of each media outlet, something that would be difficult in the current blogger world.

A system like this would encourage more people to check more facts and give them resources to help them learn how to do it. By organizing all the facts that need to be checked, it becomes more convenient for people to contribute.

Imagine if you could click a button and 5 facts were presented for you to check. You only have 10 minutes to contribute today, so you pick a fact that you think you can verify in 10 minutes. You search a couple of online resources, maybe make a phone call, and indeed you have verified that the fact is either true or false. You type in a couple of sentences explaining how you verified it, and click a submit button. Done. The next day in your email you find that the fact you checked was also confirmed by 3 other people to the moderator's satisfaction, and you had the same result as them, so you score 10 points (plus 1 if you were in your pajamas). A system like this will make it painless for more people to participate in checking media facts, and reward them (in a way) for doing so.

Another idea: newbie fact checkers could be sometimes given test questions that are already known to be true or false, and could be rated that way in a completely automatic way.

[militantmama]

The media already fears the blogosphere. Right now we are just citizens at our home computers pointing out curiosities and inconsistencies, sharing opinions and information. They really cannot touch us. I think half the appeal of the blogger watchdogs is their extreme patriotism. If we formed any type of concerted effort that was documented as such, they would accuse of us censorship. This would make us look like hypocrites because what we stand for is preserving the principles upon which this nation was founded. Censorship was not one of those principles, and while I may hate most of what they say, and know within my own heart that the liberal media spins one lie after another, our forefathers intended that they be able to do so without censorship. Besides, they do a pretty good job of hanging themselves. To some degree we need to allow them to do that or this country will never see them for what they are.

[drangundsturm]

I understand your point that further organization might invite fresh attacks from the media, but I don't see how it could be viewed as censorship. We don't have the power to shut down or censor their stories, and I don't suggest we do. In fact, since the checks will occur *after* each story is published, how can it be censorship?

I am suggesting such a site could thoroughly fact check their stories and document abuse in a systematic way. The reaction would be, one would hope, that they would be more careful about checking their sources to avoid being embarrassed. If they know that each and every story will be subjected to scrutiny they'll think twice about using questionable sources.