Trouble at the ODP (Open Directory Project)

Web Pro News ^ | May 26, 2005 | Jim Hedger

[infocats]

The Open Directory Project is the largest human edited directory of web sites and documents existing online at this time. While many search engines such as Google, Yahoo, Ask and MSN maintain largër databases of electronically spidered sites, the volunteer editors at the ODP read, sort and classify all submitted content before it is added to their search-database. Started in 1998 in reaction to difficulties webmasters had getting their content into Yahoo's then human edited directory, the Open Directory Project was a simple and effective idea.

Founded in June 1998 by Rich Skrenta and Bob Truel, the ODP drew its early inspiration from the first major open-source cooperative initiative, the GNU Project. It was even originally named after the GNU project, launched as GnuHoo. The name was quickly changed to NewHoo in order to avoid confusion between the two projects. Over time, the NewHoo morphed into the more organized Open Directory Project. The ODP is owned and operated by AOL's Netscape division which has pledged to keep the directory 100% frëe as part of Netscape's social contract with web users.

Over the years, inclusion in the ODP became increasingly important, especially after Google began using it as the primary database for a Google directory. Getting a site listed at the ODP almost guaranteed a beneficial visit from Googlebot as a listing there was seen as a vote of confidence from a live-human reviewer.

For the past twenty-four months however, webmasters and search marketers have expressed extreme frustration while waiting for their sites to get listed in DMOZ. The Open Directory is a volunteer driven initiative, and like other non-paying projects they often have a hard time finding good help. Submissions to categories are backed up for months and in some cases, even years with many of the over 1500 unique directory categories lacking volunteers assigned to edit them. A backlog in sites awaiting review is one thing but recently, accusations of bribery, favouritism and editors lashing out at critics have caused many to lower their previously favourable estimations of the Open Directory.

The submission backlog, incidentally, grew so rapidly that the ODP editors opened a discussion forum known as the Resource Zone specifically to address questïons and concerns from webmasters. After operating for over a year, the collective of ODP editors that ran the Resource Zone elected to close down the most used service available on the forum, the Site Submission Zone. While the forum was established to discuss ODP issues in an open and public space, the Site Submission Zone took far too much energy to maintain and moderate. Editors felt it did not offer users enough relevant information as much of what could or perhaps should be said to site owners would fall into the confidential category.

For several months, there have been accusations that some ODP editors are accepting payments for faster attention. Stemming from the Blog, Corrupt DMOZ Editor which was started in December 2004 by DMOZ editor Ana Thema, the blog lists several entries detailing systemic corruption throughout the Open Directory editorial structure. In her February 8 posting, Ana states, "Links are a commodity. Links from DMOZ are a hot commodity. Everything in this world is a commodity: everything. If you disbelieve that someone would be so corrupt as to sell submissions into the ODP, then Dorothy, this is your wake up call." In other posts she claims she uses at least a dozen unique editor names and maintains a network with dozens of other ODP editors. Reading Ana Thema's blog is much like watching one's first episode of the corrupt-cop drama, The Shield .

Another issue critics have had with editors at the Open Directory Project is one of favouritism. Editors have almost total control over their sections of the directory. While there is a hierarchy of editors with Meta-Editors having the power to re-edit categories that have received complaints, most meta-editors don't have a lot of spare time. This has led to some "fixing" the listings to favour their friends and associates. There are stories of search engine marketers becoming editors at the ODP and then gently favouring sites that would benefit their clients. A more sophisticated story tells of a search engine marketer manipulating ODP results to generate stronger Google page-rank scores for his clients. Another tells of ODP editors networking with each other to provide reciprocal favours.

In a case of reverse favouritism, Ana Thema posted a story at corruptdmozeditor.com from another DMOZ editor that states, "My arch competitor had a dupe content subdomain that they set up for traffïc overflow and I changed their dmoz listing to the subdomain with duplicate content and it slaughtered their rankings for a couple of months. Speaking as someone with 4 years of sabotaging experience, switch their listing from www. to non-www from time-to-time. Switch them from www.example.com to www.example.com/index.html, stuff like that."

After complaining about abuses and neglect, some webmasters might expect an apology or a reasonable explanation from the Open Directory Project. None has been forthcoming though the Resource Zone was intended to be a space for DMOZ editors to communicate with DMOZ users. A growing problem for the ODP is the lack of patience users and editors are showing with each other in various search related discussion forums. A post over at the Search Engine Watch Forums likens the accountability of some ODP editors to Seinfeld's character, the Soup-Nazi. According to the post, criticize these editors and, "NO SOUP FOR YOU! NEXT!"

In its defense, the Open Directory Project is staffed by volunteers, all of whom are humans with real lives, real jobs and other responsibilities. Given the backlog of submissions and the deterioration of the directory, it is rather difficult to see them being able to straighten out the mess quickly or easily. While many DMOZ editors put up with a lot of abuse, almost all of them (with the possible exception of Ana Thema) take great pride in the size and scope of the Open Directory Project.

For search engine marketers however, the question of relevance vs. effort comes into play. At one time, a listing at the Open Directory was mandatory in order to guarantëe strong listings at Google, Yahoo and other search engines. Today, while still helpful, the strength of a Open Directory listing has been diluted by the search engines themselves. In an article titled, ...Time for The ODP to Close?, Search Engine Watch editor Danny Sullivan suggests three ways the venerable DMOZ could reorganize and revitalize itself. Whatever it does, it should do it soon as the importance of the largest human edited directory of websites is decreasing as quickly as the backlog of submitted sites is increasing.

[infocats]

As a parenthetical aside, about four years ago, I applied to become a volunteer editor at ODP but was summarily rejected for reasons I can't recall. - infocats

[JesseJane]

~mark

[Petronski]

The ODP is also known as DMOZ, an acronym for Directory Mozilla. This name reflects its loose association with Netscape's Mozilla project, an Open Source browser initiative. The ODP was developed in the spirit of Open Source, where development and maintenance are done by net-citizens, and results are made freely available for all net-citizens.

[Wuli]

So the next "phase" of this open directory project will be requests for federal taxpayer funding to support it? No dice. Those who use this "open" "free" technology can come up with the private $$$ means to fund it, or it shouldn't exist.