Replies

Sounds like an awesome program, if only it could be used a lot more widely in our intel agencies! I know that Rep. Weldon says in his book he has been agitating for extensive intel data-mining and 'fusion' programs across all gov't classified databases since 1997-98....he says there are at least 32 major classified databases which are only accessible to their own internal searches. Of course, security and privacy are always big issues, and spy-moles like Aldrich Ames and Robert Hanssen made the various agencies guard their data even more tightly, I'd suspect.... AD only focused upon UNclassified data, as I understand it, but one of Weldon's main points is that data-mining needs to be done across all available classified and unclassified data so that all possible bits of info can be pulled togther.... e.g., someone like ATTA might have left tidbits of data in a dozen or more different databases, some classified and others unclassified..... [are Starlight and similar programs being used much at all for intel analysis and counter-terrorism since the demise of AD??]

One thing I worry about is the fact that names from Arabic, Persian, and a variety of other languages may be translated into English in a variety of ways, I believe. Is there software that can recognize numerous variants of the same name, and especially with all the abu-terrorist, al-bozo, el-scumbag, etc. variations that exist..... can intel agencies really search adequately, and are "watch lists" able to deal with such variations and also mis-spellings???

Starlight is still in the unclassified defense budget, so I would imagine it is still in the classified part as well. On the website, they state that Spire was developed for the Intel community, so I would imagine they are still using it also.
I think Lt. Col. Shaffer said they were using info from both classified and unclassified databases. Shaffer said they were using eight data sets to get the info, so it would have been probable the different variations of a name could be linked. It's been a long time since I've written a nested query, but they could put in all sorts of "where x = y" statements, look at the results and end up using good old gut instinct to weed out the different name spellings. In a less formal fashion, genealogists do this all the time.
But...notice the different results you get when you google "Tony Shaffer" (909 results) or google "Anthony Shaffer" (39,400 results). Now do an advance search within "Anthony Shaffer" results and add "Information Dominance Center" (404 results). Now add "MacDill" (3 results). All depends on that first search uncovered, doesn't it?

Here is a thread where the author tried to debunk AD based on the different spelling of Atta's name.

However, as I pointed out in post #40 in that specific case it wouldn't have mattered. Atta's name (whether he used Mohammed Atta or M el Amir) would have turned up anyway.