After September 11, 2001: NSA Begins Huge Data Mining Project Similar to ‘Total Information Awareness’

An illustration of the NIMD dataflow. [Source: LibertyThink.com] (click image to enlarge)Shortly after the 9/11 attacks, the NSA awards $64 million in research contracts for a program called Novel Intelligence from Massive Data (NIMD). [New York Times, 5/21/2003; National Journal, 1/20/2006] NIMD is one of several cutting-edge data mining technologies that not only has the capability of finding keywords among millions of electronically monitored communications, but can find hidden relationships among data points, and even critique the thinking and biases of a particular analyst and suggest alternative hypotheses differing from the human analysts’ conclusion. Like other data-mining technologies, the NSA will steadfastly refuse to discuss whether NIMD is used to analyze data from domestic surveillance operations. NIMD is designed as an preliminary sort program, to keep human analysts from becoming overwhelmed by raw data. In essence, NIMD is an early-warning system. “NIMD funds research to…help analysts deal with information-overload, detect early indicators of strategic surprise, and avoid analytic errors,” according to the “Call for 2005 Challenge Workshop Proposals” released by the Advanced Research and Development Activity (ARDA). ARDA was founded in 1998 to create, design, and field new technologies for US intelligence agencies, particularly the NSA. A selected few Congressional lawmakers (see January 18, 2006) were informed that the warrantless surveillance program authorized by President George W. Bush (see Early 2002) was designed to be an early-warning system for possible terrorist attacks or plans. Assistant Attorney General William Moschella will inform the top Democrats and Republicans on the House and Senate Intelligence committees in December 2002 that the “president determined that it was necessary following September 11 to create an early-warning detection system” to prevent more attacks. He will justify the use of programs such as NIMD by claiming, as NSA director Michael Hayden and other administration officials have repeatedly claimed, that the Foreign Intelligence Surveillance Act (FISA), which allows the government to obtain warrants to conduct domestic eavesdropping or wiretapping, “could not have provided the speed and agility required for the early-warning detection system.” Many experts outside of the Bush administration feel that NIMD and other programs do not have to operate outside of the Foreign Intelligence Surveillance Act (FISA) because of limitations in the law, but because of the fact that the programs cannot meet the law’s minimum requirements for surveillance. FISA requires that any such surveillance must have a probable cause that the target is a terrorist. NIMD has no such threshold. Steven Aftergood, an expert on intelligence and government secrecy with the Federation of American Scientists, will say in 2006, “Logistically speaking, the early-warning approach may involve a significant increase in the number of surveillance actions. It may be that neither the Justice Department nor the [Foreign Intelligence Surveillance Court, which approves wiretapping warrants] is prepared to prepare and process several thousand additional FISA applications per year, beyond the 1,700 or so approved in 2004.” [National Journal, 1/20/2006] Some experts will later express the opinion that NIMD is the controversial Total Information Awareness program in a slightly different form (see February 2003 and September 2002).