The Dark Road from the Clipper Chip to PRISM Reveals 'Crypto Wars' Never Ended

Back in the 1990s, security researchers and privacy watchdogs were alarmed by government demands that hardware and software firms build "backdoors" into their products, the millions of personal computers and cell phones propelling communication flows along the now-quaint "information superhighway."

Never mind that the same factory-installed kit that allowed secret state agencies to troll through private communications also served as a discrete portal for criminal gangs to loot your bank account or steal your identity.

To make matters worse, instead of the accountability promised the American people by Congress in the wake of the Watergate scandal, successive US administrations have worked assiduously to erect an impenetrable secrecy regime backstopped by secret laws overseen by secret courts which operate on the basis of secret administrative subpoenas, latter day lettres de cachet.

But now that all their dirty secrets are popping out of Edward Snowden's "bottomless briefcase," we also know the "Crypto Wars" of the 1990s never ended.

Documents published by The Guardian and The New York Times revealed that the National Security Agency "actively engages the US and IT industries" and has "broadly compromised the guarantees that internet companies have given consumers to reassure them that their communications, online banking and medical records would be indecipherable to criminals or governments."

"Those methods include covert measures to ensure NSA control over setting of international encryption standards," The Guardian disclosed, along with "the use of supercomputers to break encryption with 'brute force', and--the most closely guarded secret of all--collaboration with technology companies and internet service providers themselves."

According to The New York Times, NSA "had found ways inside some of the encryption chips that scramble information for businesses and governments, either by working with chipmakers to insert back doors or by surreptitiously exploiting existing security flaws, according to the documents."

In fact, "vulnerabilities" inserted "into commercial encryption systems" would be known to NSA alone. Everyone else, including commercial customers, are referred to in the documents as "adversaries."

The cover name for this program is Project BULLRUN. An agency classification guide asserts that "Project BULLRUN deals with NSA's abilities to defeat the encryption used in specific network communication technologies. BULLRUN involves multiple sources, all of which are extremely sensitive. They include CNE [computer network exploitation], interdiction, industry relationships, collaboration with other IC entities, and advanced mathematical techniques."

In furtherance of those goals, the agency created a "Commercial Solutions Center (NCSC) to leverage sensitive, cooperative relationships with industry partners" that will "further NSA/CSS capabilities against encryption used in network communications technologies," and already "has some capabilities against the encryption used in TLS/SSL. HTTPS, SSH, VPNs, VoIP, WEBMAIL, and other network communications technologies."

Time and again, beginning in the 1970s with the publication of perhaps the earliest NSA exposé by Ramparts Magazine, we learned that when agency schemes came to light, if they couldn't convince they resorted to threats, bribery or the outright subversion of the standard setting process itself, which destroyed trust and rendered all our electronic interactions far less safe.

Tunneling underground, NSA, telcos and corporate tech giants worked hand-in-glove to sabotage what could have been a free and open system of global communications, creating instead the Frankenstein monster which AT&T whistleblower Mark Klein denounced as a "Big Brother machine."

The Secret State and the Internet

Five years after British engineer Tim Berners-Lee, Belgian computer scientist Robert Cailliau and their team at CERN developed a system for assembling, and sharing, hypertext documents via the internet, which they dubbed the World Wide Web, in 1994 the Clinton administration announced it would compel software and hardware developers to install what came to known as the "Clipper Chip" into their products.

The veritable explosion of networked communication systems spawned by the mass marketing of easy-to-use personal computers equipped with newly-invented internet browsers, set off a panic amongst political elites.

How to control these seemingly anarchic information flows operating outside "normal" channels?

In theory at least, those doing the communicating--academics, dissidents, journalists, economic rivals, even other spies, hackers or "terrorists" (a fungible term generally meaning outsider groups not on board with America's imperial goals)--were the least amenable users of the new technology and would not look kindly on state efforts to corral them.

As new communication systems spread like wildfire, especially among the great unwashed mass of "little people," so too came a stream of dire pronouncements that the internet was now a "critical national asset" which required close attention and guidance.

President Clinton's Commission on Critical Infrastructure Protection released a report that called for a vast increase in funding to protect US infrastructure along with one of the first of many "cyberwar" tropes that would come to dominate the media landscape.

"In the cyber dimension," the report breathlessly averred, "there are no boundaries. Our infrastructures are exposed to new vulnerabilities--cyber vulnerabilities--and new threats--cyber threats. And perhaps most difficult of all, the defenses that served us so well in the past offer little protection from the cyber threat. Our infrastructures can now be struck directly by a variety of malicious tools."

And when a commercial market for cheap, accessible encryption software was added to the mix, security mandarins at Ft. Meade and Cheltenham realized the genie would soon be out of the bottle.

After all they reasoned, NSA and GCHQ were the undisputed masters of military-grade cryptography who had cracked secret Soviet codes which helped "win" the Cold War. Were they to be out maneuvered by some geeks in a garage who did not share or were perhaps even hostile to the "post-communist" triumphalism which had decreed America was now the world's "indispensable nation"?

Technological advances were leveling the playing field, creating new democratic space in the realm of knowledge creation accessible to everyone; a new mode for communicating which threatened to bypass entrenched power centers, especially in government and media circles accustomed to a monopoly over the Official Story.

US spies faced a dilemma. The same technology which created a new business model worth hundreds of billions of dollars for US tech corporations also offered the public and pesky political outliers across the political spectrum, the means to do the same.

How to stay ahead of the curve? Why not control the tempo of product development by crafting regulations, along with steep penalties for noncompliance, that all communications be accessible to our guardians, strictly for "law enforcement" purposes mind you, by including backdoors into commercially available encryption products.

Total Information Awareness 1.0

Who to turn to? Certainly such hush-hush work needed to be in safe hands.

The Clinton administration, in keeping with their goal to "reinvent government" by privatizing everything, turned to Mykotronx, Inc., a California-based company founded in 1983 by former NSA engineers, Robert E. Gottfried and Kikuo Ogawa, mining gold in the emerging information security market.

Indeed, one of the firm's top players was Ralph O'Connell, was described in a 1993 document published by Computer Professionals for Social Responsibility (CPSR) as "the father of COMSEC" and the "Principle NSA Technical Contact" on Clipper and related cryptography projects.

A 1993 Business Wire release quoted the firm's president, Leonard J. Baker, as saying that Clipper was "a good example of the transfer of military technology to the commercial and general government fields with handsome cost benefits. This technology should now pay big dividends to US taxpayers."

It would certainly pay "big dividends" to Mykotronx's owners.

Acquired by Rainbow Technologies in 1995, and eventually by Military-Industrial-Surveillance Complex powerhouse Raytheon in 2012, at the time the Los Angeles Times reported that "Mykotronx had been privately held, and its owners will receive 1.82 million shares of Rainbow stock--making the deal worth $37.9 million."

The Clipper chip was touted by the administration as a simple device that would protect the private communications of users while also allowing government agents to obtain the keys that unlocked those communications, an early manifestation of what has since become know as law enforcement's alleged "going dark" problem.

Under color of a vague "legal authorization" that flew in the face of the 1987 Computer Security Act (CSA), which sought to limit the role of the National Security Agency in developing standards for civilian communications systems, the administration tried an end-run around the law through an export ban on Clipper-free encryption devices overseen by the Commerce Department.

This wasn't the first time that NSA was mired in controversy over the watering down of encryption standards. During the development of the Data Encryption Standard (DES) by IBM in the 1970s, the agency was accused of forcing developers to implement changes in the design of its basic cipher. There were strong suspicions these changes had weakened the algorithm to such a degree that one critical component, the S-box, had been altered and that a backdoor was inserted by NSA.

Early on, the agency grasped CSA's significance and sought to limit damage to global surveillance and economic espionage programs such as ECHELON, exposed by British and New Zealand investigative journalists Duncan Campbell and Nicky Hager.

Before the 1987 law was passed however, Clinton Brooks, a Special Assistant to NSA Director Lieutenant General William Odom, wrote a Top Secret Memorandum which stated: "In 1984 NSA engineered a National Security Decision Directive, NSDD-145, through the Reagan Administration that gave responsibility for the security of all US information systems to the Director of NSA, removing NBS [National Bureau of Standards] from this."

Conceived as a follow-on to the Reagan administration's infamous 1981 Executive Order 12333, which trashed anemic congressional efforts to rein-in America's out-of-control spy agencies, NSDD-145 handed power back to the National Security Agency and did so to the detriment of civilian communication networks.

Scarcely a decade after Senator Frank Church warned during post-Watergate hearings into government surveillance abuses, that NSA's "capability at any time could be turned around on the American people, and no American would have any privacy left, such is the capability to monitor everything: telephone conversations, telegrams, it doesn't matter . . . there would be no place to hide," the agency was at it with a vengeance.

"This [NSDD-145] also stated," Brooks wrote, "that we would assist the private sector. This was viewed as Big Brother stepping in and generated an adverse reaction" in Congress that helped facilitate passage of the Act.

Engineered by future Iran-Contra felon, Admiral John Poindexter, President Reagan's National Security Adviser who would later serve as President George W. Bush's Director of DARPA's Information Awareness Office, the Pentagon satrapy that brought us the Total Information Awareness program, NSDD-145 stated that the "Director, National Security Agency is designated the National Manager for Telecommunications and Automated Information Systems Security."

NSA's new mandate meant that the agency would "act as the government focal point for cryptography, telecommunications systems security, and automated information systems security."

Additionally, NSA would "conduct, approve, or endorse research and development of techniques and equipment for telecommunications and automated information systems security for national security information."

But it also authorized the agency to do more than that, granting it exclusive authority to "review and approve all standards, techniques, systems and equipments for telecommunications and automated information systems security." As well, NSA was directed to "enter into agreements for the procurement of technical security material and other equipment, and their provision to government agencies, where appropriate, to private organizations, including government contractors, and foreign governments."

In other words, NSA was the final arbiter when it came to setting standards for all government and private information systems; quite a coup for the agency responsible for standing-up Project MINARET, the Cold War-era program that spied on thousands of antiwar protesters, civil rights leaders, journalists and members of Congress, as recently declassified documents published by the National Security Archive disclosed.

NSA Games the System

Although the Computer Security Act passed unanimously by voice vote in both Houses of Congress, NSA immediately set-out to undercut the law and did so by suborning the National Bureau of Standards, now the National Institute of Standards and Technology (NIST).

The battle over the Clipper Chip would be the template for future incursions by the agency for the control, through covert infiltration, of regulatory bodies overseeing civilian communications.

According to the Clinton White House, Clipper "would provide Americans with secure telecommunications without compromising the ability of law enforcement agencies to carry out legally authorized wiretaps."

Neither safe nor secure, Clipper instead would have handed government security agencies the means to monitor all communications while giving criminal networks a leg up to do the same.

In fact, as the Electronic Privacy Information Center (EPIC) discovered in documents unearthed through the Freedom of Information Act, the underlying algorithm deployed in Clipper, Skipjack, had been developed by NSA.

Cryptography expert Matt Blaze wrote a now famous 1994 paper on the subject before the algorithm was declassified, Protocol Failure in the Escrowed Encryption Standard: "The EES cipher algorithm, called 'Skipjack', is itself classified, and implementations of the cipher are available to the private sector only within tamper-resistant modules supplied by government-approved vendors. Software implementations of the cipher will not be possible. Although Skipjack, which was designed by the US National Security Agency (NSA), was reviewed by a small panel of civilian experts who were granted access to the algorithm, the cipher cannot be subjected to the degree of civilian scrutiny ordinarily given to new encryption systems."

This was precisely as NSA and the Clinton administration intended.

A partially declassified 1993 NSA memo noted that "there will be vocal public doubts expressed about having a classified algorithm in the device we propose for the US law enforcement problem, the CLIPPER chip, we recommend the following to address this." We don't know what those agency recommendations were, however; more than 20 years after the memo was written they remain secret.

The memo continued: "If such people agree to this clearance and non disclosure process, we could go over the algorithm with them to let them develop confidence in its security, and we could also let them examine the detail design of the CLIPPER chip made for the US law enforcement problem to assure themselves that there were no trapdoors or other techniques built in. This would likely require crypto-mathematicians for the algorithm examination and microelectronics chip design engineers for the chip examination."

But the extreme secrecy surrounding Skipjack's proposed deployment in commercial products was the problem. Even if researchers learned that Clipper was indeed the government-mandated backdoor they feared, non-disclosure of these facts, backed-up by the threat of steep fines or imprisonment would hardly assure anyone of the integrity of this so-called review process.

"By far, the most controversial aspect of the EES system," Blaze wrote, "is key escrow."

"As part of the crypto-synchronization process," Blaze noted, "EES devices generate and exchange a 'Law Enforcement Access Field' (LEAF). This field contains a copy of the current session key and is intended to enable a government eavesdropper to recover the cleartext."

"The LEAF copy of the session key is encrypted with a device-unique key called the 'unit key,' assigned at the time the EES device is manufactured. Copies of the unit keys for all EES devices are to be held in 'escrow' jointly by two federal agencies that will be charged with releasing the keys to law enforcement under certain conditions."

What those conditions were however, was far from clear. In fact, as we've since learned from Snowden's cache of secret documents, even when the government seeks surveillance authorization from the FISA court, the court must rely on government assurances that dragnet spying is critical to the nation's security. Such assurances, FISA court judge Reggie B. Walton noted, were systematically "misrepresented" by secret state agencies.

That's rather rich considering that Walton presided over the farcical "trial" that upheld Bush administration demands to silence FBI whistleblower Sibel Edmonds under the state secrets privilege. Edmonds, a former contract linguist with the Bureau charged that top FBI officials had systematically covered-up wrongdoing at its language division and had obstructed agents' attempts to roll-up terrorist networks before and after the 9/11 provocation, facts attested to by FBI whistleblower Coleen Rowley in her 2002 Memo to then-FBI Director Robert Mueller.

In 2009, Walton wrote that "The minimization procedures proposed by the government in each successive application and approved and adopted as binding by the orders of the FISC have been so frequently and systemically violated that it can fairly be said that this critical element of the overall BR regime has never functioned effectively."

"The Court," Walton averred, "must have every confidence that the government is doing its utmost to ensure that those responsible for implementation fully comply with the Court's orders. The Court no longer has such confidence."

Predating those critical remarks, a heavily-redacted 1993 Memo to then-Special Assistant to the President and future CIA chief, George Tenet, from FBI Director William Sessions noted that NSA "has developed a new encryption methodology and computer chip which affords encryption strength vastly superior to DES [Digital Encryption Standard], yet which allows for real time decryption by law enforcement, acting pursuant to legal process. It is referred to as 'Clipper'."

[Two redacted paragraphs] "if the devices are modified to include the 'Clipper' chip, they would be of great value to the Federal, state and local law enforcement community, especially in the area of counter narcotics, investigations, where there is a requirement to routinely communicate in a secure fashion."

But even at the time Sessions' memo was written, we now know that AT&T provided the Drug Enforcement Administration "routine access" to "an enormous AT&T database that contains the records of decades of Americans' phone calls," The New York Times reported, and had done so since 1987 under the auspices of DEA's Hemisphere Project.

Furthermore, in the wake of Snowden revelations we also learned that listening in on the conversations of drug capos is low on NSA's list of priorities. However, programs like X-KEYSCORE and TEMPORA, which copies all data flowing along fiber optic cables, encrypted and unencrypted alike, at petabyte scales, is supremely useful when it comes to building profiles of internet users by intelligence agencies.

This was an implicit goal of Clinton administration maneuvers to compel developers to insert Clipper into their product designs.

According to Sessions, "the 'Clipper' methodology envisions the participation of three distinct types of parties." [Redacted] It is proposed that the second party, the two custodians of the 'split' key infostructure [sic], be comprised of two disinterested and trustworthy non law enforcement Government agencies or entities. Although, such decision and selection are left for the Administration, a list of reccommended [sic] agencies and entities has been prepared (and included in the text), [redacted]. This party would administer and oversee all facets of the 'Clipper' program and methodology."

Based on NSDD-145's mandate, one can assume "this party" would be NSA, the agency that designed the underlying algorithm that powered Clipper.

The Sessions memo averred: "The Clipper chip provides law enforcement access by using a special chip key, unique to each device. In the AT&T TSD 3600, a unique session key is generated, external to the Clipper chip for each call."

"This session key," the memo explained, "is given to the chip to control the encryption algorithm. A device unique 'chip key' is programmed into each Clipper at the time of manufacture. When two TSD 3600s go to secure operation, the device gives out its identification (ID) number and the session key encrypted in its chip key."

Underlining a key problem with Clipper technology Sessions noted, "Anyone with access to the chip key for that identified device will be able to recover the session key and listen to the transmission simultaneously with the intended receiver. This design means that the list of chip keys associated with the chip ID number provides access to all Clipper secured devices, and thus the list must be carefully generated and protected. Loss of the list would preclude legitmate [sic] access to the encrypted information and compromise of the list could allow unauthorized access."

In fact, that "anyone" could include fabulously wealthy drug gangs or bent corporations with the wherewithal to buy chip keys from suborned government key escrow agents!

Its ubiquity would be a key selling-point for universal deployment. The memo explained, "the NSA developed chip based 'Clipper' solution works with hardware encryption applications, such as those which might be used with regard to certain telecommunications and computers devices," which of course would allow unlimited spying by "law enforcement."

Such vulnerabilities built into EES chip keys by design not only enabled widespread government monitoring of internet and voice traffic, but with a few tweaks by encryption-savvy "rogues" could be exploited by criminal organizations.

In his 1994 paper Blaze wrote that "a rogue system can be constructed with little more than a software modification to a legal system. Furthermore, while some expertise may be required to install and operate a rogue version of an existing system, it is likely that little or no special skill would be required to install and operate the modified software."

"In particular," Blaze noted, "one can imagine 'patches' to defeat key escrow in EES-based systems being distributed over networks such as the Internet in much the same way that other software is distributed today."

In the intervening years since Blaze observed how easy it would be to compromise key escrow systems by various bad actors, governments or criminals take your pick, the proliferation of malware powered botnets that infect hundreds of thousands of computers and smart phones every day--for blanket surveillance, fraud, or both--is a fact of life.

It didn't help matters when it emerged that "escrow agents" empowered to unlock encrypted communications would be drawn from the National Institute of Standards and Technology and the Automated Services Division of the Treasury Department, government outposts riddled with "No Such Agency" moles.

As EPIC pointed out, "Since the enactment of the Computer Security Act, the NSA has sought to undercut NIST's authority. In 1989, NSA signed a Memorandum of Understanding (MOU) which purported to transfer back to NSA the authority given to NIST."

The MOU required that NIST request NSA's "assistance" on all matters related to civilian cryptography. In fact, were NIST and NSA representatives on the Technical Working Group to disagree on standards, the ultimate authority for resolving disputes would rest solely with the Executive Branch acting through the President, the Secretary of Defense and the National Security Council, thus undercutting the clear intent of Congress when they passed the 1987 Computer Security Act.

EPIC noted: "The memorandum effectively returned to NSA many of the powers rejected by the Computer Security Act. The MOU contained several key goals that were to NSA's benefit, including: NSA providing NIST with 'technical security guidelines in trusted technology, telecommunications security, and personal identification that may be used in cost-effective systems for protecting sensitive computer data;' NSA 'initiating research and development programs in trusted technology, telecommunications security, cryptographic techniques and personal identification methods'; and NSA being responsive to NIST 'in all matters related to cryptographic algorithms and cryptographic techniques including but not limited to research, development, evaluation, or endorsement'."

A critique of the Memorandum in 1989 congressional testimony by the General Accounting Office (GAO) emphasized: "At issue is the degree to which responsibilities vested in NIST under the act are being subverted by the role assigned to NSA under the memorandum. The Congress, as a fundamental purpose in passing the act, sought to clearly place responsibility for the computer security of sensitive, unclassified information in a civil agency rather than in the Department of Defense. As we read the MOU, it would appear that NIST has granted NSA more than the consultative role envisioned in the act."

Five years after the GAO's critical appraisal, NSA's coup was complete.

"In 1994," EPIC noted, "President Clinton issued Presidential Decision Directive (PDD-29). This directive created the Security Policy Board, which has recommended that all computer security functions for the government be merged under NSA control."

Since PDD-29 was issued matters have only gotten worse. In fact, NIST is the same outfit exposed in Snowden documents published by The Guardian and The New York Times that allowed NSA to water down encryption and build backdoors into the Dual EC DRBG standard adopted by the Institute in 2006.

"Eventually, NSA became the sole editor."

Besieged by widespread opposition, the Clinton administration was out maneuvered in the court of public opinion and by 1996 had abandoned Clipper. However, this proved to be a pyrrhic victory for security-minded researchers and civil libertarians as we have since learned from Edward Snowden's revelations.

Befitting a military-intelligence agency, the dark core of America's deep state, NSA was fighting a long war--and they were playing for keeps.

US Cyber Command: Documents Reveal Pentagon Launching Covert Cyber Attacks

In 2008, the Armed Forces Journal published a prescient piece by Colonel Charles W. Williamson III, a staff judge advocate with the Air Force Intelligence, Surveillance and Reconnaissance Agency at Lackland Air Force Base in Texas, the National Security Agency listening post focused on intercepting communications from Latin America, the Middle East and Europe.

Titled "Carpet bombing in cyberspace," Col. Williamson wrote that "America needs a network that can project power by building an af.mil robot network (botnet) that can direct such massive amounts of traffic to target computers that they can no longer communicate and become no more useful to our adversaries than hunks of metal and plastic. America needs the ability to carpet bomb in cyberspace to create the deterrent we lack."

While Williamson's treatise was fanciful (a DDoS attack can't bring down an opponent's military forces, or for that matter a society's infrastructure), he had hit upon a theme which Air Force researchers had been working towards since the 1980s: the development of software-based weapons that can be "fired" at an adversary, potentially as lethal as a bomb dropped from 30,000 feet.

Two years later, evidence emerged that US and Israeli code warriors did something far more damaging.

Rather than deploying an "af.mil" botnet against Iran's civilian nuclear infrastructure at Natanz, they unleashed a destructive digital worm, Stuxnet. In the largest and most sophisticated attack to date, more than 1,000 centrifuges were sent spinning out of control, "no more useful" to Iranian physicists "than hunks of metal and plastic."

A line had been crossed, and by the time security experts sorted things out, they learned that Stuxnet and its cousins, Duqu, Flame and Gauss, were the most complex pieces of malware ever designed, the opening salvo in the cyberwar that has long-guided the fevered dreams of Pentagon planners.

'Plan X'

Today, that destructive capability exists under the umbrella of US Cyber Command (USCYBERCOM), one which has the potential of holding the world hostage.

Last year the Pentagon allocated $80 million dollars to defense giant Lockheed Martin for ongoing work on the National Cyber Range (NCR), a top secret facility that designs and tests attack tools for the government.

Under terms of the five year contract, Lockheed Martin and niche malware developers have completed work on a test-bed housed in a "specially architected sensitive compartmented information facility with appropriate security protocols" that "emulates the public internet and other networks, and provides for the modeling of cyber attacks."

Originally developed by the Defense Advanced Research Projects Agency (DARPA), the Pentagon's geek squad, NCR has gone live and was transitioned last year to the Office of the Secretary of Defense, federal contracts uncovered by NextGov revealed.

As Antifascist Calling reported back in 2009, "NCR will potentially serve as a new and improved means to bring America's rivals to their knees. Imagine the capacity for death and destruction implicit in a tool that can . . . cause an adversary's chemical plant to suddenly release methyl isocynate (the Bhopal effect) on a sleeping city, or a nuclear power plant to go supercritical, releasing tens of billions of curies of radioactive death into the atmosphere?"

NextGov also reported that the "Pentagon is seeking technology to coordinate and bolster cyberattack capabilities through a funding experiment called 'Plan X,' contract documents indicate."

A notice from DARPA's Information Innovation Office (I2O) informs us that "Plan X is a foundational cyberwarfare program to develop platforms for the Department of Defense to plan for, conduct, and assess cyberwarfare in a manner similar to kinetic warfare. Towards this end the program will bridge cyber communities of interest from academe, to the defense industrial base, to the commercial tech industry, to user-experience experts." (emphasis added)

Although DARPA claims "Plan X will not develop cyber offensive technologies or effects," the program's Broad Agency Announcement, DARPA-BAA-13-02: Foundational Cyberwarfare (Plan X), explicitly states: "Plan X will conduct novel research into the nature of cyberwarfare and support development of fundamental strategies needed to dominate the cyber battlespace. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems."

The document also gives notice that DARPA will build "an end-to-end system that enables the military to understand, plan, and manage cyberwarfare in real-time" as an "open platform architecture for integration with government and industry technologies."

The Military & Aerospace Electronics web site reported that DARPA has "chosen six companies so far to define ways of understanding, planning, and managing military cyber warfare operations in real-time, large-scale, and dynamic networks."

Collectively worth some $74 million, beneficiaries of taxpayer largesse include "Data Tactics Corp. in McLean, Va.; Intific Inc. in Peckville Pa.; Raytheon SI Government Solutions in Arlington, Va.; Aptima Inc. in Woburn Mass.; Apogee Research LLC in McLean, Va.; and the Northrop Grumman Corp. Information Systems segment in McLean, Va."

Additional confirmation of US government plans to militarize the internet were revealed in top secret documents provided by former NSA contractor-turned-whistleblower Edward Snowden. Those documents show that the Pentagon's goal of "dominating cyberspace" are one step closer to reality; a nightmare for privacy rights and global peace.

Such capabilities, long suspected by security experts in the wake of Stuxnet, are useful not only for blanket domestic surveillance and political espionage but can also reveal the deepest secrets held by commercial rivals or geostrategic opponents, opening them up to covert cyber attacks which will kill civilians if and when the US decides that critical infrastructure should be been switched off.

Before a cyber attack attack can be launched however, US military specialists must have the means to tunnel through or around security features built into commercial software sold to the public, corporations and other governments.

Such efforts would be all the easier if military specialists held the keys that could open the most secure electronic locks guarding global communications. According to Snowden, NSA, along with their corporate partners and private military contractors embarked on a multiyear, multibillion dollar project to defeat encryption through the subversion of the secure coding process.

Media reports published by Bloomberg Businessweek, The Wall Street Journal and The Washington Post, also revealed that US intelligence agencies are employing "elite teams of hackers" and have sparked "a new arms race" for cyberweapons where the "most enticing targets in this war are civilian--electrical grids, food distribution systems, any essential infrastructure that runs on computers," Businessweek noted.

Confirming earlier reporting, The Washington Post disclosed that the US government "carried out 231 offensive cyber-operations in 2011, the leading edge of a clandestine campaign that embraces the Internet as a theater of spying, sabotage and war, according to top-secret documents" provided by Snowden to the Post.

Since its 2009 stand-up as a "subordinate unified command" under US Strategic Command (USSTRATCOM), whose brief includes space operations (military satellites), information warfare (white, gray and black propaganda), missile defense, global command and control, intelligence, surveillance and reconnaissance (ISR), as well as global strike and strategic deterrence (America's first-strike nuclear arsenal), Cyber Command has grown from 900 personnel to a force that will soon expand to more than "4,900 troops and civilians," The Washington Post reported earlier this year.

Under the USSTRATCOM umbrella, the organization is comprised of "Army Cyber Command (ARCYBER); Air Forces Cyber (AFCYBER); Fleet Cyber Command (FLTCYBERCOM); and Marine Forces Cyber Command (MARFORCYBER)."

"The Command," according to a 2009 Defense Department Fact Sheet, "is also standing up dedicated Cyber Mission Teams" that "conduct full spectrum military cyberspace operations in order to enable actions in all domains, ensure US/Allied freedom of action in cyberspace and deny the same to our adversaries."

The Defense Department Memorandum authorizing it's launch specified that the Command "must be capable of synchronizing warfighting effects across the global security environment as well as providing support to civil authorities and international partners."

In written testimony to the Senate Armed Services Committee during 2010 confirmation hearings, NSA head General Keith Alexander agreed, and The New York Times reported that Cyber Command's target list would "include civilian institutions and municipal infrastructure that are essential to state sovereignty and stability, including power grids, banks and financial networks, transportation and telecommunications."

But what various "newspapers of record" still fail to report is that the deliberate targeting of civilian infrastructure are war crimes that cause catastrophic loss of life and incalculable suffering, as US attacks on the former Yugoslavia, Iraq and more recently, Libya, starkly demonstrate.

In a portrait of Alexander published earlier this summer by Wired, James Bamford noted that for years the US military has "been developing offensive capabilities, giving it the power not just to defend the US but to assail its foes. Using so-called cyber-kinetic attacks, Alexander and his forces now have the capability to physically destroy an adversary's equipment and infrastructure, and potentially even to kill."

While the specter of a temporary "interruption of service" haunt modern cities with blackout or gridlock, a directed cyberattack focused on bringing down the entire system by inducing widespread technical malfunction would transform "the vast edifices of infrastructure" into "so much useless junk," according to urban geographer Stephen Graham.

In Cities Under Siege, Graham discussed the effects of post-Cold War US/NATO air bombing campaigns and concluded that attacks on civilian infrastructure were not accidental; in fact, such "collateral damage" was consciously designed to inflict maximum damage on civilian populations.

"The effects of urban de-electrification," Graham wrote, "are both more ghastly and more prosaic: the mass death of the young, the weak, the ill, and the old, over protracted periods of time and extended geographies, as water systems and sanitation collapse and water-borne diseases run rampant. No wonder such a strategy has been called a 'war on public health,' an assault which amounts to 'bomb now, die later'."

A further turn in US Cyber Command's brief to plan for and wage aggressive war, was telegraphed in a 2012 Defense Department Directive mandating that autonomous weapons systems and platforms be built and tested so that humans won't lose control once they're deployed.

There was one small catch, however.

According to Deputy Secretary of Defense Ashton Carter, a former member of the Board of Trustees at the spook-connected MITRE Corporation, the Directive explicitly states it "does not apply to autonomous or semi-autonomous cyberspace systems for cyberspace operations."

Presidential Policy Directive 20: Authorizing 'Cyber-Kinetic' War Crimes

We now know, based on documents provided by Edward Snowden, that President Barack Obama "has ordered his senior national security and intelligence officials to draw up a list of potential overseas targets for US cyber-attacks," according to the 18-page top secret Presidential Policy Directive 20 published by The Guardian.

Though little commented upon at the time due to the avalanche of revelations surrounding dragnet domestic surveillance carried out by NSA, in light of recent disclosures by The Washington Post on America's bloated $52.6 billion 2013 intelligence budget, PPD-20 deserves close scrutiny.

With Syria now in Washington's crosshairs, PPD-20 offers a glimpse into Executive Branch deliberations before the military is ordered to "put steel to target."

The directive averred that Offensive Cyber Effects Operations (OCEO) "can offer unique and unconventional capabilities to advance US national objectives around the world with little or no warning to the adversary or target and with potential effects ranging from subtle to severely damaging."

These are described in the document as "cyber effects," the "manipulation, disruption, denial, degradation, or destruction of computers, information or communications systems, networks, physical or virtual infrastructure controlled by computers or information systems, or information resident thereon."

To facilitate attacks, the directive gives notice that "cyber collection" will entail "Operations and related programs or activities conducted by or on behalf of the United States Government, in or through cyberspace, for the primary purpose of collecting intelligence--including information that can be used for future operations--from computers, information or communications systems, or networks with the intent to remain undetected."

Such clandestine exercises will involve "accessing a computer, information system, or network without authorization from the owner or operator of that computer, information system, or network or from a party to a communication or by exceeding authorized access."

In fact, PPD-20 authorizes US Cyber Command to "identify potential targets of national importance where OCEO can offer a favorable balance of effectiveness and risk as compared with other instruments of national power."

Indeed, the "directive pertains to cyber operations, including those that support or enable kinetic, information, or other types of operations . . . that are reasonably likely to result in 'significant consequences'" to an adversary.

We are informed that "malicious cyber activity" is comprised of "Activities, other than those authorized by or in accordance with US law, that seek to compromise or impair the confidentiality, integrity, or availability of computers, information or communications systems, networks, physical or virtual infrastructure controlled by computers or information systems, or information resident thereon."

In other words, if such activities are authorized by the President acting as Commander-in-Chief under the dubious "Unitary Executive" doctrine, like Richard Nixon, Obama now claims that "when the President does it that means that it is not illegal," a novel reading of the US Constitution and the separation of powers as it pertains to declaring and waging war!

"Military actions approved by the President and ordered by the Secretary of Defense authorize nonconsensual DCEO [Defensive Cyber Effects Operations] or OCEO, with provisions made for using existing processes to conduct appropriate interagency coordination on targets, geographic areas, levels of effect, and degrees of risk for the operations."

This has long been spelled out in US warfighting doctrine and is fully consistent with the Pentagon's goal of transforming cyberspace into an offensive military domain. In an Air Force planning document since removed from the web, theorists averred:

Cyberspace favors offensive operations. These operations will deny, degrade, disrupt, destroy, or deceive an adversary. Cyberspace offensive operations ensure friendly freedom of action in cyberspace while denying that same freedom to our adversaries. We will enhance our capabilities to conduct electronic systems attack, electromagnetic systems interdiction and attack, network attack, and infrastructure attack operations. Targets include the adversary's terrestrial, airborne, and space networks, electronic attack and network attack systems, and the adversary itself. As an adversary becomes more dependent on cyberspace, cyberspace offensive operations have the potential to produce greater effects. (Air Force Cyber Command, "Strategic Vision," no date)

Those plans were made explicit in 2008, when the Air Force Research Lab issued a Broad Agency Announcement entitled Dominant Cyber Offensive Engagement and Supporting Technology, BAA-08-04-RIKA.

Predating current research under "Plan X" to build "an end-to-end system that enables the military to understand, plan, and manage cyberwarfare in real-time," the earlier notification solicited bids from private military contractors to build cyberweapons.

We learned that the Air Force, now US Cyber Command, the superseding authority in the realm of cyberweapons development, a mandate made explicit in PPD-20, was "interested in technology to provide the capability to maintain an active presence within the adversaries information infrastructure completely undetected. Of interest are any and all techniques to enable stealth and persistence capabilities on an adversaries infrastructure."

"This could be a combination of hardware and/or software focused development efforts."

"Following this," the solicitation read, "it is desired to have the capability to stealthily exfiltrate information from any remotely-located open or closed computer information systems with the possibility to discover information with previously unknown existence."

While the United States has accused China of carrying out widespread espionage on US networks, we know from information Snowden provided the South China Morning Post, that NSA and US Cyber Command have conducted "extensive hacking of major telecommunication companies in China to access text messages"; carried out "sustained attacks on network backbones at Tsinghua University, China's premier seat of learning"; and have hacked the "computers at the Hong Kong headquarters of Pacnet, which owns one of the most extensive fibre optic submarine cable networks in the region."

China isn't the only target of US industrial espionage.

Earlier this month, O Globo disclosed that "one of the prime targets of American spies in Brazil is far away from the center of power--out at sea, deep beneath the waves. Brazilian oil. The internal computer network of Petrobras, the Brazilian oil giant partly owned by the state, has been under surveillance by the NSA, the National Security Agency of the United States."

Top secret documents mined from the Snowden cache revealed that NSA employees are trained "step-by-step how to access and spy upon private computer networks--the internal networks of companies, governments, financial institutions--networks designed precisely to protect information."

In addition to Petrobras, "other targets" included "French diplomats--with access to the private network of the Ministry of Foreign Affairs of France--and the SWIFT network, the cooperative that unites over ten thousand banks in 212 countries and provides communications that enable international financial transactions. All transfers of money between banks across national borders goes through SWIFT," O Globo disclosed.

The 2008 Air Force solicitation stressed that the service was interested in "any and all techniques to enable exfiltration techniques on both fixed and mobile computing platforms are of interest. Consideration should be given to maintaining a 'low and slow' gathering paradigm in these development efforts to enable stealthy operation."

The Air Force however, was not solely interested in defense or industrial spying on commercial rivals; building offensive capabilities were viewed as a top priority. "Finally," the solicitation reads, "this BAA's objective includes the capability to provide a variety of techniques and technologies to be able to affect computer information systems through Deceive, Deny, Disrupt, Degrade, Destroy (D5) effects."

As Bloomberg Businessweek reported in 2011, recipients of that Broad Agency Announcement may have included any number of "boutique arms dealers that trade in offensive cyber weapons. Most of these are 'black' companies that camouflage their government funding and work on classified projects."

"Offensive Cyber Effects Operations" will be enhanced through the development and deployment of software-based weapons; the Obama administration's intent in PPD-20 is clear.

The US government "shall identify potential targets of national importance where OCEO can offer a favorable balance of effectiveness and risk as compared with other instruments of national power, establish and maintain OCEO capabilities integrated as appropriate with other US offensive capabilities, and execute those capabilities in a manner consistent with the provisions of this directive."

Evidence has since emerged these programs are now fully operational.

On the Attack: Economic, Political and Military 'Exploits'

Despite diplomatic posturing and much handwringing from the "humanitarian intervention" crowd, the Obama administration's itchy trigger finger is still poised above the attack Syria button.

The conservative Washington Free Beacon web site reported recently that US forces "are expected to roll out new cyber warfare capabilities during the anticipated military strike on Syria," and that the targets of "cyber attacks likely will include electronic command and control systems used by the Syrian military forces, air defense computers, and other military communications networks."

Whether or not that attack takes place, NSA and US Cyber Command are ramping-up their formidable resources and would not hesitate to use them if given the go-ahead.

This raises the question: what capabilities have already been rolled out?

"Under an extensive effort code-named GENIE, The Washington Post disclosed, "US computer specialists break into foreign networks so that they can be put under surreptitious US control."

According to top secret budget documents provided by Snowden, the Post revealed the "$652 million project has placed 'covert implants,' sophisticated malware transmitted from far away, in computers, routers and firewalls on tens of thousands of machines every year, with plans to expand those numbers into the millions."

"Of the 231 offensive operations conducted in 2011," the Post reported, "nearly three-quarters were against top-priority targets, which former officials say includes adversaries such as Iran, Russia, China and North Korea and activities such as nuclear proliferation. The document provided few other details about the operations."

As other media outlets previously reported, the Post noted that US secret state agencies "are making routine use around the world of government-built malware that differs little in function from the 'advanced persistent threats' that US officials attribute to China."

One firm featured in Bloomberg Businessweek's cyberwar exposé is Endgame Systems, which first gained notoriety as a result of the 2011 HBGary Federal hack by Anonymous.

The shadowy firm has received extensive funding from venture capitalists such as Bessemer Venture Partners, Columbia Capital, Kleiner Perkins Caufield & Byers and the intelligence-connected Paladin Capital Group.

Endgame is currently led by CEO Nathaniel Flick, previously the CEO of the "nonpartisan" Center for a New American Security (CNAS), a warmongering Washington think tank focused on "terrorism" and "irregular warfare."

Flick replaced Christopher Rouland, Endgame's founder and CEO in December 2012. A former hacker, Rouland was "turned" by the Air Force during the course of a 1990 investigation where he was suspected of breaking into Pentagon systems, Businessweek reported.

The Board of Directors is currently led by Christopher Darby, the President and CEO of the CIA's venture capital arm, In-Q-Tel. Earlier this year, the firm announced that Kenneth Minihan, a former NSA Director and managing partner at Paladin Capital had joined the Board.

According to Businessweek, Endgame specializes in militarizing zero-day exploits, software vulnerabilities which take months, or even years for vendors to patch; a valuable commodity for criminals or spooks.

"People who have seen the company pitch its technology," Businessweek averred, "say Endgame executives will bring up maps of airports, parliament buildings, and corporate offices. The executives then create a list of the computers running inside the facilities, including what software the computers run, and a menu of attacks that could work against those particular systems."

While the United States has accused the Technical Reconnaissance Bureau of China's People's Liberation Army of launching attacks and stealing economic secrets from US networks, American cyberoperations involve "what one budget document calls 'field operations' abroad, commonly with the help of CIA operatives or clandestine military forces, 'to physically place hardware implants or software modifications,'" according to The Washington Post.

"Endgame weaponry comes customized by region--the Middle East, Russia, Latin America, and China--with manuals, testing software, and 'demo instructions.'"

"There are even target packs for democratic countries in Europe and other US allies," Businessweek noted.

Readers will recall that Snowden documents have exposed how NSA has carried out widespread economic and political espionage against erstwhile "friends and allies" such as Brazil, France, Germany, India, the European Union and the United Nations.

Add to that list, Endgame exploits which are solely military in nature; in all probability these have been incorporated into NSA and US Cyber Command's repertoire of dirty tricks.

"Maui (product names tend toward alluring warm-weather locales) is a package of 25 zero-day exploits that runs clients $2.5 million a year," Businessweek reported. "The Cayman botnet-analytics package gets you access to a database of Internet addresses, organization names, and worm types for hundreds of millions of infected computers, and costs $1.5 million."

"A government or other entity could launch sophisticated attacks against just about any adversary anywhere in the world for a grand total of $6 million. Ease of use is a premium. It's cyber warfare in a box."

Sound familiar?

"An implant is coded entirely in software by an NSA group called Tailored Access Operations (TAO)," Snowden documents revealed. "As its name suggests, TAO builds attack tools that are custom-fitted to their targets," according to The Washington Post.

"The implants that TAO creates are intended to persist through software and equipment upgrades, to copy stored data, 'harvest' communications and tunnel into other connected networks" the Post disclosed.

"This year TAO is working on implants that 'can identify select voice conversations of interest within a target network and exfiltrate select cuts,' or excerpts, according to one budget document. In some cases, a single compromised device opens the door to hundreds or thousands of others."

This does much to explain why NSA's parallel, $800 million SIGINT Enabling Project stresses the importance of obtaining total global access and "full operating capacity" that can "leverage commercial capabilities to remotely deliver or receive information."

With "boutique arms dealers" and others from more traditional defense giants along for the ride, NSA and US Cyber Command hope their investment will help "shape the global network to benefit other collection accesses and allow the continuation of partnering with commercial Managed Security Service Providers and threat researchers, doing threat/vulnerability analysis."

"By the end of this year," the Post noted, "GENIE is projected to control at least 85,000 implants in strategically chosen machines around the world. That is quadruple the number--21,252--available in 2008, according to the US intelligence budget."

The agencies are now poised to expand the number of machines already compromised. "For GENIE's next phase, according to an authoritative reference document," the Post disclosed, "the NSA has brought online an automated system, code-named TURBINE, that is capable of managing 'potentially millions of implants' for intelligence gathering 'and active attack'."

It should be clear, given what we have learned from Edward Snowden and other sources, that the US government views the internet, indeed the entire planet, as a battlespace.

In congressional testimony earlier this year, General Alexander told the House Armed Services Committee that "Cyber offense requires a deep, persistent and pervasive presence on adversary networks in order to precisely deliver effects."

"We maintain that access, gain deep understanding of the adversary, and develop offensive capabilities through the advanced skills and tradecraft of our analysts, operators and developers."

With US Cyber Command fully funded and mobilized, those "offensive capabilities" are only a mouse click away.

'Big Data' Dynamo: How Giant Tech Firms Help the Government Spy on Us and Gut Privacy

As the secret state continues trawling the electronic communications of hundreds of millions of Americans, lusting after what securocrats euphemistically call "actionable intelligence," a notional tipping point that transforms a "good" citizen into a "criminal" suspect, the role played by telecommunications and technology firms cannot be emphasized enough.

Ever since former NSA contractor Edward Snowden began leaking secrets to media outlets about government surveillance programs, one fact stands out: The zero probability these privacy-killing projects would be practical without close (and very profitable) "arrangements" made with phone companies, internet service providers and other technology giants.

Indeed, a top secret NSA Inspector General's report published by The Guardian, revealed that the agency "maintains relationships with over 100 US companies," adding that the US has the "home field advantage as the primary hub for worldwide telecommunications."

Similarly, the British fiber optic cable tapping program, TEMPORA, referred to telcos and ISPs involved in the spying as "intercept partners." The names of the firms were considered so sensitive that GCHQ "went to great lengths" to keep their identities hidden, fearing exposure "would cause 'high-level political fallout'."

With new privacy threats looming on the horizon, including what CNET described as ongoing efforts by the FBI and NSA "to obtain the master encryption keys that Internet companies use to shield millions of users' private Web communications from eavesdropping," along with new government demands that ISPs and cell phone carriers "divulge users' stored passwords," can we trust these firms?

And with Microsoft and other tech giants, collaborating closely with "US intelligence services to allow users' communications to be intercepted, including helping the National Security Agency to circumvent the company's own encryption," can we afford to?

Hiding in Plain Sight

Ever since retired union technician Mark Klein blew the lid off AT&T's secret surveillance pact with the US government in 2006, we know user privacy is not part of that firm's business model.

The technical source for the Electronic Frontier Foundation's lawsuit, Hepting v. AT&T and the author of Wiring Up the Big Brother Machine, Klein was the first to publicly expose how NSA was "vacuuming up everything flowing in the Internet stream: e-mail, web browsing, Voice-Over-Internet phone calls, pictures, streaming video, you name it."

We also know from reporting by USA Today, that the agency "has been secretly collecting the phone call records of tens of millions of Americans" and had amassed "the largest database ever assembled in the world."

Three of those data-slurping programs, UPSTREAM, PRISM and X-KEYSCORE, shunt domestic and global communications collected from fiber optic cables, the servers of Apple, Google, Microsoft and Yahoo, along with telephone data (including metadata, call content and location) grabbed from AT&T, Sprint and Verizon into NSA-controlled databases.

But however large, a database is only useful to an organization, whether its a corporation or a spy agency, if the oceans of data collected can be searched and extracted in meaningful ways.

To the growing list of spooky acronyms and code-named black programs revealed by Edward Snowden, what other projects, including those in the public domain, are hiding in plain sight?

Add Google's BigTable and Yahoo's Hadoop to that list. Both are massive storage and retrieval systems designed to crunch ultra-large data sets and were developed as a practical means to overcome "big data" conundrums.

According to the Mountain View behemoth, "BigTable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers." Along with web indexing, Google Earth and Google Finance, BigTable performs "bulk processing" for "real-time data serving."

Down the road in Sunnyvale, Yahoo developed Hadoop as "an open source Java framework for processing and querying vast amounts of data on large clusters of commodity hardware." According to Yahoo, Hadoop has become "the industry de facto framework for big data processing." Like Google's offering, Hadoop enable applications to work with thousands of computers and petabytes of data simultaneously.

Prominent corporate clients using these applications include Amazon, AOL, eBay, Facebook, IBM, Microsoft and Twitter, among many others.

'Big Data' Dynamo

Who might also have a compelling interest in cataloging and searching through very large data sets, away from prying eyes, and at granular levels to boot? It should be clear following Snowden's disclosures, what's good for commerce is also a highly-prized commodity among global eavesdroppers.

Despite benefits for medical and scientific researchers sifting through mountains of data, as Ars Technica pointed out BigTable and Hadoop "lacked compartmentalized security" vital to spy shops, so "in 2008, NSA set out to create a better version of BigTable, called Accumulo."

Developed by agency specialists, it was eventually handed off to the "non-profit" Apache Software Foundation. Touted as an open software platform, Accumulo is described in Apache literature as "a robust, scalable, high performance data storage and retrieval system."

"The platform allows for compartmentalization of segments of big data storage through an approach called cell-level security. The security level of each cell within an Accumulo table can be set independently, hiding it from users who don't have a need to know: whole sections of data tables can be hidden from view in such a way that users (and applications) without clearance would never know they weren't there," Ars Technica explained.

The tech site Gigaom noted, Accumulo is the "technological linchpin to everything the NSA is doing from a data-analysis perspective," enabling agency analysts to "generate near real-time reports from specific patterns in data," Ars averred.

"For instance, the system could look for specific words or addressees in e-mail messages that come from a range of IP addresses; or, it could look for phone numbers that are two degrees of separation from a target's phone number. Then it can spit those chosen e-mails or phone numbers into another database, where NSA workers could peruse it at their leisure."

(Since that Ars piece appeared, we have since learned that NSA is now conducting what is described as "three-hop analysis," that is, three degrees of separation from a target's email or phone number. This data dragnet "could allow the government to mine the records of 2.5 million Americans when investigating one suspected terrorist," the Associated Press observed).

"In other words," Ars explained, "Accumulo allows the NSA to do what Google does with your e-mails and Web searches--only with everything that flows across the Internet, or with every phone call you make."

Armed with a "dual-use" program like Accumulo, the dirty business of assembling a user's political profile, or shuttling the names of "suspect" Americans into a national security index, is as now easy as downloading a song from iTunes!

And it isn't only Silicon Valley giants cashing-in on the "public-private" spy game.

Just as the CIA-funded Palantir, a firm currently valued at $8 billion and exposed two years ago as a "partner" in a Bank of America-brokered scheme to bring down WikiLeaks, profited from CIA interest in its social mapping Graph application, so too, the NSA spin-off Sqrrl, launched in 2012 with agency blessings, stands to make a killing off software its corporate officers helped develop for NSA.

Co-founded by nine-year agency veteran Adam Fuchs, Sqrrl sells commercial versions of Accumulo and has partnered-up with Amazon, Dell, MapR and Northrop Grumman. According to published reports, like other start-ups with an intelligence angle, Sqrrl is hoping to hook-up with CIA's venture capital arm In-Q-Tel.

Its obvious why the application is of acute interest to American spy shops. Fuchs told Gigaom that Accumulo operates "at thousands-of-nodes scale" within NSA data centers.

"There are multiple instances each storing tens of petabytes (1 petabyte equals 1,000 terabytes or 1 million gigabytes) of data and it's the backend of the agency's most widely used analytical capabilities."

Accumulo's analytical functions work because of its ability to perform lightning-quick searches called "graph analysis," a method for uncovering unique relationships between people hidden within vast oceans of data.

According to Forbes, "we know that the NSA has successfully tested Accumulo's graph analysis capabilities on some huge data sets--in one case on a 1200 node Accumulo cluster with over a petabyte of data and 70 trillion edges."

Considering, as Wired reported, that "on an average day, Google accounts for about 25 percent of all consumer internet traffic running through North American ISPs," and the Mountain View firm allowed the FBI and NSA to tap directly into their central servers as The Washington Post disclosed, the negative impact on civil rights and political liberties when systems designed for the Pentagon are monetized, should be evident.

Once fully commercialized, how much more intrusive will employers, marketing firms, insurance companies or local and state police with mountains of data only a mouse click away, become?

Global Panopticon

The sheer scope of NSA programs such as UPSTREAM, PRISM or X-KEYSCORE, exposed by the Brazilian daily, O Globo should give pause.

A crude illustration (at the top of this post), shows that all data collected in X-KEYSCORE "sessions" are processed in petabyte scale batches captured from "web-based searches" that can be "retrospectively" queried to locate and profile a "target."

This requires enormous processing power; a problem the agency may have solved with Accumulo or similar applications.

Once collected, data is separated into digestible fragments (phone numbers, email addresses and log ins), then reassembled at lightning speeds for searchable queries in graphic form. Information gathered in the hopper includes not only metadata tables, but the "full log," including what spooks call Digital Network Intelligence, i.e., user content.

And while it may not yet be practical for NSA to collect and store each single packet flowing through the pipes, the agency is already collecting and storing vast reams of data intercepted from our phone records, IP addresses, emails, web searches and visits, and is doing so in much the same way that Amazon, eBay, Google and Yahoo does.

As the volume of global communications increase each year at near exponential levels, data storage and processing pose distinct problems.

Indeed, Cisco Systems forecast in their 2012 Visual Networking Index that global IP traffic will grow three-fold over the next five years and will carry up to 4 exabytes of data per day, for an annual rate of 1.4 zettabytes by 2017.

This does much to explain why NSA is building a $2 billion Utah Data Center with 22 acres of digital storage space that can hold up to 5 zettabytes of data and expanding already existing centers at Fort Gordon, Lackland Air Force Base, NSA Hawaii and at the agency's Fort Meade headquarters.

Additionally, NSA is feverishly working to bring supercomputers online "that can execute a quadrillion operations a second" at the Multiprogram Research facility in Oak Ridge, Tennessee where enriched uranium for nuclear weapons is manufactured, as James Bamford disclosed last year in Wired.

As the secret state sinks tens of billions of dollars into various big data digital programs, and carries out research on next-gen cyberweapons more destructive than Flame or Stuxnet, as those supercomputers come online the cost of cracking encrypted passwords and communications will continue to fall.

Stanford University computer scientist David Mazières told CNET that mastering encrypted communications would "include an order to extract them from the server or network when the user logs in--which has been done before--or installing a keylogger at the client."

This is precisely what Microsoft has already done with its SkyDrive cloud storage service "which now has 250 million users worldwide" and exabytes of data ready to be pilfered, as The Guardian disclosed.

One document "stated that NSA already had pre-encryption access to Outlook email. 'For Prism collection against Hotmail, Live, and Outlook.com emails will be unaffected because Prism collects this data prior to encryption'."

Call the "wrong" person or click a dodgy link and you might just be the lucky winner of a one-way trip to indefinite military detention under NDAA, or worse.

What should also be clear since revelations about NSA surveillance programs began spilling out last month, is not a single ruling class sector in the United States--including corporations, the media, nor any branch of the US government--has the least interest in defending democratic rights or rolling-back America's emerging police state.

Documents Show Undersea Cable Firms Provide Surveillance Access to US Secret State

Documents published last week by the Australian web site Crikey revealed that the US government "compelled Telstra and Hong Kong-based PCCW to give it access to their undersea cables for spying on communications traffic entering and leaving the US."

The significance of the disclosure is obvious; today, more than 99 percent of the world's internet and telephone traffic is now carried by undersea fiber optic cables. An interactive submarine cable map published by the Global Bandwidth Research Service is illustrative in this regard.

Since the late 1960s as part of its ECHELON spy project, the United States has been tapping undersea cables to extract communications and signals intelligence. In fact, projects such as Operation Ivy Bells, a joint Navy-NSA secret intelligence program directed against the former Soviet Union was designed to do just that.

Prefiguring the Bush administration's warrantless wiretapping scandal which broke in 2005, the Associated Press reported that a $3.2 billion Navy Seawolf class submarine, a 453-foot behemoth called the USS Jimmy Carter, "has a special capability: it is able to tap undersea cables and eavesdrop on the communications passing through them."

A year later, AT&T whistleblower Mark Klein told Wired Magazine that NSA was tapping directly into the world's internet backbone, and was doing so from domestic listening posts the telecommunications' giant jointly built with the agency at corporate switching stations.

Whatever submarine operations NSA still carry out with the US Navy and "Five Eyes" surveillance partners (Australia, Britain, Canada, New Zealand and the US), access to information flowing through undersea cables mean that the US government is well-positioned to scoop-up virtually all global communications.

Since former NSA contractor Edward Snowden began spilling the beans last month, it should be clear that the American government's capabilities in amassing unprecedented volumes of information from cable traffic, also potentially hands the US and their corporate collaborators a treasure trove of sensitive economic secrets from competitors.

Economic Espionage

Reporting by Australian journalists confirm information published July 6 by The Washington Post. There we learned that overseas submarine cable companies doing business in the United States must maintain "an internal corporate cell of American citizens with government clearances," a cadre of personnel whose job is to ensure that "when US government agencies seek access to the massive amounts of data flowing through their networks, the companies have systems in place to provide it securely."

Inked just weeks after the 9/11 provocation, the 23-page Telstra document specifies that access to undersea cable traffic by the FBI and "any US governmental authorities entitled to effect Electronic Surveillance," is an explicit condition for doing business in the United States.

Similar agreements were signed between 1999 and 2011 with telecommunication companies, satellite firms, submarine cable operators and the US government and were published earlier this month by the Public Intelligence web site.

It has long been known that the Australian secret state agency, the Defence Security Directorate (DSD), is a key participant in US global surveillance projects. Classified NSA maps provided by Snowden and subsequently published by Brazil's O Globo newspaper, revealed the locations of dozens of US and allied signals intelligence sites worldwide. DSD currently operates four military installations involved in a top secret NSA program called X-Keyscore.

Snowden described X-Keyscore and other programs to Der Spiegel as "the intelligence community's first 'full-take' Internet buffer that doesn't care about content type . . . 'Full take' means it doesn't miss anything, and ingests the entirety of each circuit's capacity."

According to The Sydney Morning Herald, along with the "US Australian Joint Defence Facility at Pine Gap near Alice Springs," three other DSD facilities, "the Shoal Bay Receiving Station near Darwin, the Australian Defence Satellite Communications Facility at Geraldton and the naval communications station HMAS Harman outside Canberra," were identified as X-Keyscore "contributors." The paper also reported that "a new state-of-the-art data storage facility at HMAS Harman to support the Australian signals directorate and other Australian intelligence agencies" is currently under construction.

The Herald described the project as "an intelligence collection program" that "processes all signals before they are shunted off to various 'production lines' that deal with specific issues and the exploitation of different data types for analysis--variously code-named Nucleon (voice), Pinwale (video), Mainway (call records) and Marina (internet records). US intelligence expert William Arkin describes X-Keyscore as a 'national Intelligence collection mission system'."

Two of the Australian bases illustrated on the X-Keyscore map sit adjacent to major undersea cable sites transiting the Pacific and Indian Oceans.

Cozy arrangements with Telstra and other firms however, hardly represent mere passive acceptance of terms and conditions laid out by the US government. On the contrary, these, and dozens of other agreements which have come to light, are emblematic of decades-long US corporate-state "public-private partnerships."

As Bloomberg reported last month, "thousands of technology, finance and manufacturing companies are working closely with US national security agencies, providing sensitive information and in return receiving benefits that include access to classified intelligence."

It's a two-way street, Bloomberg noted. Firms providing "US intelligence organizations with additional data, such as equipment specifications" use it "to help infiltrate computers of its adversaries." In return, "companies are given quick warnings about threats that could affect their bottom line." Such sensitive data can also be used to undermine the position of their foreign competitors.

We now know, based on documents provided by Snowden, that the "infiltration" of computer networks by US secret state agencies are useful not only for filching military secrets and mass spying but also for economic and industrial espionage.

That point was driven home more than a decade ago in a paper prepared by journalist Duncan Campbell for the European Parliament.

"By the end of the 1990s," Campbell wrote, "the US administration claimed that intelligence activity against foreign companies had gained the US nearly $150 billion in exports."

"Although US intelligence officials and spokespeople have admitted using Comint [communications intelligence] against European companies . . . documents show that the CIA has been directly involved in obtaining competitor intelligence for business purposes."

At the time the Telstra pact was signed, the Australian telecommunications and internet giant was "50.1% owned" by the Australian government. Reach Global Services, is described in the document as "a joint venture indirectly owned 50% by Telstra" and "50% owned" by Hong Kong's Pacific Century CyberWorks Limited (PCCW).

With controlling interest in more than 40 undersea fiber optic cables, and with landing rights in global markets that include Hong Kong, Japan, Korea, Taiwan, Singapore, Australia, North America and Europe, the joint venture was then the largest commercial telecommunications carrier in Asia with some 82,000 kilometers of undersea cables. Reach also operates international satellite systems that cover two-third's of the planet's surface.

Such assets would be prime targets of "Five Eyes" spy agencies under terms of the UKUSA Communications Intelligence Agreement.

Telstra and PCCW restructured their partnership in 2011, with the Australian firm now controlling the lion's share of an undersea cable network that stretches "more than 364,000 kilometres and connects more than 240 markets worldwide," the South Morning China Post reported. Inevitably, the restructuring will afford the US government an even greater opportunity for spying.

Network security agreements hammered out among undersea cable firms and the US government have profound implications for global commerce. Their geopolitical significance hasn't been lost on America's closet "allies."

The Guardian revealed last month that the US is "spying on the European Union mission in New York and its embassy in Washington." In addition to the EU mission, target lists include "the French, Italian and Greek embassies, as well as a number of other American allies, including Japan, Mexico, South Korea, India and Turkey."

That list has since been supplemented by further disclosures.

Snowden told the South China Morning Post that NSA hacked into the "computers at the Hong Kong headquarters of Pacnet, which owns one of the most extensive fibre optic submarine cable networks in the region."

Recently, the firm signed major deals with the Chinese mainland's "top mobile phone companies" and "owns more than 46,000 kilometres of fibre-optic cables."

According to the paper, Pacnet "cables connect its regional data centres across the Asia-Pacific region, including Hong Kong, the mainland, Japan, South Korea, Singapore and Taiwan. It also has offices in the US."

The South Morning China Post also disclosed that Tsinghua University, "China's premier seat of learning" has sustained extensive attacks on the school's "network backbones."

Available documents based on Snowden disclosures and other sources seem to suggest that President Obama's militaristic "pivot to Asia" is also an aggressive campaign to steal commercial and trade secrets from US imperialism's Asian rivals.

Whether or not these revelations will effect negotiations over the proposed Trans-Pacific Partnership (TPP), a NAFTA-style "free trade" agreement between the US and ten Pacific Rim nations, including Chile, Japan, Malaysia, Mexico, Peru and Singapore--all prime US-UK targets of PRISM, TEMPORA and X-Keyscore--remains to be seen.

'Legal' License to Spy

If we have learned anything since Snowden's revelations began surfacing last month, it is that the US secret state relies on a body of "secret laws" overseen by a Star Chamber-like FISA court described in the polite language The New York Times as a "parallel Supreme Court," to do its dirty work.

Along with leaked NSA documents, published agreements between telecommunications firms, internet service providers and the US government should demolish the fiction that blanket surveillance is "legal," "limited in scope" or chiefly concerned with fighting "crime" and "terrorism."

Proclaiming that "US communications systems are essential to the ability of the US government to fulfill its responsibilities to the public to preserve the national security of the United States, to enforce the laws, and to maintain the safety of the public," the Telstra summary posted by Crikey should dispel any illusions on that score.

On the contrary, the agreement reveals the existence of a vast surveillance web linking private companies to the government's relentless drive, as The Washington Post explained, to "collect it all."

● All customer billing data to be stored for two years;
● Ability to provide to agencies any stored telecommunications or internet communications and comply with preservation requests;
● Ability to provide any stored metadata, billing data or subscriber information about US customers;
● They are not to comply with any foreign privacy laws that might lead to mandatory destruction of stored data;
● Plans and infrastructure to demonstrate other states cannot spy on US customers;
● They are not to comply with information requests from other countries without DoJ permission;
● A requirement to:

. . . designate points of contact within the United States with the authority and responsibility for accepting and overseeing the carrying out of Lawful US Process to conduct Electronic Surveillance of or relating to Domestic Communications carried by or through Domestic Communications Infrastructure; or relating to customers or subscribers of Domestic Communications Companies. The points of contact shall be assigned to Domestic Communications Companies security office(s) in the United States, shall be available twenty-four (24) hours per day, seven (7) days per week and shall be responsible for accepting service and maintaining the security of Classified Information and any Lawful US Process for Electronic Surveillance . . . The Points of contact shall be resident US citizens who are eligible for US security clearances.

In other words, an "internal corporate cell of American citizens," charged with providing confidential customer data to the secret state, as The Washington Post first reported.

Additional demands include:

● A requirement to keep such surveillance confidential, and to use US citizens "who meet high standards of trustworthiness for maintaining the confidentiality of Sensitive Information" to handle requests;
● A right for the FBI and the DoJ to conduct inspection visits of the companies' infrastructure and offices; and
● An annual compliance report, to be protected from Freedom of Information requests.

This is not a one-off as the other 27 Agreements published by Public Intelligence readily attest.

For example, the 31-page 2011 Agreement between the US government and Level 3 Communications, which operates in North America, Europe, Latin America and the Asia-Pacific, which acquired Global Crossing from from the Hong Kong-based Hutchison Whampoa and Singapore Technologies Telemedia (the focus of The Washington Post's July 6 report), was expanded beyond the FBI and Department of Justice to include the Department of Homeland Security and the Department of Defense, NSA's "parent" agency.

As with the 2001 Telstra agreement, "Access" to Level 3's systems by governmental entities is defined as "the ability to physically or logically undertake any of the following actions: (a) read, divert, or otherwise obtain non-public information or technology from or about software, hardware, a system or a network; (b) add, edit or alter information or technology stored on or by software, hardware, a system or a network; and (c) alter the physical or logical state of software, hardware, a system or a network (e.g., turning it on or off, changing configuration, removing or adding components or connections)."

NSA, the principle US spy agency charged with obtaining, storing and analyzing COMINT/SIGINT "products, i.e., user data, has been handed virtually unlimited access to information flowing through Level 3 fiber optic cables as it enters the US.

This includes what is described as "Domestic Communications," content, not simply the metadata, of any phone call or email that transit Level 3 systems: "'Domestic Communications' means: (a) Wire Communications or Electronic Communications (whether stored or not) from one US location to another US location; and (b) the US portion of a Wire Communication or Electronic Communication (whether stored or not) that originates or terminates in the United States."

So much for President Obama's mendacious claim that "nobody is listening to your phone calls"!

Access to the entirety of customer records and communications is clearly spelled out in the section entitled "Electronic Surveillance."

Note: the "USC." provisions refer to (18) the Stored Communications Act which compels disclosure to the government of stored wire, electronic and transactional data; a provision that greatly weakened the Fourth Amendment right to privacy. 50 USC outlines the role of War and National Defense in the United States Code and includes "foreign intelligence," "electronic surveillance authorization without court order," "internal security," including the "control of subversive activities" and the "exercise of emergency powers and authorities" by the Executive Branch.

'Electronic Surveillance,' for the purposes of this Agreement, includes: (a) the interception of wire, oral, or electronic communications as defined in 18 U.S.C. §§ 2510(1), (2), (4) and (12), respectively, and electronic surveillance as defined in 50 U.S.C. § 1801(f); (b) Access to stored wire or electronic communications, as referred to in 18 U.S.C. § 2701 et seq.; (c) acquisition of dialing, routing, addressing, or signaling information through pen register or trap and trace devices or other devices or features capable of acquiring such information pursuant to law as defined in 18 U.S.C. § 3121 et seq. and 50 U.S.C. § 1841 et seq.; (d) acquisition of location-related information concerning a service subscriber or facility; (e) preservation of any of the above information pursuant to 18 U.S.C.§ 2703(f); and (f) Access to, or acquisition, interception, or preservation of, wire, oral, or electronic communications or information as described in (a) through (e) above and comparable state laws.

Level 3 is further enjoined from disclosing what is described as "Sensitive Information," that is, "information that is not Classified Information regarding: (a) the persons or facilities that are the subjects of Lawful US Process; (b) the identity of the Government Authority or Government Authorities serving such Lawful US Process; (c) the location or identity of the line, circuit, transmission path, or other facilities or equipment used to conduct Electronic Surveillance; (d) the means of carrying out Electronic Surveillance."

In other words, we do the spying; you hand over it over and keep your mouths shut.

The electronic driftnet thrown over global communications is expedited by direct access to Level 3's equipment by the US government.

'Principal Equipment' means the primary electronic components of a submarine cable system, to include the hardware used at the NOC(s) [Network Operations Center], landing station(s) and the cable itself, such as servers, repeaters, submarine line terminal equipment (SLTE), system supervisory equipment (SSE), power feed equipment (PFE), tilt and shape equalizer units (TEQ/SEQ), optical distribution frames (ODF), and synchronous optical network (SONET), synchronous digital hierarchy (SDH), wave division multiplexing (WDM), dense wave division multiplexing (DWDM), coarse wave division multiplexing (CWDM) or optical carrier network (OCx) equipment, as applicable.

Who oversees the set-up? On paper it appears that Level 3 control their operations. However, the Agreement specifies that the firm must utilize "primary US NOCs for any Domestic Communications Infrastructure" and it "shall be maintained and remain within the United States and US territories, to be operated by Level 3, exclusively using Screened Personnel."

Who signs off on "screened personnel"? Why the US government of course, which raises the suspicion that corporate employees are little more than spook assets.

But here's where it gets interesting. "Level 3 may nonetheless use the United Kingdom NOC for routine day-to-day management of any of the Cable Systems as such management is in existence as of the Effective Date."

Why might that be the case, pray tell?

Could it be that fiber optic cables transiting the UK are already lovingly scrutinized by NSA's kissin' cousins across the pond? GCHQ, as The Guardian disclosed, is merrily ingesting "vast quantities of global email messages, Facebook posts, internet histories and calls, and shares them" with the American agency.

Therefore, since UK undersea cable traffic is already under close "management" via the British agency's TEMPORA program, described as having the "'biggest internet access' of any member of the Five Eyes electronic eavesdropping alliance," it makes sense that Level 3 is allowed to "use the United Kingdom NOC" as a hub for its "Domestic Communications Infrastructure"!

In conclusion, these publicly available documents provide additional confirmation of how major corporations are empowering the US surveillance octopus.

By entering into devil's pacts with the world's "sole superpower," giant telcos and internet firms view the destruction of privacy rights as just another item on the balance sheet, a necessary cost of doing business in America.

And business is very good.