The Snowden Files: The Inside Story of the World's Most Wanted Man

10 DON’T BE EVIL

It was an iconic commercial. To accompany the launch of the Macintosh in 1984, Steve Jobs created an advert that would captivate the world. It would take the theme of George Orwell’s celebrated dystopian novel and recast it – with Apple as Winston Smith. His plucky company would fight the tyranny of Big Brother.

As Walter Isaacson recounts in his biography of Jobs, the Apple founder was a child of the counterculture. He practised Zen Buddhism, smoked pot, walked around barefoot and pursued faddish vegetarian diets. He embodied the ‘fusion of flower power and processor power’. Even as Apple grew into a multi-billion dollar corporation, Jobs continued to identify with computing’s early subversives and long-haired pioneers – the hackers, pirates, geeks and freaks that made the future possible.

Ridley Scott of Blade Runner fame directed the commercial. It shows Big Brother projected on a screen, addressing lines of workers. These skinhead drones wear identical uniforms. Into the grey nightmare bursts an attractive young woman. She wears orange shorts and a white tank top. She is carrying a hammer! Police in riot gear run after her. As Big Brother announces ‘We shall prevail’, the heroine hurls the hammer at him. The screen explodes in a blaze of light; the workers are open-mouthed. A voice announces smoothly: ‘On January 24th, Apple Computer will introduce Macintosh. And you’ll see why 1984 won’t be like 1984.’

The 60-second advert was screened to nearly 100 million Americans during the Super Bowl, and was subsequently hailed as one of the best ever. Isaacson writes: ‘Initially the technologists and hippies didn’t interface well. Many in the counterculture saw computers as ominous and Orwellian, the province of the Pentagon and the power culture.’

The commercial asserted the opposite – that computers were cool, revolutionary and empowering, instruments of self-expression. The Macintosh was a way of asserting freedom against an all-seeing state.

Almost 30 years later, following Jobs’s death in 2011, an NSA analyst came up with a smirking rejoinder. He prepared a top-secret presentation and, to illustrate the opening slide, he pulled up a couple of stills from Jobs’s commercial – one of Big Brother, the other of the blonde heroine with the hammer and the orange shorts.

Under the heading ‘iPhone Location Services’ he typed:

‘Who knew in 1984…’

The next slide showed the late Jobs, holding up an iPhone.

‘… that this would be Big Brother…’

A third slide showed crowds of whooping customers celebrating after buying the iPhone 4; one fan had inked the name on his cheek. The analyst’s pay-off line read:

‘… and the zombies would be paying customers.’

The zombies were the public, unaware that the iPhone offered the spy agency new snooping capabilities beyond the imagination of the original Big Brother. The ‘paying customers’ had become Orwell’s mindless drones.

For anyone who thought the digital age was about creative expression and flower power, the presentation was a shocker, and an insult to Steve Jobs’s vision. It threw dirt on the hippy kaftan and trampled on the tambourine. The identity of the NSA’s analyst is unknown. But the view appeared to reflect the thinking of an agency that in the aftermath of 9/11 grew arrogant and unaccountable. Snowden called the NSA ‘self-certifying’. In the debate over who ruled the internet, the NSA provided a dismaying answer: ‘We do.’

The slides, given to Poitras and published by Der Spiegel magazine, show that the NSA had developed techniques to hack into iPhones. The agency assigned specialised teams to work on other smartphones too, such as Android. It targeted BlackBerry, previously regarded as the impregnable device of choice for White House aides. The NSA can hoover up photos and voicemail. It can hack Facebook, Google Earth and Yahoo Messenger. Particularly useful is geo-data, which locates where a target has been and when. The agency collects billions of records a day showing the location of mobile phone users across the world. It sifts them – using powerful analytics – to discover ‘co-travellers’. These are previously unknown associates of a target.

Another secret program had a logo that owed a debt to the classic 1970s Pink Floyd album Dark Side of the Moon. It showed a white triangle splitting light into a colourful spectrum. The program’s name was PRISM. Snowden leaked a 41-slide PowerPoint presentation explaining PRISM’s function.

One slide emphasised the dates when Silicon Valley’s technology companies apparently signed up and become corporate partners of the spy agency. The first to provide PRISM material was Microsoft. The date was 11 September 2007. This was six years after 9/11. Next came Yahoo (March 2008) and Google (January 2009). Then Facebook (June 2009), PalTalk (December 2009), YouTube (September 2010), Skype (February 2011) and AOL (March 2011). For reasons unknown, Apple held out for five years. It was the last major tech company to sign up. It joined in October 2012 – exactly a year after Jobs’s death.

The top-secret PRISM program allows the US intelligence community to gain access to a large amount of digital information – emails, Facebook posts and instant messages. The rationale is that PRISM is needed to track foreign terrorists living outside the US. The data-collection program does not apparently require individual warrants. Rather, federal judges give their broad approval to PRISM under the FISA. By the time Snowden revealed PRISM, at least nine technology companies were on board. (The slides show Dropbox was slated to join; Twitter was missing.)

The most bitter and contentious question is how the NSA accesses this personal data. The key slide claims the data is collected ‘directly from the servers’ of the nine ‘US service providers’, Google, Yahoo and the rest.

Speaking in Hong Kong, Snowden was adamant this ‘direct access’ was indeed how PRISM worked. He told Greenwald: ‘The US government co-opts US corporate power to its own ends. Companies such as Google, Facebook, Apple and Microsoft all get together with the NSA. [They] provide the NSA direct access to the backends of all of the systems you use to communicate, to store data, to put things in the cloud, and even just to send birthday wishes and keep a record of your life. They give [the] NSA direct access, so that they don’t need to oversee, so they can’t be held liable for it.’

The leaked PRISM documents come from a training manual for NSA staff. It sets out several steps. First, a complex ‘tasking’ process. Analysts use or ‘task’ PRISM to find a new surveillance target. Next, a supervisor reviews the analyst’s search terms, known as selectors. After that the supervisor then has to agree with the analyst’s ‘reasonable belief’ the target lives outside the US. (This bar is pretty low, and defined as ‘51 per cent confidence’.)

Once the target has been agreed, PRISM gets to work. Sophisticated FBI equipment at the tech companies extracts matching information. The FBI has its own database to weed out – or ‘research and validate’, as the slide puts it – US persons whose data may have been sucked up by mistake. (This system, however, isn’t foolproof.) The FBI then gives this data to the NSA. An array of NSA analytical tools processes it. These include MARINA, which sifts and stores internet records, MAINWAY for call records, PINWALE which does video, and NUCLEON, voice.

Another slide says that the NSA has ‘real-time reporting capability’. In other words, the agency is notified each time a target sends an email, writes a text, begins a chat, or even fires up their computer.

Snowden’s slide gives some sense of just how important PRISM has become to US intelligence efforts. As of 5 April 2013, the US had 117,675 active surveillance targets in its PRISM database. According to the Washington Post, much PRISM-derived intelligence ends up on President Obama’s desk; it accounts for one in seven intelligence reports. British spies get to read it too.

The training manual gives the impression that Silicon Valley is actively collaborating with the NSA, albeit with varying degrees of enthusiasm. The corporate logos of all of nine tech companies appear on the top of each PRISM slide. Jobs’s Apple is among them. The logos look like shiny, colourful butterflies.

Snowden says it was his concerns over PRISM that pushed him towards whistleblowing. It was one of the first documents he leaked to Greenwald and Poitras. But PRISM was only one important element in a troubling picture. Over the last decade the US had been secretly working to gather practically all communications entering and leaving the US.

The NSA’s original mission was to collect foreign intelligence. But it appears to have drifted away from its original goal, like a vast supertanker floating away from its anchor. It is now sucking in a lot of domestic communications. In this new era of Big Data, the agency moved from the specific to the general; from foreign targeting to what Snowden called ‘omniscient, automatic, mass surveillance’.

The agency’s other big operation, its highly sensitive cable-tapping program, ran parallel to GCHQ’S British TEMPORA project and was codenamed UPSTREAM. It gives the NSA direct access to the fibre-optic cables carrying internet and telephone data into, out of and around the US.

UPSTREAM is explained in one slide ‘as the collection of communications on fibre cables and infrastructure as data flows past’. The slide shows a map of the US with brown cables extending in both directions across the Pacific and Atlantic oceans. The diagram looks like the thick tentacles of an enormous sea creature. Seemingly, the US has international cable taps in South America, East Africa and the Indian Ocean. There are green loops around the cables. They link to a box marked UPSTREAM. Below is a second box labelled PRISM. Linking both boxes is an instruction to the agency’s data collectors: ‘You should use both.’

According to author James Bamford, citing earlier NSA whistleblower William Binney, UPSTREAM captures 80 per cent of communications. PRISM scoops up anything that UPSTREAM may have missed.

Snowden referred to UPSTREAM when he told Greenwald: ‘The NSA doesn’t limit itself to foreign intelligence. It collects all communications that transit the US. There are literally no ingress or egress points anywhere in the continental US where communications can enter or exit without being monitored and collected and analysed.’

Since a large amount of the world’s internet traffic travels through the US and 25 per cent of it also crosses Britain, the two spy agencies between them have the ability to hack most of the globe’s key communications. A 2009 report by the NSA’s inspector general, leaked by Snowden, acknowledges this. It says: ‘The United States carries out foreign intelligence activities through a variety of means. One of the most effective means is to partner with commercial entities to obtain access to information that otherwise would not be available.’

The report refers to ‘America’s homefield advantage as the primary hub for worldwide telecommunications’. It says that the NSA currently has relationships with over ‘100 US companies’. This private sector/spy agency collaboration stretches ‘as far back as World War Two’.

Thanks to ties to two unnamed companies in particular, the NSA is able to eavesdrop on the world, or as the inspector general puts it, access ‘large volumes of foreign-to-foreign communications transiting the United States through fibre-optic cables, gateway switches and data networks’.

The US has the same ‘advantage’ when it comes to international telephone calls. Most international calls are routed through a small number of switches or ‘choke-points’ in the international telephone system, en route to their final destination. Many are in the US. The country is a ‘major crossroads for international switched telephone traffic’, the report says. It gives striking figures: of the 180 billion minutes of telephone communications in 2003, 20 per cent came from or terminated in the US, and 13 per cent transited the US. The internet numbers are bigger. In 2002 only a small fraction of international internet traffic went via non-US routes.

The NSA–telecoms partnership was highly lucrative. In return for access to 81 per cent of international telephone calls, Washington pays the private telecom giants many hundred millions of dollars a year. It is not known how much the British government pays its own domestic ‘intercept partners’, particularly the formerly state-owned BT, and Vodafone. But the sums will be similar and substantial.

By the end of the last decade, the NSA’s capabilities were astonishing. The agency, backed by Britain and its other Five Eyes allies, had access to fibre-optic cables, telephone metadata and the servers of Google and Hotmail. The NSA’s analysts were the most powerful spies in human history. Snowden maintains they were able to target practically anybody, at any time, including the president.

‘The NSA and the intelligence community in general is focused on getting intelligence everywhere and by any means possible,’ he says. ‘Originally we saw this focus very narrowly targeted on foreign intelligence. Now we see it’s happening domestically. To do that the NSA specifically targets the communications of everyone. It ingests them by default. It collects them in its systems. It filters them and it analyses them and it measures them and it stores them for periods of time simply because that’s the easiest and most efficient and most valuable way to achieve these ends.’

Looked at as a whole, the files lend weight to Snowden’s assertion that as an NSA analyst he had super-powers.

‘While they may be intending to target someone associated with a foreign government or someone they suspect of terrorism, they are collecting your communications to do so. Any analyst at any time can target anyone. Any selector, anywhere. Whether these communications may be picked up depends on the range of the sensor networks and the authorities an analyst is empowered with. Not all analysts have the ability to target everybody. But I, sitting at my desk, certainly had the authority to wiretap anyone, from you, to your accountant, to a federal judge, and even the president, if I had a personal email [address].’

The PRISM revelations provoked a howling response from the hi-tech denizens of San Francisco’s Bay Area. First there was bafflement, then denial, followed by anger. The Santa Clara valley, where most of the big tech firms are situated, likes to see itself as anti-government. The philosophical currents that waft through Cupertino and Palo Alto are libertarian and anti-establishment, a legacy of Silicon Valley’s roots in the hacker community. At the same time, these firms vie for government contracts, hire ex-Washington staff for the inside track and spend millions lobbying for legislation in their favour.

Clearly, the allegation that they were co-operating with America’s most powerful spy agency was a corporate disaster, as well as being an affront to the Valley’s self-image, and to the view of the tech industry as innovative and iconoclastic. Google prided itself on its mission statement ‘Don’t be evil’; Apple used the Jobsian imperative ‘Think Different’; Microsoft had the motto ‘Your privacy is our priority’. These corporate slogans now seemed to rebound upon their originators with mocking laughter.

Before the Guardian published the PRISM story the paper’s US business reporter, Dominic Rushe, went through his contacts book. He called Sarah Steinberg, a former Obama administration official, and now Facebook’s PR, as well as Steve Dowling, the head of PR at Apple. He rang Microsoft, PalTalk and the others. All denied any voluntary collaboration with the NSA.

‘There was total panic. They said they had never heard of it [PRISM],’ Rushe recalls. ‘They said they hadn’t given direct access to anybody. I was totally bombarded with telephone calls from increasingly senior tech executives who had more questions than answers.’

The tech companies said that they only released information to the NSA in response to a specific court order. There were no blanket policies, they said. Facebook revealed that in the last six months of 2012 it gave the personal data of between 18,000 and 19,000 users to various US law-enforcement bodies, not just to the NSA but also to the FBI, federal agencies and local police.

Several of the companies stressed they had mounted legal challenges in the FISA courts to try and say more about secret government requests for information. Google insisted: ‘We do not provide any government, including the US government, with access to our systems.’ Google’s chief architect Yonatan Zunger remarked: ‘We didn’t fight the cold war just so we could rebuild the Stasi ourselves.’ Yahoo said it had fought a two-year battle for greater disclosure, and had challenged amendments to the 2008 Foreign Intelligence Surveillance Act. Its efforts were thus far unsuccessful.

The NSA documents, though, look explicit. They say ‘direct access’.

Asked how he might explain the discrepancy, one Google executive called it a ‘conundrum’. He dismissed the PRISM slides as a piece of flimsy ‘internal marketing’. He added: ‘There is no back-door way of giving data to the NSA. It’s all through the front door. They send us court orders. We are obliged by law to follow them.’

But in October 2013 it emerged there was indeed a back door – just one that the companies involved knew nothing about. The Washington Post revealed that the NSA was secretly tapping data from Yahoo and Google. The method was ingenious: ‘on British territory’, the agency had hacked into the private fibre-optic links that inter-connect Yahoo and Google’s own data centres around the world.

The NSA codename for this tapping operation is MUSCULAR. It appears to be the British who are doing the actual hacking on the US’s behalf. (One MUSCULAR slide says ‘Operational July 2009’, and adds: ‘Large international access located in the United Kingdom.’)

The firms go to great lengths to keep their customers’ data safe. However, they transfer their information between data centres situated in Europe and America, along leased private internet cables protected by company-specific protocols. It was these cables that the NSA had managed to hack, as they transit the UK. Curiosity focused on Level 3, reported to have been hired as a cable operator by Yahoo and Google: Level 3 is named in the top-secret British documents as an ‘intercept partner’ with the codename LITTLE. The Colorado-based corporation’s response is to say it complies with legal requests in the countries where it operates.

An NSA analyst drew a child-like sketch explaining how the program works; it shows two regions marked ‘Public Internet’ and ‘Google Cloud’. There is a smiley face at the interface where the NSA hacks data. The sketch provoked a thousand Twitter parodies. ‘With so many of these slides you get the feeling people inside the NSA are bragging about their programs,’ ProPublica’s Jeff Larson says. ‘They are saying: ‘We can break encryption! We can grab protocols!”

A document from the NSA’s acquisitions directorate reports that thanks to its back-door access the agency can break into hundreds of millions of user accounts. The data is sent back to the NSA’s Fort Meade headquarters and stored. The volumes are remarkable. In a 30-day period in late 2012, 181,280,466 new records were funnelled back to the Puzzle Palace, including metadata.

Google and Yahoo reacted with apoplexy to the tapping disclosures. Google’s chief legal officer David Drummond said he was outraged at the lengths to which the US government had gone to ‘intercept data from our private fibre networks’. Yahoo repeated that it had no knowledge of the NSA’s back-door cyber-theft.

By the autumn of 2013 all the tech companies said they were scrambling to defend their systems from this kind of NSA snooping. They stood some chance of success. For the NSA’s power to suck up the world’s communications is not quite as awesome as Snowden has made it seem. Tapping into global flows of data is one thing: being able actually to read them is quite another. Particularly if they start to be encrypted.

On 23 October 1642, two armies clashed in the English fields north of Oxford. One belonged to King Charles, the other to Parliament. The battle of Edge Hill was the first in the bloody English civil war. The fight was messy. Parliament forces fired their cannons; the royalists led a cavalry charge; inexperienced soldiers on both sides ran away. Some were keener on looting than defeating the enemy. Neither side really won. The war dragged on for another four years.

Two centuries later, on 21 July 1861, another skirmish took place. This time the Union Army was fighting the Confederates, in the first major land encounter of the American civil war. The location was Bull Run, a tributary of the Potomac in Virginia. The Northern forces expected a quick victory. Instead, the Confederate army launched a ferocious counter-attack. Brigadier General Irvin McDowell and his Union soldiers fled in the direction of Washington DC. The battle revealed there would be no easy knockout.

Many years later, American and British spies were mulling over names for two top-secret programs. Their new battles were electronic rather than territorial. It was the growing practice of encryption that was their enemy. The names they chose for their new battles were BULLRUN and EDGEHILL. Did the emphasis on civil wars have a special significance? Certainly, the spies were now about to declare war on their own domestic corporations.

Cryptography was first used in ancient Egypt and Mesopotamia. The aim, then as now, was to protect secrets. During the first and second world wars, military cryptography and cryptanalysis – the ability to decrypt coded information on enemy movements – played a key role. But it was largely the preserve of embattled nation states. Typically, those interested in codes were the British mathematicians working in secret to defeat the Nazis at wartime Bletchley Park, and the Soviets subsequently.

By the 1970s, however, encryption software such as Pretty Good Privacy (or PGP) was available to private individuals, as well as commercial organisations. Encryption thus posed an obvious challenge to western intelligence agencies, anxious to continue reading their adversaries’ messages. The Clinton administration responded by trying to insert a back door into commercial encryption systems. This would let the NSA in. The attempt met with political defeat. A bipartisan group of senators and tech executives argued this would be bad for the Valley. Plus it would violate the fourth amendment.

By 2000, as encryption was increasingly employed by service providers and individuals in everyday online communications, the NSA was spending billions of dollars finding ways to get round it. Its encrypted targets included web searches, internet chats, emails, personal data, phone calls, even banking and medical records. The challenge was to convert ‘ciphertext’ – what encrypted data looks like in its raw form: that is, mathematical nonsense – into ‘cleartext’.

In 2010 a British GCHQ document warned that over time the allies’ capacities could degrade as ‘information flows change’ and ‘widespread encryption becomes more commonplace’.

At first, the eavesdroppers seemed to face defeat, or at least stalemate. One of the leaked documents from 2006 shows that, at that date, the agency had only broken the encryption of one foreign state’s nuclear ministry, a single travel reservation system, and three foreign airlines.

It was not until 2010 that the NSA made dramatic progress, thanks to BULLRUN and EDGEHILL. It used super-computers to crack algorithms, encryption’s basic building blocks. (Algorithms generate the key which can encrypt and decrypt messages. The longer the key, the better the encryption.)

But most importantly, the Snowden files show that the NSA cheated. Despite the political defeat on back doors, the agency simply went ahead and secretly introduced ‘trapdoors’ into commercial encryption software used by millions of people. It collaborated with developers and technology companies to insert deliberate, exploitable flaws into both hardware and software. Sometimes this co-operation was voluntary; sometimes bullying legal orders enforced it. The NSA, if necessary, would steal encryption keys, almost certainly by hacking into servers where the keys were kept.

Unsurprisingly, the NSA and GCHQ were keen to keep details of these most shadowy of programs under wraps. A 2010 document from Snowden shows just how restricted knowledge was of BULLRUN – and how effective it was. The PowerPoint was used to brief British staff in Cheltenham on the NSA’s recent breakthroughs, as a result of which decrypted internet traffic was suddenly streaming across the desks of analysts.

It says: ‘For the past decade the NSA has led an aggressive, multi-pronged effort to break widely used internet encryption technologies. Cryptanalytic capabilities are now coming online. Vast amount of encrypted internet data which up to till now have been discarded are now exploitable.’

The slide says ‘major new processing systems’ must be put in place ‘to capitalise on this opportunity’. GCHQ staff previously kept in the dark about BULLRUN were astonished by the NSA’s formidable new capabilities. One internal British memo reports: ‘Those not already briefed were gobsmacked.’

Snowden’s first batch of published files did not disclose details of which companies work with the NSA on counter-encryption. Or which commercial products may have back doors. But the files do give some idea of BULLRUN’s massive dimensions. A budget report for the entire US intelligence community says that 2013 funding for the program was $254.9m. (PRISM, by contrast, costs just $20m annually.) Since 2009, the agency has splashed more than $800m on ‘SIGINT [signals intelligence] enabling’. The program ‘actively engages US and foreign IT industries to covertly influence and/or overtly leverage their commercial products’ designs’, the report says.

The joy of the program, the NSA says, is that ordinary citizens have no idea that their everyday encrypted communications are now hackable. When the NSA inserts ‘design changes’ into commercial encryption systems, the 178-page report for the fiscal year notes, ‘To the consumer and other adversaries… the systems’ security remains intact.’

James Clapper, the director of national intelligence, stresses the importance of crypto. ‘We are investing in groundbreaking cryptanalytic capabilities to defeat adversarial cryptography and exploit internet traffic,’ he writes.

The agency is not lacking in ambition. The files show the NSA is breaking the encryption systems of 4G phones. It targets online protocols used in secure banking and business transactions, such as HTTPS and Secure Sockets Layer (SSL). It wants to ‘shape’ the worldwide encryption marketplace. Soon it expects to get access to ‘data flowing through a hub for a major communications provider’ and to a ‘major internet peer-to-peer voice and text communications system’. That sounds like Skype.

Meanwhile, the British were pressing on with their own parallel EDGEHILL project. One file shows that the British spies have succeeded in breaking into three internet providers and 30 types of Virtual Private Networks (VPN) used by businesses to access their systems remotely. By 2015 it hoped to have penetrated 15 internet companies and 300 VPNs.

The spy agencies insist that their ability to defeat encryption is essential to their mission, and that without it they would be unable to track terrorists or gather valuable foreign intelligence. The problem, as the New York Times points out, is that the NSA’s anti-encryption stealth campaign may have disastrous unwanted consequences.

By inserting deliberate weaknesses into encryption systems, the agency has made those systems exploitable. Not just by government agencies, who may be acting with good intentions, but by anybody who can get hold of encryption keys – such as hackers or hostile intelligence agencies. Paradoxically, in its quest to make Americans more secure, the NSA has made American communications less secure; it has undermined the safety of the entire internet.

The main US agency for setting security norms in cyberspace is the National Institute of Standards and Technology (NIST). It appears the NSA has corrupted this, too. A Snowden document reveals that in 2006 the NSA put a back door into one of the institute’s main encryption standards. (The standard generates random prime numbers used to encode text.) The agency then encouraged another international standards body – and the rest of the world – to adopt it, boasting: ‘Eventually the NSA became the sole editor.’

Both US and UK agencies have also devoted considerable efforts to cracking Tor, the popular tool to protect online anonymity. Ironically, the US government is one of Tor’s biggest backers. The State Department and the Department of Defense – which houses the NSA – provide around 60 per cent of its funding. The reason is simple: journalists, activists and campaigners in authoritarian countries such as Iran use Tor to protect themselves from political reprisals and online censorship.

Thus far, however, the NSA and GCHQ have been unable to de-anonymise most Tor traffic. Instead, the agencies have attacked web browsers such as Firefox, which allows them control over a target’s end computer. They have also developed the ability to ‘stain’ some traffic as it bounces around the Tor system.

Despite their best endeavours, the truth appears to be that NSA and GCHQ have not yet won cryptography’s new civil war. With the right training and some technical expertise, corporations and individuals (as well, no doubt, as terrorists and paedophiles) are still successfully using cryptography to protect their privacy.

In a Q&A with Guardian readers while in hiding in Hong Kong, Snowden himself said: ‘Encryption works. Properly implemented strong crypto systems are one of the few things that you can rely on.’

And he should know.