Climate e-mails: Were they really hacked or just sitting in cyberspace?

More than two months after the moment that thousands of confidential emails, documents and computer code from the University of East Anglia (UEA) was released online it remains a mystery who was behind the hack.

Even Sir David King, the government's former chief scientist, remains confused. This week, he sought to blame the leak on a foreign intelligence agency, only to admit later he had no evidence.

The university called in police last November, insisting they were victims of a criminal "theft" of data. Under Superintendent Julian Gregory, a group was pulled together from the counter-terrorism squad and Scotland Yard's electronic crimes unit, which also included two officers from the national domestic extremism team who have expertise in pursuing "climate extremists."

So far, the police investigation has got nowhere. It is not even clear whether the crime of computer data interception has actually occurred. What if the hacker was given a legitimate password? What if the data was accidentally open to public access?

The known facts are these. Over the weekend starting Friday 13 November 2009, someone copied files from a backup server at the UEA's Climate Research Unit (CRU) in Norwich. They were then posted anonymously on the internet and various bloggers were alerted.

This collage of 'marooned' climate sceptics -- including Senator James Inhofe and Dr Patrick Michaels -- was sent as an attachment with one of the leaked documents. It was originally sent to Phil Jones in 2007 by the US scientist Thomas Peterson. An editorial from Nature was attached. It read: 'The IPCC report has served a useful purpose in removing the last ground from under the sceptics' feet, leaving them looking marooned and ridiculous'
Within days, their contents spread around the world and were being hailed by CRU's enemies as evidence of anything from poor science to a full-blown criminal conspiracy.

There were 4,660 files in various folders: 3,587 were documents, raw data and code. Some that list tree-ring data are dated back to to 1991. Another 1,073 were emails, dating from 1996 to 12 November 2009.

King suggested earlier this week that these files must have been collected by a highly sophisticated organisation from different places over many years. That is not correct. Nor was the selection of material skilfully arranged to pick out embarrassing items.

UAE has confirmed that all of this material was simply sitting in an archive on a single backup CRU server, available to be copied.

The Guardian has carried out a detailed analysis of the emails and documents.

The emails appear to have been chosen by targeting the backups of a few key personnel wihtin CRU – the director Prof Phil Jones, his deputy Dr Keith Briffa, Dr Tim Osborn and Dr Mike Hulme. Although there are dozens of staff within CRU, only 66 of the 1,073 of the email messages were not directly sent to or from those four people.

The emails may have been filtered by using a few simple scientifically pertinent search words such as "Yamal" – a series of tree ring data from Siberia – or emails directed to US addresses. But many are completely innocuous, or indeed show the climate researchers in a good light, holding rigorous internal debates.

The leaked file was called FOIA.zip and one posting gave a [fake] email address at "foia.org". An abbreviation often used for the US Freedom of Information Act, it suggests again that the leaker was familiar with the attempts by US bloggers and others to get release of tree ring and similar data. Was the leaker American? Was he or she one of the regular readers of the blogs?

These are questions the police now appear to be asking, to judge by their current round of interviews, using the rather limited tools of overseas phone calls and formal email questionnaires.

On Tuesday, 17 November 2009, the leaked data was passed anonymously to the small group who, for some time, had been targeting CRU and its director Phil Jones. The technique involved hacking into the server of climate science blog RealClimate, and then extruding the material via a series of exotic foreign "proxy" servers.

This, and the timing immediately before UN's Copenhagen climate summit, has aroused intense suspicions among some. Could a corporation be behind the hack? While the fallout from the hack did not have a direct effect on the Copenhagen negotiations, its timing ensured maximum publicity.

It was also well-timed to influence US Senate discussions on a climate change bill. Such a manoeuvre would be consistent with the well-known "stealth" agenda of lobbyists of using citizens groups to spearhead opposition to both health care reform and climate legislation during 2009.

The biggest blog involved was California weatherman Anthony Watts' WattsUpWithThat (WUWT). Watts previously had a book published by the right-wing Heartland Institute, financed by ExxonMobil until 2006. He claims poorly sited US weather stations could in theory be skewing temperature data, although a recent analysis using his data found this was not the case.

WUWT's moderator is Charles Rotter, whose San Francisco flatmate is Steve Mosher, an "open-source software developer," and co-author of an excitable instant book on "climategate."

Information also went that first day to the more technical Climate Audit site of former Toronto mining consultant Stephen McIntyre.

Others in the loop later included Illinois aeronautical engineer Patrick Condon's site the Air Vent and Warren Meyer's Coyote Blog.

The very first release was a sort of prank. NASA scientist Gavin Schmidt in New York, an opponent of the sceptics, says that at 6:20 a.m. his time, someone tried to upload the files onto his own RealClimate website via a Turkish server.

The hacker seems to have used a technique called "privilege escalation vulnerability" to become an administrator, rather than an ordinary user of the site.

Schmidt says the hacker "disabled access from the legitimate users, and uploaded a file FOIA.zip to our server. They then created a draft post."

It read as follows: "We feel that climate science is, in the current situation, too important to be kept under wraps. We hereby release a random selection of correspondence, code, and documents. Hopefully it will give some insight into the science and the people behind it. This is a limited time offer, download now: HYPERLINK "http://ftp.tomcity.ru/incoming/free/FOI2009.zip" \t "_blank" http://ftp.tomcity.ru/incoming/free/FOI2009.zip."

There followed 20 "samples" with headline phrases from the emails such as "1059664704.txt * Mann: 'dirty laundry'" and "1075403821.txt * Jones: Daly death 'cheering news.' " John Daly was a sceptic whose death the embattled Jones appeared to welcome.

Schmidt swiftly spotted the hack and took it down. He also alerted CRU in Norwich. But even as he did that, a cryptic comment appeared on McIntyre's site. "A miracle has happened," it said, providing a link via the RealClimate website which immediately led to four unidentified downloads. McIntyre says he never noticed this posting at the time, and like all the other bloggers, denies all knowledge of its origin.

As dawn broke in California, the link to the Russian server was next posted to WUWT, where Rotter alerted Watts. Awaiting approval to put it on the site, Rotter says he gave a CD copy to Mosher, who began poring over its contents.

Mosher called him in Toronto, says McIntyre. "I couldn't believe my ears. Mosh ... asked me to confirm emails attributed to me – which I did. They didn't give me the email link." Links were next posted to the Air Vent via a Saudi server, and to Meyer's Coyote Blog in Arizona.

Not until 19 November 2009 did a key email arrive for McIntyre from England. It was from his own contact at UEA, the isotope specialist Paul Dennis.

With the subject line "Interesting!", it attached the text of an alert from Dennis's own head of department at Norwich. This warned that "climate change sceptics" had obtained and posted up a "large volume of files and emails," and urged colleagues to check for viruses.

The bloggers say this gave them the confirmation they had been waiting for. "These actions reassured Mosher that the files were genuine," explains McIntyre.

Mosher says he also received a posting direct from the secret leaker, complaining that nothing was happening. He replied, he says: "A lot is happening behind the scenes. It is not being ignored. Much is being co-ordinated among major players and the media. Thank you very much. You will notice the beginnings of activity on other sites now. Here soon to follow."

But McIntyre was meanwhile guarded with his source in Norwich. He emailed him back: "I haven't seen such a website. You'd think there'd be discussion on the blogs of something like that. I'll definitely stay tuned!!" Only after the bloggers had launched their great scoop did he inform Dennis.

The use of foreign servers proved to be a red herring. The Mail on Sunday claimed the Russians must therefore be behind it, and King speculated about a "highly sophisticated" cyber attack. In fact the use of so-called "open proxy" servers to remain anonymous is on page one of any whistleblowers' manual.

A programme called TOR, for example, can be downloaded which will automatically switch between a random variety of servers. Digital forensic examination of the archive of emails and documents suggests that it was first created around 30 September 2009, and subsequently added to during October and finally in November – when one of Osborn's sets of program code was added – just ahead of the full-blown leak.

Significantly, that analysis suggests that the archive was created on a machine running five hours behind GMT, which would put it on the east coast of North America.

The leaker therefore knew something about computers, just as they knew something about climate science. But it didn't require the skills of Government Communications Headquarters or a foreign intelligence agency, as has been suggested. The hacking of the RealClimate blog exploited the fact that its wordpress platform has security holes well known to hackers.

Some commentators point to a previous pattern of leaks that is strikingly similar to what happened in November. On 24 July 2009, McIntyre says he received a freedom of information (FOI) refusal from CRU. He announced it on his website. The next day McIntyre announced that he had got hold of a mass of data.
He was initially coy about it. He said: "Folks, guess what. I'm now in possession of a CRU version giving data for every station in their station list."

The next day he said: "I learned that the Met Office/CRU had identified the mole. They are now aware that there has in fact been a breach of security … My guess is that they will not make the slightest effort to discipline the mole."

This was a tease. There was no human "mole," just a security breach. Rotter in San Francisco later blogged that "In late July I discovered they had left station data versions from 2003 and 1996 on their server without web page links but accessible all the same. They were stale versions of the requested data ... just sitting in cyberspace waiting for someone to download."

McIntyre later admitted that "I downloaded from the public CRU ftp site ... No hacking was involved".

David Holland, a British engineer who had been making FOI requests, says he too found CRU files accidentally open. In December 2008, he notified the university that "the search engine on your home page is broken and falling through to a directory." They thanked him and said it was caused by a "misconfiguration of the webserver." Holland says he didn't download anything since he knew it could be traced back to his computer.
 
After the July incident, perhaps CRU failed to batten down the hatches, either through technical failings or because someone inside was subverting the efforts. So what happened in November?

Rotter blogged his theory last year. "In the past I have worked at organisations where the computer network grew organically in a disorganised fashion. Security policies often fail as users take advantage of shortcuts ... one of these is to share files using an ftp server ... This can lead to unintentional sharing with the rest of the internet."

He added that files were perhaps put "in an ftp directory which was on the same central processing unit as the external webserver, or even worse, was on a shared driver somewhere to which the webserver had permissions to access. In other words, if you knew where to look, it was publicly available."

If this hypothesis turns out to be true, UEA may end up looking foolish. For there will be no one to arrest.

Link:  http://www.guardian.co.uk/environment/2010/feb/04/climate-change-email-hacker-police-investigation