How Does NSA's XKEYSCORE Project Work?

by Duran

Before going into the operation of the XKEYSCORE project from a technical level, I hope you can read three documents that will help you understand this article.  The three documents were exposed by Edward Snowden and can be downloaded from the Internet.  You can search and get it from Google with keywords [XKEYSCORE presentation], "the collection strategies and requirements center", and "digital network exploitation, digital network intelligence, and computer network exploitation".  I will not introduce the contents of the three documents mentioned above, but give the analysis conclusions directly.

Analysis Portion

XKEYSCORE is an important system of NSA's global network surveillance system, which is an international metadata collection tool.  Around 150 sites and more than 700 servers have been established around the world.  These globally-distributed sites, called collection sites, are responsible for collecting metadata from various networks and use the full-take method.  Therefore, in theory, NSA can master all the activities of the target network.

Next, I'll explain how XKEYSCORE works.

First of all, the project is based on several concepts: Digital Network Exploitation (DNE), Digital Network Intelligence (DNI), and Computer Network Exploitation (CNE).  The specific meaning is described in those three background documents.  The relationship among the three can be represented by this graph:

CNE is the foundation and the most critical part of the whole system.  It supports the data source of the whole project.  DNI extracts valuable information from DNE data.  The quote from the document is "DNI is the resultant intelligence that DNE produces."

Secondly, the whole system adopts distributed cluster architecture, which is easy to expand and manage, and can add more servers to store data.  For instance, its query hierarchy, the federated query system, can do "one query scans all sites."  It can be seen from here that all intercepted data is stored on the local server of each site, because this is the most convenient and reliable way.  If an intelligence analyst based in the United States wants to find a certain target, he only needs to submit the query keyword to the system, and the system will send the query request to the server of each site for query.

Of course, he can set the query range.  For example, if the target data is in Europe, he can set the query request only to the European site server.  The XKEYSCORE system will filter the collected data, discard the irrelevant data, and then the remaining part will be indexed by metadata.  These different types of data will be set to different storage time limits, because most of these sites are inside the embassy and consulate, where there is not much space for computer room.

Finally, and very most importantly, how is the data obtained?

The CNE mentioned in the document will be discussed here.  We can think of exploitation as another word: hack.

The data was "stolen" by NSA hackers.  One of the famous departments is Tailored Access Operations (TAO).  If you're interested, you can check out the speech by Rob Joyce, its director, on the Internet.  These hackers should be deployed in the United States and embassies and consulates.  They are responsible for hacking or infiltrating individual targets or entire target networks (end-point) and control network node routers (mid-point) for rerouting and hijacking traffic.  The data will be sent back to the servers at each site.

The next job is to process and analyze this data through the XKEYSCORE system, which is explained in detail in the related materials slide.  As for how NSA collects traffic, you can learn about HAMMERSTEIN, one of the tools they use, which is used to forward VPN traffic packets in routers, and CHERRY BLOSSOM, a router vulnerability tool, as well as the NSA hacker tool set which was exposed on the web.

Some Digressions

According to the WikiLeaks document in 2018, the U.S. consulate general in Frankfurt may be the secret center of hacking activities in Europe.  Obviously, it makes sense because, geographically speaking, Frankfurt is located in the center of Europe, which is the best location to intercept the network traffic of the whole of Europe.  NSA hackers can, of course, attack overseas network facilities directly from the United States, but some jobs must be done on-site.  That's why some hackers need to be sent to work in embassies and consulates.

One interesting thing about the XKEYSCORE map is that there is a red dot on the vast territory of China, which shows that XKEYSCORE servers are also deployed here.  According to relevant information, the red dot is located in Chengdu, China, the location of the (former) Consulate General of the United States.  It has long been an open secret that the U.S. Consulate General in Chengdu was responsible for collecting PLA data, instigating rebellion, supporting the Dalai Lama, splitting Tibet, and supporting terrorism.

In addition, due to the diplomatic friction between China and the United States, China closed the consulate in July 2020.  This also illustrates the importance of the consulate's intelligence activities in China.  Local residents found that in the early morning on the day of evacuation, five transport trucks loaded with five containers left the consulate.  More recently, several moving service vehicles entered the consulate to transport away the office supplies and personal necessities of the staff.  What were the contents of these five containers?  Why were they so heavy that they needed a crane to be lifted?  I think these boxes contained some non-exposure devices, including the XKEYSCORE servers.

There is also an inference that can be verified from public information that NSA can completely control the network infrastructure produced by American manufacturers, e.g. Cisco.

In 2013, Mandiant released a report exposing Chinese military hackers and relevant evidence documents.  One of them revealed that China Telecom authorized optical cable for People's Liberation Army Unit 61398 to access the Shanghai 005 center.  This was an internal document of the service provider.  How did Mandiant get it?  There is no doubt that they penetrated into the ISP's office intranet, where it was able to utilize the relevant equipment vulnerabilities.  You know, at that time, most of the Chinese ISPs used equipment from American companies.  There are many ways to plant backdoors into these devices, such as attacking through vulnerabilities, intercepting logistics on the way, and embedding Trojans directly, etc.

It is not difficult to understand why the U.S. government is trying to block other countries from purchasing Huawei products.  In addition to commercial factors, the most important and deep reason is that once these countries use Huawei's devices, the United States will not be able to easily hack and control this network infrastructure, and they will also lose control of network traffic.

This is a serious threat to the global intelligence collection activities of the United States.  Also, it's not hard to understand why Five Eyes alliance countries unanimously refuse to use Huawei products.