Telecom Informer
by The Prophet
Hello, and greetings from the Central Office! It's a typical late autumn day in the Great Northwest, raining cats and dogs and maybe hamsters too. It's cold and windy, tree branches are down all over town, and the outside plant crews are out having the kinds of days that make them question their careers. And as miserable as it is outside, and as impossible as it is to get icky-pic out of my clothes, I'd far rather be working with them than doing what I'm tasked with today.
Earlier this week, the legal department received roughly a truckload of subpoenas. Someone is suing one of our customers, and their lawyers are demanding Call Detail Records (CDR). This customer has specially configured lines of service, running on an ancient legacy system. The tools that the legal department ordinarily uses to pull this information unfortunately don't work in the ancient system that serves this customer.
Fortunately for the legal department, and unfortunately for me, I wrote the most comprehensive set of internal documentation for this particular legacy system, so guess who they called when they couldn't find anyone else? And that's how I ended up here in the Central Office, operating under a very strict deadline, and trying to figure out how to re-ink the ribbon on a dot matrix printer. There is no easy way to output the records from this system electronically (we do have electronic output for billing, but that doesn't show call details for any local calls), and I don't have time to figure it out, given the strict timelines. Legal didn't specify the required format, so they're getting the call detail records printed - in ASCII - on dusty tractor feed paper! I'll send it to them via interoffice mail in a large-sized banker's box.
Most people don't realize the level of granular detail that call detail records contain about who they call, when they're calling, for how long, and from where (both virtually and physically). Our records contain everything that engineering teams need to troubleshoot a problem circuit - trunk, circuit, etc. Mobile phone records go far beyond the level of detail that we have here in the Central Office; they can include the IP address issued to the handset, the physical location of the towers that handled the call, and even the physical location of the handset (when 911 calls are made). They additionally contain details of text and picture messages sent and received - sometimes even including the content. These can be maintained for an exceptionally long period of time - up to seven years!
That's why attorneys love call detail records, and often seek to subpoena them. "Oh, you claim you weren't cheating?" a divorce attorney might say. "Then why was your phone making calls to your wife from the neighborhood where the woman you were cheating with lives, and the side of the tower that picked up your call is the one pointed at her house?" It's "smoking gun" evidence like this that attorneys are seeking when they subpoena these records and, of course, law enforcement makes extensive use of call detail records too. While access isn't guaranteed in civil cases, very broad access is granted in criminal cases. A law called 18 USC § 2703 outlines the rules under which law enforcement is granted access to these "business records," and they aren't very strict.
The process starts with a "letter of preservation," where an attorney demands that records be preserved. This must be for a specifically defined period of time, and typically includes the following data for a mobile phone carrier:
- Subscriber billing and account information (typically name, address, and other subscriber contact details such as email address, other phone numbers, etc.).
- Any details that the carrier has on when and where the service was set up, whether the phone is an individual, commercial, or family subscription, and any associated telephone numbers on the account.
- ESN, MEID/IMEI, and IMSI information of the device.
- Call Detail Records (CDR) for telephone calls, SMS text messages, MMS picture messages.
- Records of data sessions, including IP addresses, amount of data transferred, and URLs of websites visited.
- Metadata for all stored voicemail messages. Typically, they'll request the content too, and carriers have to preserve that. But in the end, courts often only grant access to the metadata.
- Cell tower location information.
- RTT/PCMD data (this is diagnostic data used to troubleshoot call quality; attorneys like to have this so they can push back on unreliability claims).
At the time a preservation request is received, phone companies don't provide any information to the requesting party. In most cases, the legal department will typically use an automated internal tool (written by a consulting company they hired) to gather all of the information that is requested. They store it in a case management system (creating a new file for each case), and in theory, we never need to be involved or even know that any of this is happening. In many cases, an information preservation request is filed, but no subpoena materializes so the data never leaves our systems. If the court approves a subpoena, the legal department will use a secure managed file transfer service to deliver the records to the court (and only the specific records approved by the court, which doesn't always match what we were requested to preserve).
Of course, there is "in theory" and "in practice" and you probably know where I'm going with this. In practice, the legal department hired the cheapest offshore vendor they could find to build their glitchy automated tool (presumably out of chewing gum, string, and a discarded tennis shoe). It is buggy and breaks a lot, and it throws inscrutable error messages, so we often get involved to help troubleshoot. Having learned through experience why the tool breaks, we can usually manage to adjust our systems around the tool's expectations in order to help the legal department get the information they need. Occasionally, it entirely breaks. In this case, we'll pull the records manually and drop them in a folder on their unreliable document management system I'll call "SwearPoint" for no particular reason. We can work with them to try to fix it, but the engineers are in Bangladesh and the vendor doesn't allow us to talk to them, so we have to hope that their "technical" project managers actually understand the problem and can get it fixed. You can imagine the hilarity that ensues.
And then, of course, there are legacy systems like this one, which was deployed in the early 1980s. Two corporate entities ago, business decisions were made to replace aging legacy systems with new, modern ones. And then, there were new corporate owners with a new corporate strategy whose playbook was largely "raise rates as much as possible while running the network into the ground." The new owners have more or less the same "harvest" strategy. Because the systems are still technically slated for retirement, no new investments are being made into them. This includes compatibility work for internal tools, such as those used by the legal department.
And with that, it's time to ink this dot matrix printer ribbon again. I found an ancient inking kit in storage, which still somehow works! Naturally, though, when I opened the bottle I managed to spill ink all over my hands. I'm going to be here all night listening to a nine pin chorus, and dealing with paper jams in the tractor feed. But I'll meet the legal department's deadline, and might even be named Employee of the Month! Have a wonderful Thanksgiving, and I'll see you again in the winter.
References