Palo Alto NGFW Insider
by ]{nightVision
With companies moving toward Next Generation Firewalls (NGFWs) and all their new capabilities, I was curious as to what really makes this different.
As a hacker, the first thought is "Let's void the warranty and take a look inside!" Luckily, I was able to get a hold of one through a friend who was willing to let me find out how secure it would be for his company. To inspire others to "void warranties," I'll explain the entire process, as each investigation becomes a learning experience.
Before starting, here are some initial observations that led me down this path.
The firewalls are sold as appliances and closer inspection shows it's simply some type of Linux. The standard access you have is through a console port, SSH, or a Web GUI. The console port and SSH logins give you a limited shell, apparently locked down to a set of commands you are shown. One of the commands gives you a pretty standard top output for CPU, memory, and processes, so it's looking more like a Linux system beneath the shell.
Let's Check Out the Internals
First, I upgraded the system to the latest version of software (5.0.2) to make sure I was looking at the most up to date code they had available.
I have the smaller model, so I opened the system to see what trouble I could get into. There are basically two other ways to attempt to get access to the system. On the motherboard itself is a JTAG connector, and the hard drive appears to be a standard SATA hard drive. So to begin with, I removed the hard drive and attached it to my forensics hard drive controller. Granted, any USB-to-SATA controller would work. The next question became what type of OS was I going to connect this to.
So my system happens to be Windows, and the drive is formatted with more standard Linux partitions, which normally isn't compatible. However, people have made programs you can install to let you read these types of partitions. I'd recommend one that reads but doesn't write to the disk. Most won't guarantee they won't corrupt the file systems on writing, but reading is pretty safe. So I copied all of the files from the individual partitions onto my server, and started poking around. It turns out this isn't a perfect solution, as I'll cover later.
Initial Observations
Looking at the file structure shows this is a flavor of Linux.
So there are a couple of areas we can look at on the file system to learn more about the system and determine what other "opportunities" we might have.
(sda2) /etc/mtab documents how these partitions are loaded so that we know where to look for configuration info:
Maintenance Mode can be entered upon startup to reset the system and do limited recovery work like fsck. This mode is another interesting area to research in the future.
(sda2) /var/log/dmesg is the file saved from the system startup. Much of this can be seen if you login to the serial console when you power this on. This also shows on startup that an internal flash drive was mounted, which appears to contain the /boot information, since I didn't find that on the hard drive partitions.
MIPS - a flavor of Linux being run to handle the CPU on the motherboard. This causes some challenges for us as these aren't as common and documented, much less than Linux on Intel processors. There is shellcode for exploiting MIPS systems, but I'm sure this would take quite a bit more investigation to take advantage of. Conversely, the same challenges it presents us leads to a more secure platform for a security device.
/etc/passwd and /etc/shadow - good news as an owner, there appear to be no backdoor users, you appear to only have the PA accounts enabled, all the rest appear to be expired so you can't use them. For root, they expired the password, and further set the /etc/security/access.conf file so you can't login as root.
Now, looking at the login shell for all other enabled users, they are defined to use /usr/local/bin/cli as their shell.I In looking for the CLI, I found a limitation with using Windows to look at the Linux partition. The Linux partition software didn't know how to handle links, so it ignored them. Later, while looking at the actual RPM for CLI, I could see it created a soft link to pan_cli in the same directory.
Web GUI
(sda2) /var/appweb/htdocs is the root directory for the main web application.
Looking around shows you what lies beneath the surface, something I don't think they wanted you to know. These URLs are pointing at the Management interface, leaving it at the default IP address.
Whoops, they left a Console Debugging application on the system: https://192.168.1.1/php/utils/debug.php
Login to the interface and open this link in another window. Click the debug checkbox. Now when you go back to your GUI and do anything, all the underlying data is captured in the debugger. This includes cookies, data sent and received by the GUI, etc.
Come on developers, a little cleanup couldn't hurt you: https://192.168.1.1/php/readme.txt
Under /var/appweb, you'll also see the directories used for GlobalProtect (their VPN portal) and CP (their Captive Portal). These are less interesting, as they are relatively simple sites.
Content Updates
To me, the value in an NGFW is the combining of AV, Intrusion Prevention System (IPS), and botnet detection to the standard firewall and URL capabilities.
For a lot of companies, this is their "secret sauce" that they want to keep private to show their value. Looking around, I found the content updates stored in (sda5) /mgmt/updates. So looking in the subdirectories, you find the content files. Looking at them in a hex editor, you can see the .db files are actually SQLite databases. Loading up SQLite DB Browser, you can look at all of the data.
curav/botnet.db - tables - malware_domains, ddns_domains (malware and dynamic DNS).
curav/virus.xml.db - virus information number, name, description, and severity.
curav/virus_signatures.db - virus name, action, signature (looks like mostly pattern matching).
curcontent/global/threats.xml.db - IPS threats, references, actions, categories. Interesting for threats - they actually stored the signatures in an XML file.
curcontent/global/global.xml and some more interesting XMLs to check out.
If you search other directories under (sda5) /mgmt/updates, you'll find other .db and .xml files that store configuration or interesting information.
BrightCloud URL database seems to be done differently, probably due to the sheer number of URLs and performance concerns. This looks to be a non-standard database showing category names, but all of the URLs are hashes. I'm sure this makes the URL checking much quicker and happens to hide this information from us. Palo Alto has now come out with their own PAN URL database, but I don't have a license to pull down those update file.
Interesting Finds
/etc/yum.repos.d/panos.repo - Yum package manager repositories. Shows the IP address of their internal repository server.
So if anyone ever visits Palo Alto corporate, check out this server!
Future Work
From this, there are a couple of "opportunities" I plan to explore for a Part Two:
- The JTAG connection looks interesting, but may be of limited use.
- Looking at the flash ROMs to learn more about the system itself.
- Document installed packages and determine if they are all up to date, or, if there are older ones, are they vulnerable?
- Use write access to drive to re-enable the root user and/or create another user and change the /etc/passwd file to give them a standard shell.
- Maintenance Mode - are there more options or possibilities here?
And many more... the more I dig, the more ideas come to light.