Linux-Based Rootkits

Understanding rootkits is important, as they’re a common feature of cases where an attacker has successfully gained root/administrator access to a system, and because the effects can be damaging. I also expect all ‘Advanced Persistent Threats’ will involve a rootkit. The typical method is roughly this: hack the target, get root access, install some form of backdoor to maintain that access, and hide the activity from the system administrators. A very skilled attacker could have access to a backdoored system indefinitely.

I once shared the commonly-held belief several years ago that Linux was immune to malware. While there’s some truth to that, it’s largely because most of today’s malware’s used for getting most financial gain for minimum effort, so it makes more sense to target the most common platforms, and that happens to be Microsoft Windows.
The process for installing programs on a Linux machine is also more convoluted than Windows’ ‘click-and-run’ method, so it’s harder to get malware installed through social engineering.

Introducing the Rootkit

In itself, deploying a user-level rootkit is a very straightforward process on a UNIX/Linux for someone with administrator/root access, whether it’s the system’s owner or an attacker. This is primarily because Linux allows executables to be created, modified and swapped to automate whatever system-level tasks. A rootkit would have the following:

  • The main executable, or more usually several performing different functions.
  • Modified startup script.
  • A Command and Control (C&C) server.

The executable(s) are planted somewhere on the system, probably /usr/bin, and the entries are added to whatever script defines which services are called when the system is booted. Of course, one alternative is to plant an installation script that automates all this. Usually the rootkit is hidden by replacing existing executables with malicious ones. For example, the top and ls binaries can be replaced with versions that hide certain files and processes. Not even remote access to the infected machine would be logged.

Because the IP addresses of both target machine and the attacker will change regularly, and because the attacker must somehow locate the target across the Internet each time, they must both establish connections to something with a fixed domain/IP address – the C&C server. With the rootkit telling the server which IP address and port number it’s listening on, the server can relay stuff between the two endpoints. This also provides a proxy for the attacker, and therefore a layer of anonymity to hide behind.

The way to detect this is have another device on the network intercepting traffic, and logging all the IP addresses the hosts are connecting to. With so many binaries on the system, it seems finding the infected ones would be like searching for needles in a haystack, but we only need to look for the ones the attacker must modify, and check those against binaries we know are clean.

The Linux Rootkit (LRC) Backdoor

The Linux Rootkit, introduced in Malware: Fighting Malicious Code, by Ed Skoudis, includes a substitute login executable, which provides a backdoor login separate to the standard one, with its own username and password stored within the executable itself. This means the attacker’s login details won’t appear in the system’s password file, and if the system administrator does reset all the passwords, this won’t affect the backdoor.

Kernel-Mode Rootkits

What I’ve been describing so far is a ‘user-mode’ rootkit, which runs at the user-level just like any program, and it’s relatively easy to determine its presence. We know roughly which executables the attacker will replace, and they can be compared with versions known to be clean. A system administrator could also check the kernel’s output more directly by viewing the contents of the /proc directory instead of running those executables. It’s even possible to automate much of the checking with a relatively simple shell script, and there’s a good chance a targeted organisation would already do this, so something more covert would be used – a kernel-level rootkit.

The kernel is the core of the operating system. It’s essentially the interface between software applications and the hardware, converting between system calls and hardware signals. If we wanted to check the network interface, the number of processes in memory, or which filesystems are mounted, the kernel reads the raw signals from the relevant hardware devices and passes the information to whatever user-level application. The kernel has many other functions, but that’s the long and short of it.

If we wanted to add extra functions and support for certain non-generic hardware, we can add Loadable Kernel Modules using the insmod command, and those modules get loaded into the memory space of the kernel. They become part of the operating system’s core, and this is where kernel-mode rootkits come in. It’s possible to undermine the whole operating system by modifying the kernel, either by changing its source or by making it load a malicious kernel module. Such a rootkit would provide the attacker with similar functionality to a user-level rootkit, but goes further by preventing system calls that reveal the malicious activity reaching the application layer.

As luck would have it, I stumbled on one example of a kernel-level rootkit known as Adore, which appears to feature in a case study I’m currently doing. The source code for this can also be downloaded from Packet Storm Security.

How a Rootkit Could Improve Security

A rootkit, or something very similar that allows remote access and control while remaining hidden, could be used to our advantage as a security measure. It’s possible for an organisation to install a custom rootkit on all the devices it owns, while establishing a C&C server accessible over a VPN link. If any of the devices are reported mislaid or stolen, the organisation could use this system to trace those devices, disable them and erase any sensitive information they might store.


Case Study

The following is a summary of what’s presented in the case study. It begins with the discovery of the following in file inetd.conf on a workstation running Red Hat Linux: netstat stream tcp nowait root /usr/lib/netstat netstat A graphical CPU load monitor showed 100% load, the top command indicated only 90%, and there were apparently no running processes listed that accounted for the discrepancy. There were also no entries in /var/log/secure between the 7th and 14th September – a period of 7 days after Red Hat was installed on the workstation.

So, certain processes and actions were being concealed for whatever reason, and they weren’t being revealed by the ps, netstat, ls or top programs. It was reasonable to assume initially those executables were modified in some way to hide information, but this possibility was ruled out by comparing them with versions known to be clean.

The next test performed involved capturing and analysing traffic from the workstation using another host on the same network, and with the tcpdump and nmap tools. This revealed a list of open ports for unknown services, which again were being hidden from the application layer of the workstation itself. Having discovered this, ngrep was used for analysing the output from tcpdump, focussing on ports 3457 and 32411 in particular.

Analysis of Events

It’s certain that malware of some form was installed on the workstation, it was establishing connections on ports 3457 and 32411, while hiding whatever activity was associated with those ports. This was more sophisticated than a backdoor enabling remote access to the infected workstation. According to Sophos Security:

‘A Trojan program claims to have one function—and may even appear to carry it out—but actually does something different, usually without your knowledge. Trojans are often distributed with pirated software applications and keygens that create illegal license codes for downloadable software.’ (Sophos, 2009, pp. 77)

‘A rootkit is a piece of software that hides programs or processes running on a computer. It is often used to conceal computer misuse or data theft.’ (Sophos, 2009, pp.67)

Two things were obvious from the case study: a) there were unaccounted processes consuming 10% of CPU resources, suggesting hidden processes were running. The states of ports 3457 and 32411 were being hidden for a reason. b) Both ports were listening, which means they were expecting traffic, which in turn means the system had established a connection with something. The above description matches the symptoms of a rootkit, as defined by Sophos Security.

One of the central questions is how the attacker gained root access to the workstation in order to install the malware. As we saw in the case study, the /var/log/secure entries for the period between 7th and 14th September were apparently erased, which suggests the attacker had penetrated the system sometime during that period, and most likely gained root access through a vulnerability before covering his/her tracks.

The timing of the attack is also important. It occurred within a week of Red Hat Linux being installed on the workstation, so it’s likely the perpetrator was among the first to discover and exploit whatever vulnerability, instead of working from a database of known vulnerabilities. This suggests a reasonable amount of skill.

Basic Linux Rootkit

The rc.local file, which specifies the services to call on startup, was modified to call shell script initd, which in turn called /usr/sbin/rpc.status while loading a kernel module ipv6.o using the insmod command.

The file /usr/sbin/rpc.status was a user-level executable that contacted a specified IP address, and performed a series of actions once a connection was established. This is likely the main executable providing the backdoor into the system, although it doesn’t account for the hidden files, processes and network activity.

The Linux Rootkit, outlined in Malware: Fighting Malicious Code, by Ed Skoudis, includes a substitute login executable, which provides a backdoor login in addition to the standard one. If the attacker wants root access to the target system, he/she enters the backdoor username and password, which are both built into the executable. The backdoor is also unaffected if the administrator changes the password. (Skoudis, 2008, pp. 311 – 314)

Command and Control Server

Because the IP addresses of both the infected workstation and the attacker will change regularly, they must both establish connections to something with a fixed domain/IP address – a Command and Control Server. With the rootkit telling the server which IP address and port number it’s listening on, the server can relay information between the two endpoints. This also provides a proxy for the attacker, and therefore a layer of anonymity to hide behind.

The output from tcpdump revealed communication between IP addresses 10.6.6.6:32411 and 192.168.4.20:1844. This was the IP address of the local machine and another server outside the local network.

The rpc.status file appears to specify the IP address for the rootkit to contact, and the sequence of actions to perform once the connection is established.

The site http://www.xxxxxxxx.org is referred to in the ipv6.o file, and further investigation has revealed this is a chat site.
The service’s Acceptable Use Policy states the traffic going through its servers is not monitored or logged, which further adds to the complications of tracing an attacker.

It’s possible xxxxxxx.org was being used as a C&C server, by using a specific messaging channel to send commands to the target machine. The owners of xxxxxxx.org are likely unaware of this.

The Kernel-Mode Rootkit

The kernel is the core of the operating system. It’s essentially the interface between software applications and the hardware, converting between system calls and hardware signals. If we wanted to check the network interface, the number of processes in memory, or which filesystems are mounted, the kernel reads the raw signals from the relevant hardware components and passes the information to whatever user-level application.

Extra functions and hardware support for non-generic components can be added through Loadable Kernel Modules using the insmod command, and those modules get loaded into the memory space of the kernel. They become part of the operating system’s core, and this is where ‘kernel-mode’ rootkits come in. It’s possible to undermine the whole operating system by modifying the kernel, either by changing its source or by making it load a malicious kernel module. Such a rootkit would provide the attacker with similar functionality to a user-level rootkit, but goes further by preventing system calls that reveal the malicious activity reaching the application layer.

According to the recovered syslog entry, a series of actions were performed between 15:46 and 16:52 on September 20th, and this included the creation or modification of the ipv6.o module.

The Adore Rootkit

The ipv6.o module refers to a file called adore.c, which appears to be such a Kernel Module Rootkit introduced in Malware: Fighting Malicious Code.
Adore is apparently one of the most common Linux rootkits (Skoudis, 2008) and is a near perfect match for the malware described in the case study. The source code for Adore is available from Packet Storm Security’s web site. (Packet Storm Security, 2004)

The Adore Rootkit Source Files

The rootkit consists of several C and header files, with a shell script that automates installation once the files are planted on the target system. Earlier ipv6.o was identified as the malicious kernel module. Like most the other files, this was renamed to make it harder to identify the malicious code.

The rootkit also has an interface component, ava.c, which enables the attacker to control it remotely. Adore also enables some control through a hidden file in /proc, the exact filename being defined by the person installing it.

Conclusion

To summarise the incident, Red Hat Linux was installed on a workstation on 7th September 2000, and within a week somebody hacked the system, discovered a vulnerability, gained root access and installed the Adore kernel-mode rootkit to maintain access to the system. If the installation of the operating system and software applications were up to date, it’s unlikely their vulnerabilities were widely known, and so it’s likely the attacker was a skilled person who wasn’t relying on readily-available exploits.

This process involved accessing the system over Telnet, and modifying/creating files ipv6.o, init and rpc.status. The file rc.local was also modified to call shell script /usr/bin/initd on startup.

The /user/bin/initd calls the rpc.status backdoor and loads the ipv6.o kernel module with the insmod command. When run, rpc.status establishes a connection with a command and control server at xxxxxx.org, while ipv6.o hides this activity from the application layer.

As I’ve already pointed out, rootkits aren’t isolated factors – they are a final stage in system penetration attacks, where root access has already been gained. This means the best way to prevent an infection is to minimise the chances of potential attackers reaching this stage in the first place, which in turn means keeping software updated, setting strong passwords, properly configuring firewalls – in general, anything that improves the overall security of the system and reduces the number of vulnerabilities the attacker can exploit.

References

ANDROID SECURITY TEST. 2011. CarrierIQ. http://androidsecuritytest.com/features/logs-and-services/loggers/carrieriq/. [WWW]. (17th January 2012).

PACKET STORM SECURITY. 2004. File Archive: adore-ng-0.41.tgz. [WWW]. http://packetstormsecurity.org/files/32843/adore-ng-0.41.tgz.html. (17th March 2012).

SKOUDIS, E. 2008. Malware: Fighting Malicious Code. Massachussets, US. Prentice Hall. p. 303 – 428.

SOPHOS. 2009. Threatsaurus: The A-Z of Computer and Data Security Threats. [PDF]. Sophos Group. http://www.sophos.com/medialibrary/PDFs/other/. (12th March 2012).