Rootkit Techniques & How to Detect Them

In 2022, the Symbiote rootkit infected Linux servers across Latin American financial institutions — hiding itself inside running processes so completely that standard ps and netstat commands showed nothing suspicious. That’s the rootkit promise: persistence through invisibility. Understanding how they hide is the first step to finding them.

How Rootkits Hide: The /proc Discrepancy Trick

Most Linux rootkits manipulate what the kernel reports to userspace. They hook system calls like getdents64 to filter their own files and processes out of directory listings. The result: ls and ps lie to you, but the raw kernel data structures don’t.

A classic detection move is comparing what ps reports against what actually lives inside /proc. Run this on a suspect host — here, web01.corp.internal (192.0.2.14):

# List all numeric directories in /proc (each = a running PID)
for pid in /proc/[0-9]*; do
  pid_num=$(basename $pid)
  if ! ps -p $pid_num > /dev/null 2>&1; then
    echo "[HIDDEN] PID $pid_num exists in /proc but not in ps"
    cat /proc/$pid_num/cmdline 2>/dev/null | tr '\0' ' '
    echo
  fi
done

Sample output on a compromised box:

[HIDDEN] PID 3847 exists in /proc but not in ps
/usr/sbin/.sysd-net --connect 192.0.2.99 --port 4444

That output is your smoking gun. PID 3847 is invisible to ps because the rootkit’s hooked getdents64 filters it out — but /proc is a live kernel filesystem that the hook missed. The cmdline reveals a reverse shell phoning home to 192.0.2.99 on port 4444. Your next move: isolate the host immediately, capture a memory image with LiME, and block 192.0.2.99 at the perimeter before pulling the plug.

Running rkhunter Against a Live System

rkhunter (Rootkit Hunter) is a scanner that checks for known rootkit signatures, suspicious file permissions, and tampered binaries. It’s not magic — a sufficiently advanced rootkit can fool it — but it catches the common stuff fast.

Install and run a full scan on db02.corp.internal (192.0.2.31):

sudo apt install rkhunter -y
sudo rkhunter --update
sudo rkhunter --check --sk --rwo

--sk skips the keypress prompts. --rwo means report warnings only — no noise, just hits. Trimmed output:

[13:42:07] Checking for Diamorphine rootkit            [ Warning ]
[13:42:07]   Found suspicious file: /usr/lib/libprocesshider.so
[13:42:08] Checking /usr/bin/ps                        [ Warning ]
[13:42:08]   File properties have changed:
[13:42:08]   Current hash: 3d9f1c8a...  Stored hash: b7e24d01...
[13:42:09] Checking for hidden ports                   [ Warning ]
[13:42:09]   Found hidden port: 4444/TCP

Three separate signals, all pointing the same direction. libprocesshider.so is a known userland rootkit component that abuses LD_PRELOAD to intercept library calls. The ps binary hash mismatch means the attacker replaced the system binary — a dead giveaway. Hidden port 4444 aligns with what we already saw in the /proc walk on the other host. At this point you’re not investigating a maybe — you’re doing incident response.

Grab the full log at /var/log/rkhunter.log for your IR report. Cross-reference the file hashes against a known-good baseline from your configuration management system or a clean OS image.

Checking for LD_PRELOAD Hijacking

Userland rootkits often don’t touch the kernel at all. Instead, they abuse LD_PRELOAD — an environment variable that forces the dynamic linker to load a library before everything else, letting the attacker override standard functions like readdir and fopen. Check two places every time:

# Check system-wide preload file
cat /etc/ld.so.preload

# Check for LD_PRELOAD set in init environments
strings /proc/1/environ | grep LD_PRELOAD

On a clean system, /etc/ld.so.preload is empty or absent. If you see this:

/usr/lib/libprocesshider.so

That library is being injected into every process on the machine. The attacker gets function-level control over anything that runs. Don’t just delete the file — it’ll likely respawn from a cron job or systemd unit. Find the persistence mechanism first: grep -r libprocesshider /etc/cron* /etc/systemd/system/ ~/.bashrc /etc/profile.d/ before you remove anything.

What To Do Now

Pick one production Linux host you haven’t audited recently. SSH in and run the /proc vs ps comparison script from the first example, then check /etc/ld.so.preload. Both take under two minutes. If either returns output, stop everything and treat it as an active compromise. If they come back clean, schedule a weekly rkhunter --check --rwo via cron and pipe the output to your SIEM. You can’t defend against what you can’t see — so start looking.