The attacks are coming from inside SNMP!

The snmp binary (/nova/bin/snmp) is part of the system package. However, there are various other packages that want to add their own functionality to snmp. For example, the dhcp package. In the image below, you can see that /pckg/dhcp has an /snmp/ subdirectory.

Functionality added to snmp by the dhcp package

When the snmp binary starts up, it will loop over all of the directories in /pckg/ and look for the /nova/lib/snmp/ subdirectory. Any shared object in that subdirectory gets passed to dlopen() and then the shared object’s autorun() is invoked.

Since the dhcp package is mounted as read-only, an attacker can’t modify the loaded shared object. However, as we’ve established, /pckg/ is read-write so an attacker can introduce their own directory structure (e.g. /pckg/snmp_xploit/nova/lib/snmp/). Any shared object stored there would be loaded by snmp.

One of these things is not like the others

It’s pretty neat that an attacker can hide within a process that lives in read-only space! But it’s even more useful when combined with a vulnerability that can write files to disk like CVE-2019–3943 or CVE-2018–14847.

I wrote a proof of concept to illustrate the use case with CVE-2019–3943. Essentially, an authenticated attacker can create the /pckg/ directory structure using the vulnerability’s directory traversal.

Once the directories are created, the attacker needs to drop a shared object on disk. Luckily, CVE-2019–3943 can do that as well. Obviously, a real attacker can execute anything from their shared object, but for the proof of concept I create the 6.41+ backdoor file directly from a constructor function.

The PoC will even stop and restart the SNMP process to ensure the shared object gets loaded without a reboot of the system.

Since /pckg/ is in tmpfs space, the directory structure the script creates would be removed on a reboot even if the PoC didn’t delete it.

I’m in your /rw/lib, executing as one of your dudes

Similar to the above, I found that I could get system binaries to load libraries out of /flash/rw/lib. This is because /rw/lib/ is the first entry in the LD_LIBRARY_PATH environment variable.

Load libraries from read-write space? What could go wrong.

The great thing about loading libraries from /rw/lib/ is that, because it’s persistent file space, the shared object will persist across reboots. The only challenge is figuring out which library we want to hijack. The obvious choice is libc.so since it’s guaranteed to be loaded… everywhere. But RouterOS uses uClibc and, quite frankly, I didn’t want to deal with that.

Thankfully, I came upon this.

Hello libz!

/nova/bin/fileman loads libz. fileman is the system binary that handles reading and writing from the user’s /rw/disk directory via Winbox or Webfig. It gets executed when the user navigates to the “Files” interface, but it shuts down after the user has navigated away and it remains idle for a minute.

To compile the malicious library, I simply downloaded libz 1.2.11 and added this constructor to deflate.c:

void __attribute__((constructor)) lol(void)

{

int fork_result = fork();

if (fork_result == 0)

{

execl("/bin/bash", "bash", "-c",

"mkdir /pckg/option; mount -o bind /boot/ /pckg/option",

(char *) 0); exit(0);

}

}

You can see, once again, I’ve just chosen to create the backdoor file. For this proof of concept, I cross compiled the new libz.so to MIPS big endian so that I could test it on my hAP router.

Once again, the proof of concept uses CVE-2019–3943 to create the “lib” directory and drops the library on disk.

However, unlike the SNMP attack, /rw/lib/libz.so will survive reboots and it actually gets loaded quite early in the startup sequence. Which means after every reboot, the backdoor file will get created during start up.

Signature verification matters until it doesn’t

One of the more interesting things stored in /flash/ is the files in /flash/var/pdb/.

“Hey, aren’t those the names of all the packages I have installed?”

It turns out that this is where RouterOS stores all of the installed NPK files. Oddly, as root, they are all writeable. I can tell you from experience, you don’t want to overwrite the system package.

Haha! Did I just get you to watch the system rebooting over and over again?

When I learned I could break the entire system by messing around with the system package, I got kind of curious. What if I was a little more careful? What if I just overwrote the package’s squashfs filesystem? Would that get mounted?

I wrote a tool called modify_npk to test this out. The tool is pretty simple, it takes in a valid MikroTik NPK (e.g. dude-6.44.5.npk) and a user-created squashfs. The tool removes the valid MikroTik squashfs section and inserts the user’s malicious squashfs. In theory, modify_npk generates a perfectly well formed NPK… just with a new internal squashfs.

The problem is that MikroTik enforces signature verification when installing NPK packages. If you try to install a modify_npk package then RouterOS will flag it as broken and reject it. See wrasse.npk in the following log file:

I’m not broken you’re broken

Which is obviously good! We can’t have weirdos installing whatever they want on these systems. But what if we install it ourselves from our root shell?

Don’t feel bad. I didn’t know echo * was a thing either.

In theory, RouterOS should always run a signature check on the stored NPK before mounting their filesystems. Since they are all read-write it only makes sense, right?

Oops

In the above image, you can see wrasse was successfully installed on the system, bad signature and all! Obviously, that should mean the squashfs I created was mounted.

┬┴┬┴┤(･_├┬┴┬┴

Of course, just having the malicious squashfs mounted isn’t the end, because the filesystem I created actually contains an rc script that will create the backdoor file at startup.

This is quite useful as it will persist through reboots. Although, users can catch this particular attack by using the “Check Installation” feature.

MikroTik silently patched this bug in 6.42.1. I say “silently” because I don’t see any specific release note or communication to the community that indicates that they decided to enforce signature verification on every reboot.

RC scripts everywhere

RouterOS uses rc scripts to start processes after boot and to clean up some processes during shutdown. The OS has a traditional /etc/rc.d/run.d/ file structure, that we will talk about, but it also has (or had) other places that rc scripts are executed from as well.

/flash/etc/

As mentioned, RouterOS has a traditional /etc/ directory, but since the directory is read-only attackers can’t modify or introduce scripts. However, RouterOS does have a second /etc/ off of the persistent read-write /flash/ space.

At first glance, it doesn’t appear all that useful as far as rc scripts go. However, as BigNerd95 pointed out in his Chimay-Red repository, you can create an /rc.d/run.d/ subdirectory off of /flash/etc/ and any rc script stored within will be treated as a normal rc script on startup and shutdown.

In the example below, you can see I create /flash/etc/rc.d/run.d/ and echo the script S89lol into place. After a reboot, the script is executed and the developer backdoor is created.

This behavior was removed after 6.40.9. Up until then, however, this was a very simple and convenient persistence mechanism.

/rw/RESET

RouterOS has a bunch of scripts sitting in /etc/rc.d/run.d/, but there are two I want to specifically talk about. The first one is S08config and that is because through 6.40.5 it contained the following logic:

elif [ -f /rw/RESET ]; then

/bin/bash /rw/RESET

rm -rf /rw/RESET

Meaning that if /rw/RESET existed then S08config would execute it as a bash script at start up. This is an obvious persistence mechanism. So obvious that it was actually observed in the wild:

Somehow this forum user obtained MikroTik’s debug package and was able to examine some files post exploitation. Here we can see the attacker using /rw/RESET to execute their /rw/info binary. Perhaps seeing this used in the wild is why MikroTik altered S08config’s behavior.

/rw/DEFCONF

Similar to /rw/RESET, the contents of /rw/DEFCONF can be executed thanks to an eval statement in S12defconf.

defcf=$(cat /rw/DEFCONF)

echo > /ram/defconf-params

if [ -f /nova/bin/flash ]; then

/nova/bin/flash --fetch-defconf-params /ram/defconf-params

fi

(eval $(cat /ram/defconf-params) action=apply /bin/gosh "$defcf";

cp "$defcf" $confirm; rm /rw/DEFCONF /ram/defconf-params) &

This was first introduced in 6.40.1, but unlike /rw/RESET this hasn’t been fixed as of 6.45.3. In fact, this is the method that Cleaner Wrasse will use to establish reboot persistence on the router. I wrote a proof of concept using CVE-2019–3943 to show how a remote authenticated attacker can abuse /rw/DEFCONF to achieve the backdoor and establish persistence.

/pckg/

As we saw in the signature verification portion of this writeup, each package off of /pckg/ can have an /etc/rc.d/run.d/ directory containing rc scripts. /pckg/ is part of a tmpfs, so while anything an attacker creates in /pckg/ won’t persist across reboots, new rc scripts will get executed at shutdown.

How is that useful? One thing I didn’t mention about /rw/DEFCONF is that its existence on the system can cause issues with logging in. Cleaner Wrasse avoids this issue by staging a file in /rw/.lol and then creating an rc script in /pckg/ that creates the /rw/DEFCONF file on shutdown. In that way, Cleaner Wrasse avoids the login problem but ensures /rw/DEFCONF exists when the system starts up again.

Simply copy /rw/.lol to /rw/DEFCONF on shutdown. Easy mode.

The symlink of survival

Many of the proofs of concepts I mention in this blog use CVE-2019–3943, but it was patched for good in May 2019 (6.43.15 Long-term). Unless you use Kirils Solovjovs’ USB jailbreak, there are no more public methods to enable the backdoor file and root the device. So how am I able to do this?

Root shell on most recent release: 6.45.3 Stable

The answer is simple. When I was still able to exploit the router using CVE-2019–3943, I created a hidden symlink to root in the user’s /rw/disk directory.

The .survival symlink points to /

After an upgrade, you need only FTP into the router and traverse the symlink to root. From there you can achieve execution in one of the many ways that you want. In the following image, I drop libz.so into /rw/lib/ to enable the backdoor.

RouterOS doesn’t offer a way for a normal user to create a symlink, so you can only do it via exploitation. But RouterOS doesn’t try to remove the symlink either. As long as that’s the case, we can continue using the survival symlink to reestablish the root shell after upgrade.

Neither Winbox or Webfig displays hidden files. It’s probably worthwhile to occasionally check your user directory via FTP to ensure nothing is hidden there.