The ReadyNAS 312 – Coming up with a Plan B

Note: This post is part 2 in a ongoing, multi-part series. The previous post can be found here.

The situation so far…

Before diving deeper into this, let’s sum up all our findings from post 1:

• We have a Netgear ReadyNAS 312 which is now running outdated and obsolete software, with no vendor-provided updates in sight.

• The NAS itself seems like a fairly standard X86 PC built into a small form-factor chassis, with all capabilities one would expect from a normal PC. It might even have a hidden VGA-port somewhere!

• There’s also an internal flash-volume somewhere, still unexplored.

• The NAS’s firmware ReadyNAS OS is based on Debian Linux (Jessie), and Plan A was to simply dist-upgrade that in place to a more modern version.

• Simply upgrading the base system might leave the device in a non-working state, due to the proprietary ReadyNAS components installed alongside not being built for newer Debian versions.

Deciding upon the road ahead

I consider myself a fairly proficient Linux-user and “home lab”-grade Linux-admin. I do need a NAS at home, but I do not need that NAS to run a Netgear specific implementation.

Basically, if I can get this thing to run Debian, and plain Debian only, I can again turn that into a workable NAS in no time. I’ll be happy and then this unit can be considered fully “repurposed”.

So Plan B is pretty not that far off from Plan A: I want to get this unit to run an up to date version of Debian, but I don’t care about ReadyNAS-specific stuff at all.

Mapping out what we have

So with our goal re-adjusted and settled, let’s see how we can get there!

In my experience, if you want to rebuild something, it’s often helpful to see how the stuff you already have was built. You’re probably going to be reusing quite a bit of it.

Or at least, that’s what you hope.

The booting

If we want to reuse the existing settings supporting the boot-process in already place, we need to know what’s where. This unit is probably too old to be rocking UEFI, and having the efibootmgr command missing confirms this.

root@ReadyNAS:~# efibootmgr
-bash: efibootmgr: command not found

Which means there’s almost a 100% chance that the NAS is using legacy BIOS MBR boot.

Even if it’s not required with UEFI, it’s still commonplace to use Linux-specific boot-loaders like GRUB (or on an old unit like this, even LILO). But when legacy BIOS MBR boot is used, you have no other option. You need such a boot-loader or it won’t boot.

Which boot-loader is being used, and how it’s booting the system is something we expect to find traces of somewhere in /etc, where its configuration should be stored.

Stranger things, NAS edition

Trying to find the configuration-files for the boot-loader, I uncovered a several strange things:

• There’s no boot-loader configuration/templates in /etc.
• There’s also no boot-loader config-files in the /boot-folder. In fact that folder is empty, and there’s no /boot-volume mounted, at all.
• While I expected to find a base Debian install on a firmware-volume, and my NAS-data on a separate RAID-volume, I instead found two different RAID-volume(s) created from the inserted NAS disks.
• These two volumes contained both my NAS-data and the Debian Jessie root-filesystem.

Hold on now. Hold horses. We need to fully realize the gravity of that last finding.

This NAS evidently runs its firmware from the disks inserted into it, which is also used for storing your data. And that firmware has no boot-code.

Clearly we’re missing something here. Clearly!

So let’s just take a look at what a simple mount tells us:

The only block-devices mounted are mdadm-type devices.

root@ReadyNAS:~# mount | grep "/dev/md"
/dev/md0 on / type btrfs (subvolid=5,subvol=/)
/dev/md127 on /data type btrfs (subvolid=5,subvol=/)
/dev/md127 on /apps type btrfs (subvolid=258,subvol=/.apps)
/dev/md127 on /home type btrfs (subvolid=256,subvol=/home)
/dev/md127 on /run/nfs4/home type btrfs (subvolid=256,subvol=/home)

And probing into that, mdadm tells us that these two soft-raid devices are created on top of partitions of the two same physical volumes:

So we have 2 raid devices, created from different partitions (1 and 3) across 2 physical volumes (/dev/sda and /dev/sdb) …?

I’m not sure what’s industry standard in the NAS-business, but I sure as heck don’t construct my RAIDs that way. And what’s up with the unused partition 2? So many questions!

Apart from that we can see it’s using btrfs, which is a standard Linux file-system, so I should be able to mount those volumes using any standard distro. My data should be fully portable to a new setup I create.

Phew! That’s good news at least.

The mysterious missing boot-code

We know for a fact that something has to boot this system, because we can observe the system being booted. Time to check the kernel logs!

The command dmesg will give us the full kernel boot-log which is probably more than we need. For now we’re mostly interested in storage devices. Then this one-liner here will help us sum it up:

root@ReadyNAS:~# dmesg | grep logical
[ 3.494963] sd 0:0:0:0: [sda] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB)
[ 3.969874] sd 1:0:0:0: [sdb] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB)
[ 8.695234] sd 6:0:0:0: [sdc] 489472 512-byte logical blocks: (251 MB/239 MiB)

Whoah. Who’s that guy called sdc ? Looking into /dev/ there’s no mention of him there!

That means that the device itself is not only boote from a file-system which is hotswapped/exchanged for another one sometimes during the boot-process. It’s also booted from a device which someone is working hard to keep hidden from the running system itself.

That sneaky bastard!

There may be ways to restore this sdc-device into the device-tree, but that’s past my level of tinkering.

What I feel like now is that it’s time (to try) to turn this process around 180-degrees.

• So far I’ve been working from the inside of the OS, trying to work my way out to the machine.
• But clearly we now need to do the opposite: Work our way from the outside, from the machine, into the OS. We need to see this thing boot.

Now as far as I know we have 2 possibilities for that:

• Open up the case and try to locate the VGA-pins on the mainboard and solder on a connector. Clearly a hardware-mod.
• Hope there’s a more accessible serial-port somewhere we can make use of.

Lucky for us there’s a simple way to check for the latter:

root@ReadyNAS:~# dmesg | grep "Kernel command line"
[ 0.000000] Kernel command line: console=tty0 console=ttyS0,115200 hpet=disable initrd=initrd.gz reason=normal BOOT_IMAGE=kernel

Translation: There is a console attached to to /dev/ttyS0, to a serial-port device.

Now why would someone put that into their kernel-config unless there is a serial-port to wire up?

We’ll dive into that in the next part!

(Continued in part 3)