Compiling a Custom Linux Kernel & Adding systemd (Part 2)

In Part 1, we built a minimal Linux system using BusyBox and a prebuilt kernel. You got a shell running inside a container, mounted pseudo-filesystems, and saw the boot sequence from init to prompt.

That was the "hello world" of custom Linux. Now we're doing the real thing.

Today we're going to compile our own kernel from source, configure only the hardware support we actually need, and compress it to under 10MB. Then we'll rip out that hand-written init script and replace it with systemd — the same init system that runs on virtually every modern Linux distribution. Finally, we'll get a real service running (OpenSSH) so you can actually log into your custom system remotely.

This is the post that turns "I sort of understand Linux" into "I know exactly what every layer does." Let's go.

Prerequisites

You need everything from Part 1, plus a few extras:

• A Linux machine (kernel compilation doesn't work well on macOS/Windows — use a VM if needed)
• 4GB+ RAM (compilation is memory-hungry)
• 10GB free disk space (source tree + build artifacts + tools)
• 30-45 minutes (most of that is compile time — you can grab coffee)

Install the build dependencies:

sudo apt-get update sudo apt-get install -y build-essential libncurses-dev bison flex libssl-dev \ libelf-dev bc rsync cpio wget xz-utils systemd-container 

That systemd-container package is key — it gives us systemd-nspawn, which is like Docker but uses your custom kernel directly. More on that later.

Step 1: Download the Kernel Source

mkdir -p ~/kernel-build && cd ~/kernel-build # Download latest stable kernel (6.12.x as of writing) wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.12.tar.xz # Extract tar -xf linux-6.12.tar.xz cd linux-6.12 

The full source tree is about 1.4GB uncompressed. That's 30+ million lines of code spanning every CPU architecture, every driver, every filesystem Linux supports. Your job is to trim that down to what you actually need.

Step 2: Configure the Kernel — What to Keep, What to Kill

This is where most people either panic or get obsessed. Don't do either. The kernel config is just a giant list of yes/no questions: "Do you need Bluetooth support?" "Do you need Apple ADB keyboard support?" "Do you need the Siemens R3964 line discipline?"

You answer maybe 200 of these. The kernel has thousands.

Start from a Minimal Base

# Start with the tinyconfig — bare minimum, almost nothing enabled make tinyconfig 

tinyconfig gives you a kernel that can boot and... that's about it. No networking, no filesystem beyond the bare minimum, no device drivers worth mentioning. It compiles in under 2 minutes and produces a ~2MB compressed image. But it's useless.

We need to add just enough to be practical.

What We Need

Here's my checklist for a kernel that boots in a container, has networking, and can run systemd:

# Enable 64-bit support (if on x86_64) scripts/config --enable CONFIG_64BIT # Essential for booting scripts/config --enable CONFIG_PRINTK scripts/config --enable CONFIG_BUG scripts/config --enable CONFIG_MULTIUSER scripts/config --enable CONFIG_FUTEX # Filesystems — we need at least ext4 and proc scripts/config --enable CONFIG_EXT4_FS scripts/config --enable CONFIG_PROC_FS scripts/config --enable CONFIG_SYSFS scripts/config --enable CONFIG_TMPFS scripts/config --enable CONFIG_DEVTMPFS scripts/config --enable CONFIG_DEVTMPFS_MOUNT # Networking — systemd won't work without this scripts/config --enable CONFIG_NET scripts/config --enable CONFIG_INET scripts/config --enable CONFIG_UNIX scripts/config --enable CONFIG_NETDEVICES scripts/config --enable CONFIG_NET_CORE scripts/config --enable CONFIG_PACKET scripts/config --enable CONFIG_UNIX_DIAG # cgroups — systemd is built on these scripts/config --enable CONFIG_CGROUPS scripts/config --enable CONFIG_CGROUP_FREEZER scripts/config --enable CONFIG_CGROUP_DEVICE scripts/config --enable CONFIG_CGROUP_CPUACCT scripts/config --enable CONFIG_CGROUP_PERF scripts/config --enable CONFIG_CGROUP_BPF scripts/config --enable CONFIG_MEMCG scripts/config --enable CONFIG_BLK_CGROUP # Namespaces — container/security isolation scripts/config --enable CONFIG_NAMESPACES scripts/config --enable CONFIG_USER_NS scripts/config --enable CONFIG_PID_NS scripts/config --enable CONFIG_NET_NS scripts/config --enable CONFIG_UTS_NS scripts/config --enable CONFIG_IPC_NS # systemd needs these specific kernel features scripts/config --enable CONFIG_FHANDLE scripts/config --enable CONFIG_EPOLL scripts/config --enable CONFIG_SIGNALFD scripts/config --enable CONFIG_TIMERFD scripts/config --enable CONFIG_EVENTFD scripts/config --enable CONFIG_INOTIFY_USER scripts/config --enable CONFIG_FANOTIFY scripts/config --enable CONFIG_AUTOFS_FS scripts/config --enable CONFIG_PROC_SYSCTL # devtmpfs — automatic device node creation scripts/config --enable CONFIG_DEVTMPFS scripts/config --enable CONFIG_DEVTMPFS_MOUNT # Kernel module support (so we can load .ko files later if needed) scripts/config --enable CONFIG_MODULES scripts/config --enable CONFIG_MODULE_UNLOAD # Security features scripts/config --enable CONFIG_SECCOMP scripts/config --enable CONFIG_SECCOMP_FILTER # tmpfs at /run and /tmp (systemd expects these) scripts/config --enable CONFIG_TMPFS_POSIX_ACL # Make sure we can boot in a container (no framebuffer, no keyboard, etc.) scripts/config --disable CONFIG_VT scripts/config --disable CONFIG_INPUT scripts/config --disable CONFIG_SERIO scripts/config --disable CONFIG_DRM scripts/config --disable CONFIG_SOUND scripts/config --disable CONFIG_USB scripts/config --disable CONFIG_WIRELESS scripts/config --disable CONFIG_WLAN scripts/config --disable CONFIG_BT scripts/config --disable CONFIG_WIRELESS_EXT scripts/config --disable CONFIG_MAC80211 

The "Why" Behind These Choices

I'm not listing these randomly. Every single enable here has a reason:

• cgroups and namespaces — systemd is fundamentally a cgroup manager. Without these kernel features, systemd cannot start. Period. If you ever wonder why systemd "requires" so much from the kernel, this is why.
• FHANDLE, EPOLL, SIGNALFD, TIMERFD, EVENTFD — these are Linux-specific system calls that systemd uses heavily for event loop management. They're more efficient than traditional select()/poll() and systemd assumes they exist.
• DEVTMPFS + DEVTMPFS_MOUNT — without this, you'd need a udevd running to populate /dev. With it, the kernel auto-populates device nodes at boot. For a minimal system, this is essential.
• SECCOMP — lets systemd sandbox services. Even in a toy system, it's good practice and costs almost nothing.

Everything I'm disabling (VT, INPUT, SOUND, USB, BLUETOOTH, WIRELESS) is stuff a container doesn't need. Your kernel is running inside a container — the host kernel handles the real hardware. We just need the kernel to manage processes, memory, and networking.

The Interactive Way (Optional)

If you want to see what you're enabling and tweak things visually:

make menuconfig 

This opens an ncurses interface where you can browse categories, read help text for each option, and see what's enabled. I recommend doing this at least once just to see how massive the kernel config really is. It's humbling.

Step 3: Compile

# How many CPU cores do you have? nproc # Compile using all cores (replace 8 with your nproc output) make -j8 2>&1 | tail -20 

On a modern 8-core machine, this takes about 15-20 minutes. On a 2-core VM, plan for 45 minutes. Go make coffee. Actually, go make lunch.

When it finishes, check what you got:

# The raw kernel image ls -lh vmlinux # The compressed boot image (this is what we care about) ls -lh arch/x86/boot/bzImage 

My build produced:

vmlinux: ~32MB (uncompressed, with debug symbols) bzImage: ~7.8MB (compressed, bootable) 

Under 10MB. And that's with networking, cgroups, namespaces, and all the systemd prerequisites. The average distribution kernel is 80-120MB. We just threw away 90%+ of the code and kept only what we need.

Stripping It Further

If you want to go even leaner:

# Strip debug symbols from the modules find . -name '*.ko' -exec strip --strip-debug {} \; # Or compile without debug info from the start make clean scripts/config --disable CONFIG_DEBUG_INFO make -j8 

Without debug symbols, my bzImage dropped to 6.2MB. That's a full Linux kernel with networking and cgroup support in less space than a single high-res photo.

Step 4: Build the Root Filesystem with systemd

This is where Part 2 gets interesting. In Part 1, our init was a 20-line shell script. Now we're replacing it with the same init system that runs on Fedora, Ubuntu, Arch, and Debian.

Why systemd Gets Hate (And Why It Doesn't Deserve All Of It)

I know, I know. The "systemd controversy" is older than most junior sysadmins. But here's the thing: systemd is just a process manager that understands cgroups, sockets, and dependencies. Yes, it does more than traditional init systems. But "more" isn't automatically bad — it's only bad when you don't understand what it's doing. Which is exactly why we're building it from scratch.

When you've compiled a kernel, set up cgroups by hand, and watched systemd start on top of it, you'll understand exactly what it does. No magic, no mystery.

Create the Root Filesystem

cd ~/kernel-build mkdir -p rootfs # Use debootstrap to get a minimal Debian root filesystem sudo debootstrap --variant=minbase --arch=amd64 trixie rootfs http://deb.debian.org/debian # Or if you don't want to use debootstrap, copy from Part 1 and add systemd manually: # The debootstrap approach is faster and gives us a real package manager 

Install systemd and OpenSSH

# Chroot into our new root filesystem sudo chroot rootfs /bin/bash # Inside the chroot: apt-get update apt-get install -y --no-install-recommends systemd systemd-sysv openssh-server # Clean up apt cache to keep the image small apt-get clean # Set a root password (you'll need this for SSH) echo "root:customlinux" | chpasswd # Exit the chroot exit 

Create a systemd Service

Let's create a simple service that proves systemd is actually managing things:

# Create a custom service that logs boot time sudo tee rootfs/etc/systemd/system/boot-timer.service > /dev/null << 'EOF' [Unit] Description=Boot Timer Service After=network.target [Service] Type=oneshot ExecStart=/bin/bash -c 'echo "System booted at $(date)" > /var/log/boot-timer.log' RemainAfterExit=yes [Install] WantedBy=multi-user.target EOF # Enable it sudo chroot rootfs systemctl enable boot-timer.service 

This service runs once at boot, writes the timestamp to a log file, and that's it. Simple, verifiable, and it proves systemd is working.

Set Up the Root Filesystem Structure

# Make sure the necessary directories exist sudo mkdir -p rootfs/{run,tmp,proc,sys,dev,sys/fs/cgroup} # Create the machine-id (systemd requires this) sudo systemd-id128 new > rootfs/etc/machine-id sudo chmod 0444 rootfs/etc/machine-id 

Step 5: Boot It with Your Custom Kernel

Here's the moment of truth. We're going to boot our custom-compiled kernel with a root filesystem that runs systemd:

cd ~/kernel-build # Boot using systemd-nspawn (container runtime that uses your actual kernel) sudo systemd-nspawn \ --boot \ --kernel=linux-6.12/arch/x86/boot/bzImage \ --root=rootfs \ --machine=custom-linux 

If everything worked, you'll see something like:

Spawning container custom-linux on /home/you/kernel-build/rootfs. Press Ctrl-] three times within 1s to kill container. [ OK ] Started Journal Service. [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target Network. [ OK ] Started Boot Timer Service. [ OK ] Reached target Multi-User System. Debian GNU/Linux trixie/sid custom-linux tty1 custom-linux login: 

That's systemd running on your custom-compiled kernel. Log in with root / customlinux.

Verify Everything Works

Once logged in, run through these checks:

# Confirm it's our custom kernel uname -a # Should show 6.12.0-custom (or similar) # Confirm systemd is PID 1 ps -p 1 -o comm= # Should print: systemd # Check that our custom service ran cat /var/log/boot-timer.log # Should show: System booted at <timestamp> # Check cgroups are working systemd-cgls # Should show a tree of cgroups with your services # Check systemd unit status systemctl status boot-timer.service # Should show: active (exited) # Verify networking ip addr show # Should show lo (loopback) at minimum ping -c 1 127.0.0.1 

Test SSH

# Start the SSH daemon (inside the container) systemctl start sshd # Check it's running systemctl status sshd # From another terminal on your host, SSH in: ssh root@$(sudo systemd-nspawn --machine=custom-linux --pipe /bin/hostname -I | awk '{print $1}') 

You just SSH'd into a Linux system running a kernel you compiled yourself, managed by systemd, with a custom service you wrote. That's not a toy — that's the same architecture that runs production servers worldwide, just distilled down to its essentials.

Step 6: The Before/After Comparison

Remember the comparison table from Part 1? Here's the updated version:

The kernel itself is dramatically smaller. The total filesystem is larger because Debian's userspace is bigger than BusyBox — but you get a real package manager, real services, and real systemd. That's the tradeoff.

Things That Will Go Wrong (And How to Fix Them)

Based on my own experience (and the mistakes I made building this the first time):

"Kernel panic: not syncing: VFS: unable to mount root fs"
You forgot to enable CONFIG_EXT4_FS or your rootfs is corrupted. Check your filesystem config and make sure the rootfs directory is valid.

"Failed to connect to bus: No such file or directory"
systemd can't find its socket. Make sure /run/systemd exists and D-Bus is installed. Inside the chroot: apt-get install dbus.

"systemd[1]: Failed to mount cgroup"
Missing cgroup kernel features. Go back to Step 2 and make sure all the CONFIG_CGROUP_* options are enabled. Also check that cgroup2 filesystem is supported.

SSH connection refused
The SSH daemon isn't running or isn't installed. systemctl status sshd will tell you. Make sure you installed openssh-server in the chroot and that root login is permitted in /etc/ssh/sshd_config.

Kernel compiles but is 80MB+
You didn't start from tinyconfig or you enabled too many drivers. Run make tinyconfig and re-enable only the options in Step 2. Use make menuconfig to check what's enabled under "Device Drivers" — that's usually where the bloat hides.

Why This Matters at 3 AM

You're on call. A production server kernel panics. The error mentions cgroups and PID 1. In the old version of you, that's a mystery — you reboot and hope. In the new version of you, you know:

• PID 1 is the init system — if it crashed, the whole system goes down. That's by design.
• cgroups are how the kernel groups processes — if a cgroup is misconfigured, services can't start.
• The kernel panics because something in userspace told it to do something impossible — the stack trace tells you exactly what.
• You can compile a debug kernel, boot it, and reproduce the issue because you've done it before.

That's the difference between the sysadmin who reboots and the sysadmin who debugs. And it starts with understanding what's actually under the hood.

Clean Up

When you're done experimenting:

# Stop the container sudo machinectl terminate custom-linux # Remove the build artifacts (optional — they take ~10GB) rm -rf ~/kernel-build/linux-6.12 

Keep the rootfs directory if you want to boot it again later without recompiling.

Related reads:

• Building Your Own Linux from Scratch (Part 1)
• Your OS Has Been Hiding Things From You (Windows & Linux Edition)
• The Art of Reading Logs Like a Detective: Finding Needles in Haystacks
• Stop Doing Things Manually: 5 Scripts