hyperreal.coffee

USB 3.1 Type-C to RJ45 Gigabit Ethernet adapter

The Amazon Basics Aluminum USB 3.1 Type-C to RJ45 Gigabit Ethernet Adapter works well with FreeBSD 14.1-RELEASE. It uses the AX88179 chipset from ASIX Electronics Corp.

Install the ports tree

Source: Chapter 4. Installing Applications: Packages and Ports | FreeBSD Documentation Portal

Ensure the FreeBSD source code is checked out

1sudo git clone -o freebsd -b releng/14.1 https://git.FreeBSD.org/src.git /usr/src

Check out the ports tree

1sudo git clone --depth 1 https://git.FreeBSD.org/ports.git -b 2024Q3 /usr/ports

To switch to a different quarterly branch:

1sudo git -C /usr/ports switch 2024Q4

drm-61-kmod

Install from the ports tree.

1cd /usr/ports/graphics/drm-61-kmod
2sudo make install clean

Alternatively, for Alderlake GPUs:

1sudo pkg install drm-kmod

Edit /etc/rc.conf

1kld_list="i915kms"

Add user to video group:

1sudo pw groupmod video -m jas

Mount filesystems in single-user mode

When booted into single-user mode.

1fsck
2mount -u /
3mount -a -t zfs
4zfs mount -a

You should now be able to edit files, add/remove packages, etc.

Mount encrypted zroot in LiveCD

Boot into the LiveCD environment.

1mkdir /tmp/mnt
2geli attach /dev/nda0p4
3zpool import -f -R /tmp/mnt zroot
4zfs mount zroot/ROOT/default

The root directory of the zroot, zroot/ROOT/default, is labeled to not be automounted when imported, hence the need for the last command.

== Setup Podman (FreeBSD >= 14) ==

The following is a condensed version of the guide found at CloudSpinx: Install Podman and run Containers in FreeBSD 14.

1sudo pkg install podman-suite
2sudo mount -t fdescfs fdesc /dev/fd

Add the following line to /etc/fstab:

1fdesc /dev/fd fdescfs rw 0 0

Enable the Podman service.

1sudo sysrc podman_enable="YES"

Container networking requires a NAT to allow the container network’s packets to reach the host’s network. Copy the sample pf.conf for Podman.

1sudo cp -v /usr/local/etc/containers/pf.conf.sample /etc/pf.conf

Change v4egress_if and v6egress_if to the host’s main network interface in /etc/pf.conf.

1v4egress_if="igc0"
2v6egree_if="igc0"

Enable and start PF.

1sudo sysrc pf_enable="YES"
2sudo service pf start

FreeBSD >= 13.3 has support for rerouting connections from the host to services inside the container. To enable this, load the PF kernel module, then use sysctl to activate PF support for this rerouting.

1echo 'pf_load="YES"' | sudo tee -a /boot/loader.conf
2sudo kldload pf
3sudo sysctl net.pf.filter_local=1
4echo 'net.pf.filter_local=1' | sudo tee -a /etc/sysctl.conf.local
5sudo service pf restart

The rerouting rules will only work if the destination address is localhost. Ensure the following exists in /etc/pf.conf.

1nat-anchor "cni-rdr/*"

Container images and related state is stored in /var/db/containers. Create a ZFS dataset for this with the mountpoint set to that directory.

1sudo zfs create -o mountpoint=/var/db/containers zroot/containers

If the system is not using ZFS, change storage.conf to use the vfs storage driver.

1sudo sed -I .bak -e 's/driver = "zfs"/driver = "vfs"/' /usr/local/etc/containers/storage.conf

If there are any errors caused by the /var/db/containers/storage database, remote it.

1sudo rm -rfv /var/db/containers/storage

Note: Podman can only be run with root privileges on FreeBSD at this time.

Enable the Linux service.

1sudo sysrc linux_enable="YES"
2sudo service linux start

To run Linux containers, add the --os=linux argument to Podman commands.

1sudo podman run --os=linux ubuntu /usr/bin/cat "/etc/os-release"

Everything else should work as expected.

Install Linux VM in Bhyve

Based on How to install Linux VM on FreeBSD using bhyve and ZFS, but condensed and collated for my use-case.

Setting up the network interfaces

Make the tap device UP by default in /etc/sysctl.conf.

1echo "net.link.tap.up_on_open=1" >> /etc/sysctl.conf
2sysctl net.link.tap.up_on_open=1

Load the kernel modules needed for bhyve.

1kldload vmm
2kldload nmdm

Make sure the modules are loaded at boot time.

1echo 'vmm_load="YES"' >> /boot/loader.conf
2echo 'nmdm_load="YES"' >> /boot/loader.conf
3echo 'if_tap_load="YES"' >> /boot/loader.conf
4echo 'if_bridge_load="YES"' >> /boot/loader.conf

Create the bridge and tap device. If you already have a bridge created, use that instead. We’ll assume this is the case, and the bridge is called igb0bridge.

1ifconfig bridge create

If a bridge is already created and the main network interface igc0 is attached to it, the following command is not necessary.

1ifconfig igb0bridge addm igc0

Create tap interface and attach it to the igb0bridge. <syntaxhighlight lang=“bash” ifconfig tap0 create ifconfig igb0bridge addm tap0

If there wasn’t a bridge already being used for jails, then /etc/rc.conf should contain the following:

1cloned_interfaces="igb0bridge tap0"
2ifconfig_igb0bridge="addm igc0 addm tap0 up"

If there was already a bridge used for jails, then /etc/rc.conf should contain the following:

1cloned_interfaces="igb0bridge tap0"
2ifconfig_igb0bridge="inet 10.0.0.8/24 addm igc0 addm tap0 up"

Setting up the ZFS volumes for Linux bhyve VM

1zfs create -V128G -o volmode=dev zroot/debianvm

Downloading Debian installer ISO

1cd /tmp/
2DEBIAN_VERSION=12.10.0
3wget "https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-${DEBIAN_VERSION}-amd64-netinst.iso"

Installing Debian in VM

Install the grub-bhyve binary to allow booting of non-FreeBSD guest OSes.

1pkg install grub2-bhyve bhyve-firmware

Install Debian by running bhyve with the netinstall iso image and the zvol attached.

 1bhyve -c 4 -m 8G -w -H \
 2  -s 0,hostbridge \
 3  -s 3,ahci-cd,/tmp/debian-12.10.0-amd64-netinst.iso \
 4  -s 4,virtio-blk,/dev/zvol/zroot/debianvm \
 5  -s 5,virtio-net,tap0 \
 6  -s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait \
 7  -s 30,xhci,tablet \
 8  -s 31,lpc \
 9  -l com1,stdio \
10  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
11  debianvm

When the command runs, use a remote VNC view to connect to and start the netinstall iso.

The following step is required to boot from UEFI.

Run the Debian installer with desired configuration. When you reach the “Finish the installation” stage, select “Go Back”, then select “Execute a shell”. Once in the shell, run the following commands:

1mkdir /target/boot/efi/EFI/BOOT/
2cp -v /target/boot/efi/EFI/debian/grubx64.efi /target/boot/efi/EFI/BOOT/bootx64.efi
3exit

Now continue with “Finish the installation”.

Booting Debian bhyve VM

The instance of the virtual machine needs to be destroyed before it can be started again.

1bhyvectl --destroy --vm=debianvm

Boot the Debian VM.

 1bhyve -c 4 -m 8G -w -H \
 2  -s 0,hostbridge \
 3  -s 4,virtio-blk,/dev/zvol/zroot/debianvm \
 4  -s 5,virtio-net,tap0 \
 5  -s 29,fbuf,tcp=0.0.0.0:5900,w=1024,h=768 \
 6  -s 30,xhci,tablet \
 7  -s 31,lpc \
 8  -l com1,stdio \
 9  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
10  debianvm

Starting the Debian VM on boot with a shell script

 1#!/bin/sh
 2# Name: startdebianvm
 3# Purpose: Simple script to start my Debian 10 VM using bhyve on FreeBSD
 4# Author: Vivek Gite {https://www.cyberciti.biz} under GPL v2.x+
 5-------------------------------------------------------------------------
 6# Lazy failsafe (not needed but I will leave them here)
 7ifconfig tap0 create
 8ifconfig em0bridge addm tap0
 9if ! kldstat | grep -w vmm.ko 
10then
11 kldload -v vmm
12fi
13if ! kldstat | grep -w nmdm.ko
14then
15 kldload -v nmdm
16fi
17bhyve -c 1 -m 1G -w -H \
18-s 0,hostbridge \
19-s 4,virtio-blk,/dev/zvol/zroot/debianvm \
20-s 5,virtio-net,tap0 \
21-s 29,fbuf,tcp=0.0.0.0:5900,w=1024,h=768 \
22-s 30,xhci,tablet \
23-s 31,lpc -l com1,stdio \
24-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
25debianvm

Create a crontab entry:

1@reboot /path/to/startdebianvm

Installing a Linux jail

Create the ZFS datasets for the base jail and Linux jail.

1sudo zfs create naspool/jails/debian
2sudo zfs create naspool/jails/14.2-RELEASE

Download the base userland system for FreeBSD.

1fetch https://download.freebsd.org/ftp/releases/amd64/amd64/14.2-RELEASE/base.txz

Extract the base userland into the base jail’s directory.

1sudo tar -xf base.txz -C /jails/14.2-RELEASE --unlink

Copy DNS and timezone files.

1sudo cp -v /etc/resolv.conf /jails/14.2-RELEASE/etc/resolv.conf
2sudo cp -v /etc/localtime /jails/14.2-RELEASE/etc/localtime

Update the base jail to the latest patch level.

1sudo freebsd-update -b /jails/14.2-RELEASE/ fetch install

Create a ZFS snapshot from the base jail.

1sudo zfs snapshot naspool/jails/14.2-RELEASE@base

Clone the base jail to create a thin jail for the Linux distribution.

1sudo zfs clone naspool/jails/14.2-RELEASE@base naspool/jails/debian

Enable the Linux ABI.

1sudo sysrc linux_enable="YES"
2sudo service linux start

Run the jail command with a quick configuration.

 1sudo jail -cm \
 2  name=debian \
 3  host.hostname="debian" \
 4  path="/jails/debian" \
 5  interface="igc0" \
 6  ip4.addr="10.0.0.21" \
 7  exec.start="/bin/sh /etc/rc" \
 8  exec.stop="/bin/sh /etc/rc.shutdown" \
 9  mount.devfs \
10  devfs_ruleset=11 \
11  allow.mount \
12  allow.mount.devfs \
13  allow.mount.fdescfs \
14  allow.mount.procfs \
15  allow.mount.linprocfs \
16  allow.mount.linsysfs \
17  allow.mount.tmpfs \
18  enforce_statfs=1

Access the jail.

1sudo jexec -u root debian

Install the debootstrap program and prepare the Debian environment.

1pkg install debootstrap
2debootstrap bookworm /compat/debian

When the process finishes, stop the jail from the host system.

1sudo service jail onestop debian

Add an entry in /etc/jail.conf for the Debian jail.

 1debian {
 2  # STARTUP/LOGGING
 3  exec.start = "/bin/sh /etc/rc";
 4  exec.stop = "/bin/sh /etc/rc.shutdown";
 5  exec.consolelog = "/var/log/jail_console_${name}.log";
 6
 7  # PERMISSIONS
 8  allow.raw_sockets;
 9  exec.clean;
10  mount.devfs;
11  devfs_ruleset = 11;
12
13  # HOSTNAME/PATH
14  host.hostname = "${name}";
15  path = "/jails/${name}";
16
17  # NETWORK
18  ip4.addr = 10.0.0.21;
19  interface = igc0;
20
21  # MOUNT
22  mount += "devfs       $path/compat/debian/dev       devfs     rw 0 0";
23  mount += "tmpfs       $path/compat/debian/dev/shm   tmpfs     rw,size=1g,mode=1777 0 0";
24  mount += "fdescfs     $path/compat/debian/dev/fd    fdescfs   rw,linrdlnk 0 0";
25  mount += "linprocfs   $path/compat/debian/proc      linprocfs rw 0 0";
26  mount += "linsysfs    $path/compat/debian/sys       linsysfs  rw 0 0";
27  mount += "/tmp        $path/compat/debian/tmp       nullfs    rw 0 0";
28  mount += "/home       $path/compat/debian/home      nullfs    rw 0 0";
29}

Start the jail.

1sudo service jail start debian

The Debian environment can be accessed using the following command:

1sudo jexec debian chroot /compat/debian /bin/bash

Reply to this post by email ↪