feat: jumbo frames on the guest link

docs/GUEST-SETUP.md

@@ -75,6 +75,12 @@ Enable-NetFirewallRule -DisplayGroup "Remote Desktop"
 Put your public key in `C:\ProgramData\ssh\administrators_authorized_keys` for an Administrator
 account. `vm-native-verify` uses that key.
 
+SSH into the guest fails with `Corrupted MAC on input` until the host has the NIC offload rule
+from the `vfio-native` package. The emulated NIC's TX offloads corrupt integrity-checked traffic on
+the host side of the tap; SMB tolerates it, SSH does not. This holds for both `igb` and `e1000e`.
+The package installs a udev rule that turns the offloads off on every libvirt tap as it appears,
+and `vm-native-setup` says so if it is missing.
+
 ## 3. Make the NVMe driver boot-critical
 
 The disk is about to move from virtio to emulated NVMe, and Windows only loads boot-start drivers
@@ -121,14 +127,35 @@ Two things it asks or warns about:
 
 Before the first boot, if the host has less free memory than the guest's RAM, free and compact
 it so the guest lands on transparent hugepages; `vm-native-setup` prints the two commands when it
-applies. The NIC stays `igb`, so the network survives the driver removal in the next step. Do not use
+applies. The NIC stays `igb`, so the network survives the driver removal in the next step: Windows
-virtiofs for host files: it is a virtio device the scanner names, and its shared memory backing
+has an in-box driver for the Intel 82576 it emulates. Do not use virtiofs for host files: it is a
-blocks transparent hugepages for the whole guest. Share over SMB on the `igb` link instead.
+virtio device the scanner names, and its shared memory backing blocks transparent hugepages for the
+whole guest. Share over SMB on the `igb` link instead.
 
-The interface asks for MTU 9000, and the package's hook turns GSO and GRO on for the tap. Together
+### Jumbo frames
-they take the inbound link from 2.9 to 14 Gbit/s; neither does anything alone. The libvirt network
+
-needs `<mtu size='9000'/>` too, and the guest needs *Jumbo Packet* 9014 with its interface MTU at
+`vm-native-setup` puts `<mtu size='9000'/>` on the guest interface. The emulated NIC is packet-rate
-9000 - setting the adapter property alone leaves the IP MTU at 1500 and gains nothing.
+bound rather than bandwidth bound, so larger frames cut the per-packet cost the guest pays on
+receive - the direction that matters when you read files off an SMB share. Measured on igb, host to
+guest: 2922 Mbit/s at 1500, 14232 Mbit/s at 9000, byte-for-byte identical either way.
+
+Both ends have to agree or the frames are simply dropped. The libvirt network needs it too:
+
+```
+virsh net-edit default        # add <mtu size='9000'/>
+virsh net-destroy default && virsh net-start default
+```
+
+and in the guest, set the adapter's *Jumbo Packet* to 9014 and the interface MTU to 9000:
+
+```powershell
+$n = (Get-NetAdapter -Physical | Where-Object Status -eq 'Up').Name
+Set-NetAdapterAdvancedProperty -Name $n -RegistryKeyword "*JumboPacket" -RegistryValue 9014
+Set-NetIPInterface -InterfaceAlias $n -NlMtu 9000
+```
+
+Check `Get-NetIPInterface` afterwards: setting *Jumbo Packet* alone leaves the IP MTU at 1500 and
+you get none of the benefit.
 
 ## 5. Remove the virtio drivers and the agents
 

packaging/vfio-native/PKGBUILD

@@ -48,6 +48,7 @@ package() {
     # this coexists with whatever hook the host already has.
     install -Dm755 scripts/libvirt-hook-cpu-isolation.sh \
         "${pkgdir}/etc/libvirt/hooks/qemu.d/10-cpu-isolation.sh"
-    install -Dm755 scripts/libvirt-hook-vnet-offload.sh \
+    # emulated NIC offloads corrupt integrity-checked traffic on libvirt taps; host-wide by nature
-        "${pkgdir}/etc/libvirt/hooks/qemu.d/20-vnet-offload.sh"
+    install -Dm644 scripts/99-vfio-native-vnet-offload.rules \
+        "${pkgdir}/usr/lib/udev/rules.d/99-vfio-native-vnet-offload.rules"
 }

packaging/vfio-native/vfio-native.install

@@ -49,11 +49,11 @@ post_install() {
 
   'native' is a good default if you would rather not maintain anything.
 
-  A hook at /etc/libvirt/hooks/qemu.d/20-vnet-offload.sh turns GSO and GRO on
+  A udev rule (99-vfio-native-vnet-offload.rules) turns TX offloads off on every
-  for the guest's tap. With the interface's MTU 9000 that takes the inbound link
+  libvirt tap as it appears. The emulated e1000e NIC corrupts integrity-checked
-  from 2.9 to 14 Gbit/s; neither does anything alone. The libvirt network needs
+  traffic with them on; SSH to the guest fails with "Corrupted MAC on input".
-  <mtu size='9000'/> too, and the guest needs Jumbo Packet 9014 with its
+  It applies to every VM on the host; the throughput cost on a host<->guest
-  interface MTU at 9000.
+  link is not measurable.
 
   A libvirt hook is installed at /etc/libvirt/hooks/qemu.d/10-cpu-isolation.sh.
   It keeps host processes off the cores the guest is pinned to, automatically,

scripts/99-vfio-native-vnet-offload.rules

@@ -0,0 +1,8 @@
+# vfio-native: the emulated e1000e NIC's TX checksum and segmentation offloads
+# corrupt packets on the host side of a libvirt tap. SMB tolerates it; SSH fails
+# with "Corrupted MAC on input" and any integrity-checked protocol breaks.
+# Measured on a Zen 4 host with QEMU 11.1.1. Disabling the offloads on every
+# libvirt tap as it appears fixes it; on a host<->guest link the throughput
+# cost is not measurable. libvirt's <driver><host .../> attributes are ignored
+# for e1000e, and a libvirt hook must not call virsh, hence udev.
+ACTION=="add", SUBSYSTEM=="net", KERNEL=="vnet*", RUN+="/usr/bin/ethtool -K %k tx off gso off gro off tso off"

scripts/libvirt-hook-vnet-offload.sh

@@ -1,15 +0,0 @@
-#!/bin/bash
-# libvirt qemu hook: turn GSO and GRO on for the guest's tap.
-#
-# QEMU leaves them off and sets the tap up after udev has run, so it has to
-# happen here. Paired with MTU 9000 they are the difference between 2.5 and
-# 14 Gbit/s into the guest; neither helps alone. Tap names come from the domain
-# XML on stdin, so the hook never calls virsh, which would deadlock libvirtd.
-
-[ "$2" = started ] || exit 0
-
-for tap in $(grep -oE "<target dev='(vnet|tap|macvtap)[^']*'" | sed "s/.*dev='//; s/'$//"); do
-    ethtool -K "$tap" gso on gro on 2>/dev/null
-done
-
-exit 0