# Why each VMAware check fires, and what it took to clear it

The engineering record behind the fidelity work. `README.md` says *what to apply and when*; this
says *why*, with the source references. Written 2026-09-03 at 2/85, revised 2026-09-05 at **1/85**
on both VMAware v2.8.1 and VMAware HEAD (commit 95fecc2) - the one survivor is `GPU_CAPABILITIES`,
and VMAware's own verdict is "Running on bare metal", likeliness 20%, confirmation false.

Source read against: VMAware `src/vmaware.hpp`, QEMU 11.1.1, kernel 7.2.3 `arch/x86/kvm`.

---

## CLEARED - CPUID_SIGNATURE

**Not about hypervisor leaves.** The guest's `0x40000000` reads all zeros, and so does the
bare-metal host, so it was never the trigger.

The AMD branch (`cpuid_signature()`, vmaware.hpp:6941-6970) reads **leaf 7 subleaf 0, EDX** and
returns true if bit 26 (`IBRS`/`IBPB`), 27 (`STIBP`) or 31 (`SSBD`) is set. AMD reserves those
three to zero and enumerates its mitigations in `0x80000008.EBX` instead.

Measured with an unprivileged CPUID probe:

| | leaf 7.0 EDX | bits 26/27/31 | `0x80000008.EBX` 12/14/15/24/25 |
| --- | --- | --- | --- |
| host, bare metal | `0x10000010` | `0 0 0` | `1 1 1 1 1` |
| guest, before | `0x9c000010` | set | passed through |
| guest, after | `0x30000010` | `0 0 0` | `1 1 1 1 1` |

KVM synthesises the Intel-style `SPEC_CTRL` interface on AMD, and `host-passthrough` enables it.
Three `<feature policy='disable'>` lines fix it. All three are needed - the check ORs the bits.

Verified in-guest afterwards that every AMD-native mitigation bit survives, so the guest keeps
the interface Windows actually uses on AMD. Residual: KVM also synthesises bit 29
(`arch-capabilities`), which bare metal lacks. Nothing reads it, so it is left alone.

---

## CLEARED - SVM_EXCEPTIONS

**A bug KVM documents against itself.**

`svm.c:272` is unconditional:

```c
svm->vmcb->save.efer = efer | EFER_SVME;
```

Hardware therefore never sees the guest's real `SVME=0`. A CPL3 `VMLOAD` passes the SVME check,
hardware notices CPL is not zero, and injects `#GP` *before* the VMLOAD intercept can fire. Bare
metal raises `#UD`, and `#UD` is the one exception `svm_exceptions()` treats as innocent
(vmaware.hpp:14551).

There is a `FIXME` at `svm.c:1076-1080` describing exactly this.

The diagnostic that pinned it: the scored run printed `[  DETECTED  ]` with **no debug line**.
Both noisy exits in the function call `vma_debug`, so it must have fallen through to the bare
`return true` at vmaware.hpp:14570 - which forces `svm_visible == true` and a non-`#UD` fault.

Fix, in `svm_recalc_instruction_intercepts()`:

```c
	if (!(vcpu->arch.efer & EFER_SVME)) {
		svm_set_intercept(svm, INTERCEPT_VMLOAD);
		svm_set_intercept(svm, INTERCEPT_VMSAVE);
		svm_set_intercept(svm, INTERCEPT_CLGI);
		svm_set_intercept(svm, INTERCEPT_STGI);
		set_exception_intercept(svm, GP_VECTOR);   /* added */
	} else {
```

`gp_interception()` then decodes the SVM opcode, sees `!is_guest_mode`, and calls
`svm_invoke_exit_handler(SVM_EXIT_VMLOAD)` -> `vmload_interception` ->
`nested_svm_check_permissions` (nested.c:1664), which queues `UD_VECTOR`.

The shipped `0001` also keeps the intercept across a guest clearing `SVME` later (the stock
`svm_set_efer()` dropped it there), drops it again on `SVME=1` where the erratum workaround does
not need it, and never arms it for SEV guests, whose instructions KVM cannot decode - the same
exclusion the erratum path already makes.

**Do NOT disable `svm` in guest CPUID as an alternative.** It does not change the exception, and
it moves the check into its `!svm_visible` branch, which is *worse* - "SVM hypervisor hiding CPU
capabilities", weight 150.

**Do NOT use `kvm.enable_vmware_backdoor=1`** as the no-rebuild shortcut. It arms the same
intercept, but permanently opens two well-known VMware detection surfaces in every guest on
the host: `emulate.c:2562` stops faulting CPL3 `IN`/`OUT` on ports `0x5658`/`0x5659`, and
`emulate.c:3905` makes CPL3 `RDPMC` of pseudo-counters `0x10000`-`0x10002` return host TSC,
which is architecturally impossible on real silicon. VMAware is blind to both. pafish,
al-khaser and other detection tooling are not.

---

## CLEARED - FIRMWARE

**A chain that reports only its first hit.** `firmware()` (vmaware.hpp:8860) runs six sections
in fixed order against each firmware buffer, DSDT first, and the first match returns. Fixing one
link costs a full QEMU rebuild and boot and only reveals the next - so bundle.

Links, in order:

1. ~~SMI Resources reservation string~~ - `_UID` renamed
2. ~~PRTP/PRTA routing symmetry~~ - renamed `IRQP`/`IRQA`
3. ~~Sequential PIRQ names, vmaware.hpp:9142~~ - `GSIA-H` -> `APCA-H`
4. ~~PNP0A06 resource stubs, vmaware.hpp:9148~~ - `GPER` -> `RSRA`, `PHPR` -> `RSRB`
5. ~~FACP C2/C3 latencies, vmaware.hpp:9242~~ - `0xfff` -> `0xffe`
6. Debug Port OperationRegion at `0x0402` - already dead on this tree
7. HPET register-validation loop - dormant, domain sets `hpet present='no'`. Re-arms if enabled.
8. Dummy SATA, DMAR, APIC source overrides - already dead on this tree

Link 3 requires **all four** of `LNKE`, `LNKH`, `GSIE`, `GSIH`, which is QEMU's eight-link plus
eight-GSI layout. Real boards have four PCI link devices and no GSI-named ones, so rename the
`GSI*` half and leave `LNK*` alone. Those names are declared by `build_gsi_link_dev()` and
referenced only through `build_q35_routing_table("GSI")`, which derives every name from one
3-character prefix - so a prefix change moves every reference with it.

Do not pick `IRQ` as the replacement: `_SB.PCI0.IRQA` is already the APIC routing package and a
bare NameSeg would bind to it.

`0xffe` keeps "C-state not supported" semantics (anything above 100 / 1000 means unsupported)
while not matching the check's exact-equality test against `0x0FFF`.

**WAET must stay out.** `"WAET"` is target index 18 in the section-2 scan and the table's
signature sits at offset 0, i.e. an instant hit. An earlier sed-based patcher claimed it restored
WAET and never did: `build_waet()` has zero call sites, so its `grep -q build_waet` guard was
satisfied by the definition alone and the insert never ran. Guard on a call site, not on a
definition.

---

## OPEN - GPU_CAPABILITIES (weight 20)

**Not a GPU check.** Four lines: `GetDC(nullptr)`, then `GetDeviceCaps(hdc, COLORMGMTCAPS)`,
DETECTED if the result lacks `CM_GAMMA_RAMP` (0x2) or is `CM_NONE`. No adapter enumeration, no
DXGI, no WMI, no device blacklist, no EDID - so the patched `SAM`/`SyncMaster` EDID override is
irrelevant to this check.

A plausible theory said it was a harness artefact: scoring runs over SSH, OpenSSH on Windows
lands in session 0, and a non-interactive window station has no gamma LUT. **Tested and
disproven.** Re-ran as a scheduled task with `/it`, confirmed `SessionId=1` and
`ScreenBounds={0,0,640,480}` - same score, still fires.

So it is genuine: `<video><model type='none'/>` leaves Windows on a stub display with no gamma
ramp. Real GPU passthrough fixes it. Out of scope without a spare card to pass through;
`vm-native-gpu` is the path.

An emulated adapter is not a workaround: `qxl` and `virtio-gpu` are on the DEVICES blacklist
(weight 100), and any display device can arm `BOOT_LOGO` (weight 90, brands QEMU) by giving
OVMF a GOP.

---

## CLEARED - KVM_INTERCEPTION

**The debug string is a red herring**, and it cost this project two wrong turns. It says "KVM
attempting to patch instructions on the fly", which points straight at
`KVM_X86_QUIRK_FIX_HYPERCALL_INSN`. That is not the cause.

VMAware runs its stubs at CPL3. KVM's emulator declares VMCALL in `group7_rm0[1]`
(`emulate.c:3971`) as:

```c
I(SrcNone | Priv | EmulateOnUD, em_hypercall),
```

`Priv` but **no** `PrivUD`. So `emulate.c:5348`:

```c
	if ((ctxt->d & Priv) && ops->cpl(ctxt)) {
		if (ctxt->d & PrivUD)
			rc = emulate_ud(ctxt);
		else
			rc = emulate_gp(ctxt, 0);
		goto done;
	}
```

takes the `#GP(0)` branch, which Windows surfaces as STATUS_ACCESS_VIOLATION. The quirk code in
`emulator_fix_hypercall()` sits downstream of that CPL check and is never reached. Disabling the
quirk is a measured no-op - confirmed on three purpose-built `/dev/kvm` harnesses showing
quirk-on and quirk-off producing byte-identical `#GP` with unmodified code bytes.

The earlier "cleared INTERCEPT_VMMCALL, effect: none" log entry was therefore correct, and my
own mid-session reinterpretation of it as "closed for a bad reason" was wrong.

Both stubs need to raise `#UD`, and **either half alone leaves the detection standing** - fix
stub 0 only and the loop reaches stub 1, whose silent `-KVM_EPERM` trips the generic-hypervisor
branch instead. That is why every previous single-sided attempt measured as "no effect".

**Half A.** `PrivUD` (`emulate.c:171`, `((u64)1 << 51)`, "#UD instead of #GP on CPL > 0") added
to the VMCALL entry in `group7_rm0`. The flag had zero users in the tree before this. `Priv`
already routes to the CPL check; `PrivUD` picks the other branch. RIP does not advance.

**Half B.** `kvm_emulate_hypercall()` injects `#UD` for a CPL>0 hypercall instead of the silent
`-KVM_EPERM`. Placed **after** the Xen and Hyper-V dispatch, so enlightened guests never reach
it - and `kvm_hv_hypercall` already does the identical `cpl != 0 -> kvm_queue_exception(UD_VECTOR)`
at `hyperv.c:2546`, so this makes the KVM-PV path consistent with the Hyper-V one rather than
inventing new behaviour.

The blunt alternative - clearing `INTERCEPT_VMMCALL` - was correctly rejected: it breaks CPL0
hypercalls and bugchecks the Hyper-V domains.

Blast radius checked on the live host: `svm_patch_hypercall` writes VMMCALL and issues it at
CPL0; Linux paravirt alternative-patches to VMMCALL, also CPL0; every `KVM_HC_*` already failed
at CPL>0, so only the *shape* of the failure changes. Confirmed empirically by booting a
Hyper-V-enlightened Windows guest and a Linux paravirt guest on the patched module - both boot
and execute normally.

Half A ships as `patches/kvm/0002-KVM-x86-emulator-UD-not-GP-for-VMCALL-at-CPL-0.patch`, half B as
`patches/kvm/0003-KVM-x86-UD-for-hypercalls-issued-at-CPL-0.patch`. Both land in `kvm.ko`, not
`kvm-amd.ko`, so verify `/sys/module/kvm/srcversion` - checking only `kvm_amd` reports success on
a stale build.

---

## CLEARED - TIMER

**There is no clock to lie to.** The check does not use the TSC. It times a bare `CPUID` (EAX=0)
against eight `_mm_lfence()`, using a second thread on another core spinning `counter++` on a
64-byte-aligned volatile; the "tick" is a cross-core cache-line bounce (the code says so at
vmaware.hpp:7480, "this is a cache-based counter").

`serialize_available = cpu::is_intel()` is hard-false on AMD, so the LFENCE branch always runs.
Threads are pinned *inside* the guest with `SetThreadGroupAffinity`. The estimator takes a
500-1000 sample batch per trial, an interquartile mean of the middle 50%, then the **minimum
across 5 trials**, with samples bracketed by counter-edge spin-waits and followed by
`burn_random_cycles()` so a hypervisor cannot predict when to freeze the counter thread.

**There are TWO detectors, and they OR together** (vmaware.hpp:7484, 7488, 7493). Clearing one
achieves nothing.

### Detector 2 was never the wall - the probe was wrong

The first version of `bench/timerprobe.c` reported the exception-latency detector at **6.169**
against a threshold of 2.5, and two days of work treated that as a hypervisor floor. It was a
measurement error. VMAware's software side is `RtlCaptureContext` + `ZwRaiseException` inside an
SEH frame with `EXCEPTION_EXECUTE_HANDLER`, which is a syscall in, a kernel dispatch, and an
`RtlUnwindEx` out. The probe used `RaiseException` under a vectored handler, which never enters the
kernel on the software side, so its reference window was several times too short and the ratio
several times too high.

Rebuilt mechanism for mechanism (`__C_specific_handler` scope table, `ZwRaiseException`,
`EXCEPTION_EXECUTE_HANDLER`), the probe reads **1.47**, and VMAware's own debug line agrees on
the same boot with the stock `#DB` intercept in place:

    TIMER: Exception > VMM -> 10167 | nVMM -> 7066 | Ratio -> 1.439

Detector 2 passes on stock KVM. No bare-metal Windows baseline was taken - the brief asked for
one, no physical Windows machine was available - so the only claim made here is the measured one:
the guest is under the threshold with `#DB` intercepted, by a margin of 1.1.

### Detector 1 is a world switch, and the only way to not pay it is to not exit

Measured: `cpuid 2054-2383 / lfence 262-310 ticks`, ratio 7.6-8.2, threshold 2.5. The exit itself
is ~1800 ticks, on the order of 400 ns, an ordinary Zen 4 world switch. Cross-CCD counter
placement, the exit fastpath, TSC offsetting and CPUID-leaf overrides were all measured and all
failed (table below).

`EXPERIMENTAL-0006-runtime-cpuid-passthrough` clears `INTERCEPT_CPUID` on every guest entry once
`cpuid_passthrough=1`, so `CPUID` runs on the silicon. Three things follow from raw CPUID, and the
patch handles each:

- **The brand string reverts to the host SKU.** The AMD Processor Name String MSRs
  (`0xC0010030-35`) back `CPUID 0x80000002-4` directly and are writable and per-thread on Zen 4,
  so the declared SKU is written on the pinned core at entry and restored in `svm_vcpu_put()`.
  `THREAD_MISMATCH` stays clear; VMAware's own thread database resolves `7700X` to 16 threads.
- **Raw CPUID advertises RDPRU** (`0x80000008 EBX[4]`), which KVM masks out of its own CPUID and
  intercepts with `kvm_handle_invalid_op()`, i.e. `#UD`. VMAware's `INTERRUPT_SHADOW` and
  `SINGLE_STEP` both have an RDPRU variant gated on exactly that bit, and both report the `#UD` as
  an "exception anomaly, hypervisor seems to be present with CPUID interception disabled". Two
  full scans fired both checks before this was understood, and an exact-stub reproducer with a
  `kvm_exit` trace showed the TF `#DB` after a native CPUID landing at the right RIP every time -
  the single-step path was never at fault. The RDPRU intercept is cleared together with the CPUID
  one; the guest then reads the pinned core's real `MPERF`/`APERF`.
- **Raw CPUID must only be enabled after boot.** Windows enumerates KVM's synthetic leaf-1 bits
  (`x2apic`, `tsc-deadline`) during boot and hangs if they vanish mid-enumeration. The module
  parameter is runtime-toggled for this reason.

The intercept is also withheld from any vCPU thread whose allowed-CPU mask is wider than one CPU
(`cpumask_weight(current->cpus_ptr) == 1`), so the brand override cannot leak onto a host core: the
patch verifies the 1:1 pinning itself rather than trusting the operator.

Measured with the passthrough on, VMAware HEAD debug, console session:

    TIMER: Instruction > VMM -> 219 | nVMM -> 290 | Ratio -> 0.755
    TIMER: Exception   > VMM -> 10147 | nVMM -> 7551 | Ratio -> 1.344
    VM detections: 1/85   ===== CONCLUSION: Running on bare metal =====

The `#DB` intercept is left in place. Clearing it too was measured (detector 2 drops to 0.96) and
gains nothing the score needs, while it removes the single-step re-injection KVM relies on for
its own NMI-window logic. An earlier note here blamed that clear for the `INTERRUPT_SHADOW` and
`SINGLE_STEP` detections; that was wrong, RDPRU was the cause both times.

Measured and dead before the passthrough:

| Attempt | Result |
| --- | --- |
| BetterTiming TSC compensation | no detection change, 6x slower boot (6.1 s -> 38.6 s CPU) |
| Hypervisor-Phantom CPUID override | ratio halved to ~810 ticks, **still detected**, and added `SINGLE_STEP` (10 -> 11) |
| Clear `INTERCEPT_CPUID` from boot | guest never boots |
| All four intercepts cleared | triple fault, `EFER=0`, instant |
| CPUID exit fastpath in `svm_exit_handlers_fastpath` | the fastpath runs *after* the world switch (`svm.c:4429`, called at `:4664`), so it cannot remove the exit that is being measured |
| Cross-CCD topology to grow the baseline | the measured ceiling falls ~1.7x short, and VMAware picks its own nearby core pair anyway |
| Both of the above combined | still short |

---

## CLEARED - DBVM (weight 150, VMAware HEAD only)

Added to VMAware after v2.8.1; the release build does not fire it, HEAD does, with
`DBVM: ICEBP failed to advance guest RIP`. Named after a hypervisor-backed debugger, but the sub-check
that fires is generic: it clears DR0-DR7 through `NtSetContextThread`, executes `F1 C3`
(`icebp; ret`) under SEH, and compares the exception context's RIP against `stub + 1`.

`ICEBP` (`INT1`) raises a trap-like `#DB`, so the frame RIP on hardware is the next instruction.
The first theory here - that SVM's `#DB` exception intercept catches it and `db_interception()`
re-injects it with RIP still on the `F1` byte - was wrong, and it was wrong in a way that only a
trace could show: on Zen 4 the exception intercept never fires for ICEBP at all, and every warm
reproducer read the correct `+1`. What the `kvm_exit` trace of a cold selftest shows instead:

    kvm_exit: reason npf rip 0x402ec4 ... intr_info 0x80000301     <- ICEBP at 0x402ec4
    kvm_inj_exception: #DB [reinjected]

The `#DB` was being delivered when it hit a nested page fault on an unmapped IDT/handler page.
The exit carries the pending `#DB` in `EXITINTINFO` with the saved RIP on the ICEBP, and
`svm_complete_interrupts()` re-queues it as a plain hardware exception, which the next `VMRUN`
injects at that RIP. So the guest sees the ICEBP's own address whenever the delivery touches a
page KVM has not mapped yet - deterministic in a fresh VM, sporadic in a running Windows guest
(page compaction and reclaim keep unmapping NPT entries), which is why the scanner hit it and the
warm probes did not.

`0004-KVM-SVM-intercept-ICEBP-and-skip-it-before-injecting-its-DB` enables SVM's dedicated
`INTERCEPT_ICEBP`, which fires before the `#DB` exists, skips the instruction with
`svm_skip_emulated_instruction()` and queues the `#DB` with the same DR6 payload
`db_interception()` uses. Once RIP has been advanced, an injection that is itself interrupted is
re-injected with the advanced RIP. This is the shape VMX already has (`is_icebp()` in
`vmx/vmcs.h`, skipped in `handle_exception_nmi()`).

Proof, at CPL0 with no Windows involved - a KVM selftest (`x86/icebp_test.c`, shipped as 5/5 of
the series) executes `icebp` under a `#DB` handler that records the frame RIP:

    unpatched svm.c:   0x402ec4 != 0x402ec5 (db_rip != next_rip)    FAIL
    with 0004:         PASS

and the two full scans with 0004 in place no longer fire `DBVM`.

---

## Method notes worth keeping

**Score from the console session.** OpenSSH on Windows puts you in session 0, the services
session. Every score this project took for its first two days came from there. Use a scheduled
task with `/it` and confirm `(Get-Process -Id $PID).SessionId` is 1 - `schtasks /run` reports
success regardless of where the task actually ran.

**One change per boot.** The 70-second silent power-off from an ACPI namespace failure has no
error message and no log line. The only way to attribute it is to have changed one thing.

**Read the debug line, not the count.** `[  DETECTED  ]` with no debug line is not "cleared" -
some targets return silently (`Xen` at vmaware.hpp:9174, `BXPC` at :9186).

---

## Status

    VMAware v2.8.1 debug:  1/85   GPU_CAPABILITIES         Running on bare metal
    VMAware HEAD debug:    1/85   GPU_CAPABILITIES         Running on bare metal, likeliness 20%

with `0001-0004` in the modules, `EXPERIMENTAL-0006` built in and enabled after boot on a 16 vCPU
guest pinned 1:1 to one CCD, declaring a Ryzen 7 7700X. The remaining check needs a real GPU passed
through, which needs a spare card. `bench/timerprobe.c` reproduces both `TIMER` detectors and
`bench/`-adjacent probes for the others live in `docs/TESTING.md`.