372 lines
19 KiB
Markdown
372 lines
19 KiB
Markdown
# Why each VMAware check fires, and what it took to clear it
|
|
|
|
The engineering record behind the fidelity work. `README.md` says *what to apply and when*; this
|
|
says *why*, with the source references. Written 2026-09-03 at 2/85, revised 2026-09-05 at **1/85**
|
|
on both VMAware v2.8.1 and VMAware HEAD (commit 95fecc2) - the one survivor is `GPU_CAPABILITIES`,
|
|
and VMAware's own verdict is "Running on bare metal", likeliness 20%, confirmation false.
|
|
|
|
Source read against: VMAware `src/vmaware.hpp`, QEMU 11.1.1, kernel 7.2.3 `arch/x86/kvm`.
|
|
|
|
---
|
|
|
|
## CLEARED - CPUID_SIGNATURE
|
|
|
|
**Not about hypervisor leaves.** The guest's `0x40000000` reads all zeros, and so does the
|
|
bare-metal host, so it was never the trigger.
|
|
|
|
The AMD branch (`cpuid_signature()`, vmaware.hpp:6941-6970) reads **leaf 7 subleaf 0, EDX** and
|
|
returns true if bit 26 (`IBRS`/`IBPB`), 27 (`STIBP`) or 31 (`SSBD`) is set. AMD reserves those
|
|
three to zero and enumerates its mitigations in `0x80000008.EBX` instead.
|
|
|
|
Measured with an unprivileged CPUID probe:
|
|
|
|
| | leaf 7.0 EDX | bits 26/27/31 | `0x80000008.EBX` 12/14/15/24/25 |
|
|
| --- | --- | --- | --- |
|
|
| host, bare metal | `0x10000010` | `0 0 0` | `1 1 1 1 1` |
|
|
| guest, before | `0x9c000010` | set | passed through |
|
|
| guest, after | `0x30000010` | `0 0 0` | `1 1 1 1 1` |
|
|
|
|
KVM synthesises the Intel-style `SPEC_CTRL` interface on AMD, and `host-passthrough` enables it.
|
|
Three `<feature policy='disable'>` lines fix it. All three are needed - the check ORs the bits.
|
|
|
|
Verified in-guest afterwards that every AMD-native mitigation bit survives, so the guest keeps
|
|
the interface Windows actually uses on AMD. Residual: KVM also synthesises bit 29
|
|
(`arch-capabilities`), which bare metal lacks. Nothing reads it, so it is left alone.
|
|
|
|
---
|
|
|
|
## CLEARED - SVM_EXCEPTIONS
|
|
|
|
**A bug KVM documents against itself.**
|
|
|
|
`svm.c:272` is unconditional:
|
|
|
|
```c
|
|
svm->vmcb->save.efer = efer | EFER_SVME;
|
|
```
|
|
|
|
Hardware therefore never sees the guest's real `SVME=0`. A CPL3 `VMLOAD` passes the SVME check,
|
|
hardware notices CPL is not zero, and injects `#GP` *before* the VMLOAD intercept can fire. Bare
|
|
metal raises `#UD`, and `#UD` is the one exception `svm_exceptions()` treats as innocent
|
|
(vmaware.hpp:14551).
|
|
|
|
There is a `FIXME` at `svm.c:1076-1080` describing exactly this.
|
|
|
|
The diagnostic that pinned it: the scored run printed `[ DETECTED ]` with **no debug line**.
|
|
Both noisy exits in the function call `vma_debug`, so it must have fallen through to the bare
|
|
`return true` at vmaware.hpp:14570 - which forces `svm_visible == true` and a non-`#UD` fault.
|
|
|
|
Fix, in `svm_recalc_instruction_intercepts()`:
|
|
|
|
```c
|
|
if (!(vcpu->arch.efer & EFER_SVME)) {
|
|
svm_set_intercept(svm, INTERCEPT_VMLOAD);
|
|
svm_set_intercept(svm, INTERCEPT_VMSAVE);
|
|
svm_set_intercept(svm, INTERCEPT_CLGI);
|
|
svm_set_intercept(svm, INTERCEPT_STGI);
|
|
set_exception_intercept(svm, GP_VECTOR); /* added */
|
|
} else {
|
|
```
|
|
|
|
`gp_interception()` then decodes the SVM opcode, sees `!is_guest_mode`, and calls
|
|
`svm_invoke_exit_handler(SVM_EXIT_VMLOAD)` -> `vmload_interception` ->
|
|
`nested_svm_check_permissions` (nested.c:1664), which queues `UD_VECTOR`.
|
|
|
|
The shipped `0001` also keeps the intercept across a guest clearing `SVME` later (the stock
|
|
`svm_set_efer()` dropped it there), drops it again on `SVME=1` where the erratum workaround does
|
|
not need it, and never arms it for SEV guests, whose instructions KVM cannot decode - the same
|
|
exclusion the erratum path already makes.
|
|
|
|
**Do NOT disable `svm` in guest CPUID as an alternative.** It does not change the exception, and
|
|
it moves the check into its `!svm_visible` branch, which is *worse* - "SVM hypervisor hiding CPU
|
|
capabilities", weight 150.
|
|
|
|
**Do NOT use `kvm.enable_vmware_backdoor=1`** as the no-rebuild shortcut. It arms the same
|
|
intercept, but permanently opens two well-known VMware detection surfaces in every guest on
|
|
the host: `emulate.c:2562` stops faulting CPL3 `IN`/`OUT` on ports `0x5658`/`0x5659`, and
|
|
`emulate.c:3905` makes CPL3 `RDPMC` of pseudo-counters `0x10000`-`0x10002` return host TSC,
|
|
which is architecturally impossible on real silicon. VMAware is blind to both. pafish,
|
|
al-khaser and other detection tooling are not.
|
|
|
|
---
|
|
|
|
## CLEARED - FIRMWARE
|
|
|
|
**A chain that reports only its first hit.** `firmware()` (vmaware.hpp:8860) runs six sections
|
|
in fixed order against each firmware buffer, DSDT first, and the first match returns. Fixing one
|
|
link costs a full QEMU rebuild and boot and only reveals the next - so bundle.
|
|
|
|
Links, in order:
|
|
|
|
1. ~~SMI Resources reservation string~~ - `_UID` renamed
|
|
2. ~~PRTP/PRTA routing symmetry~~ - renamed `IRQP`/`IRQA`
|
|
3. ~~Sequential PIRQ names, vmaware.hpp:9142~~ - `GSIA-H` -> `APCA-H`
|
|
4. ~~PNP0A06 resource stubs, vmaware.hpp:9148~~ - `GPER` -> `RSRA`, `PHPR` -> `RSRB`
|
|
5. ~~FACP C2/C3 latencies, vmaware.hpp:9242~~ - `0xfff` -> `0xffe`
|
|
6. Debug Port OperationRegion at `0x0402` - already dead on this tree
|
|
7. HPET register-validation loop - dormant, domain sets `hpet present='no'`. Re-arms if enabled.
|
|
8. Dummy SATA, DMAR, APIC source overrides - already dead on this tree
|
|
|
|
Link 3 requires **all four** of `LNKE`, `LNKH`, `GSIE`, `GSIH`, which is QEMU's eight-link plus
|
|
eight-GSI layout. Real boards have four PCI link devices and no GSI-named ones, so rename the
|
|
`GSI*` half and leave `LNK*` alone. Those names are declared by `build_gsi_link_dev()` and
|
|
referenced only through `build_q35_routing_table("GSI")`, which derives every name from one
|
|
3-character prefix - so a prefix change moves every reference with it.
|
|
|
|
Do not pick `IRQ` as the replacement: `_SB.PCI0.IRQA` is already the APIC routing package and a
|
|
bare NameSeg would bind to it.
|
|
|
|
`0xffe` keeps "C-state not supported" semantics (anything above 100 / 1000 means unsupported)
|
|
while not matching the check's exact-equality test against `0x0FFF`.
|
|
|
|
**WAET must stay out.** `"WAET"` is target index 18 in the section-2 scan and the table's
|
|
signature sits at offset 0, i.e. an instant hit. An earlier sed-based patcher claimed it restored
|
|
WAET and never did: `build_waet()` has zero call sites, so its `grep -q build_waet` guard was
|
|
satisfied by the definition alone and the insert never ran. Guard on a call site, not on a
|
|
definition.
|
|
|
|
---
|
|
|
|
## OPEN - GPU_CAPABILITIES (weight 20)
|
|
|
|
**Not a GPU check.** Four lines: `GetDC(nullptr)`, then `GetDeviceCaps(hdc, COLORMGMTCAPS)`,
|
|
DETECTED if the result lacks `CM_GAMMA_RAMP` (0x2) or is `CM_NONE`. No adapter enumeration, no
|
|
DXGI, no WMI, no device blacklist, no EDID - so the patched `SAM`/`SyncMaster` EDID override is
|
|
irrelevant to this check.
|
|
|
|
A plausible theory said it was a harness artefact: scoring runs over SSH, OpenSSH on Windows
|
|
lands in session 0, and a non-interactive window station has no gamma LUT. **Tested and
|
|
disproven.** Re-ran as a scheduled task with `/it`, confirmed `SessionId=1` and
|
|
`ScreenBounds={0,0,640,480}` - same score, still fires.
|
|
|
|
So it is genuine: `<video><model type='none'/>` leaves Windows on a stub display with no gamma
|
|
ramp. Real GPU passthrough fixes it. Out of scope without a spare card to pass through;
|
|
`vm-native-gpu` is the path.
|
|
|
|
An emulated adapter is not a workaround: `qxl` and `virtio-gpu` are on the DEVICES blacklist
|
|
(weight 100), and any display device can arm `BOOT_LOGO` (weight 90, brands QEMU) by giving
|
|
OVMF a GOP.
|
|
|
|
---
|
|
|
|
## CLEARED - KVM_INTERCEPTION
|
|
|
|
**The debug string is a red herring**, and it cost this project two wrong turns. It says "KVM
|
|
attempting to patch instructions on the fly", which points straight at
|
|
`KVM_X86_QUIRK_FIX_HYPERCALL_INSN`. That is not the cause.
|
|
|
|
VMAware runs its stubs at CPL3. KVM's emulator declares VMCALL in `group7_rm0[1]`
|
|
(`emulate.c:3971`) as:
|
|
|
|
```c
|
|
I(SrcNone | Priv | EmulateOnUD, em_hypercall),
|
|
```
|
|
|
|
`Priv` but **no** `PrivUD`. So `emulate.c:5348`:
|
|
|
|
```c
|
|
if ((ctxt->d & Priv) && ops->cpl(ctxt)) {
|
|
if (ctxt->d & PrivUD)
|
|
rc = emulate_ud(ctxt);
|
|
else
|
|
rc = emulate_gp(ctxt, 0);
|
|
goto done;
|
|
}
|
|
```
|
|
|
|
takes the `#GP(0)` branch, which Windows surfaces as STATUS_ACCESS_VIOLATION. The quirk code in
|
|
`emulator_fix_hypercall()` sits downstream of that CPL check and is never reached. Disabling the
|
|
quirk is a measured no-op - confirmed on three purpose-built `/dev/kvm` harnesses showing
|
|
quirk-on and quirk-off producing byte-identical `#GP` with unmodified code bytes.
|
|
|
|
The earlier "cleared INTERCEPT_VMMCALL, effect: none" log entry was therefore correct, and my
|
|
own mid-session reinterpretation of it as "closed for a bad reason" was wrong.
|
|
|
|
Both stubs need to raise `#UD`, and **either half alone leaves the detection standing** - fix
|
|
stub 0 only and the loop reaches stub 1, whose silent `-KVM_EPERM` trips the generic-hypervisor
|
|
branch instead. That is why every previous single-sided attempt measured as "no effect".
|
|
|
|
**Half A.** `PrivUD` (`emulate.c:171`, `((u64)1 << 51)`, "#UD instead of #GP on CPL > 0") added
|
|
to the VMCALL entry in `group7_rm0`. The flag had zero users in the tree before this. `Priv`
|
|
already routes to the CPL check; `PrivUD` picks the other branch. RIP does not advance.
|
|
|
|
**Half B.** `kvm_emulate_hypercall()` injects `#UD` for a CPL>0 hypercall instead of the silent
|
|
`-KVM_EPERM`. Placed **after** the Xen and Hyper-V dispatch, so enlightened guests never reach
|
|
it - and `kvm_hv_hypercall` already does the identical `cpl != 0 -> kvm_queue_exception(UD_VECTOR)`
|
|
at `hyperv.c:2546`, so this makes the KVM-PV path consistent with the Hyper-V one rather than
|
|
inventing new behaviour.
|
|
|
|
The blunt alternative - clearing `INTERCEPT_VMMCALL` - was correctly rejected: it breaks CPL0
|
|
hypercalls and bugchecks the Hyper-V domains.
|
|
|
|
Blast radius checked on the live host: `svm_patch_hypercall` writes VMMCALL and issues it at
|
|
CPL0; Linux paravirt alternative-patches to VMMCALL, also CPL0; every `KVM_HC_*` already failed
|
|
at CPL>0, so only the *shape* of the failure changes. Confirmed empirically by booting a
|
|
Hyper-V-enlightened Windows guest and a Linux paravirt guest on the patched module - both boot
|
|
and execute normally.
|
|
|
|
Half A ships as `patches/kvm/0002-KVM-x86-emulator-UD-not-GP-for-VMCALL-at-CPL-0.patch`, half B as
|
|
`patches/kvm/0003-KVM-x86-UD-for-hypercalls-issued-at-CPL-0.patch`. Both land in `kvm.ko`, not
|
|
`kvm-amd.ko`, so verify `/sys/module/kvm/srcversion` - checking only `kvm_amd` reports success on
|
|
a stale build.
|
|
|
|
---
|
|
|
|
## CLEARED - TIMER
|
|
|
|
**There is no clock to lie to.** The check does not use the TSC. It times a bare `CPUID` (EAX=0)
|
|
against eight `_mm_lfence()`, using a second thread on another core spinning `counter++` on a
|
|
64-byte-aligned volatile; the "tick" is a cross-core cache-line bounce (the code says so at
|
|
vmaware.hpp:7480, "this is a cache-based counter").
|
|
|
|
`serialize_available = cpu::is_intel()` is hard-false on AMD, so the LFENCE branch always runs.
|
|
Threads are pinned *inside* the guest with `SetThreadGroupAffinity`. The estimator takes a
|
|
500-1000 sample batch per trial, an interquartile mean of the middle 50%, then the **minimum
|
|
across 5 trials**, with samples bracketed by counter-edge spin-waits and followed by
|
|
`burn_random_cycles()` so a hypervisor cannot predict when to freeze the counter thread.
|
|
|
|
**There are TWO detectors, and they OR together** (vmaware.hpp:7484, 7488, 7493). Clearing one
|
|
achieves nothing.
|
|
|
|
### Detector 2 was never the wall - the probe was wrong
|
|
|
|
The first version of `bench/timerprobe.c` reported the exception-latency detector at **6.169**
|
|
against a threshold of 2.5, and two days of work treated that as a hypervisor floor. It was a
|
|
measurement error. VMAware's software side is `RtlCaptureContext` + `ZwRaiseException` inside an
|
|
SEH frame with `EXCEPTION_EXECUTE_HANDLER`, which is a syscall in, a kernel dispatch, and an
|
|
`RtlUnwindEx` out. The probe used `RaiseException` under a vectored handler, which never enters the
|
|
kernel on the software side, so its reference window was several times too short and the ratio
|
|
several times too high.
|
|
|
|
Rebuilt mechanism for mechanism (`__C_specific_handler` scope table, `ZwRaiseException`,
|
|
`EXCEPTION_EXECUTE_HANDLER`), the probe reads **1.47**, and VMAware's own debug line agrees on
|
|
the same boot with the stock `#DB` intercept in place:
|
|
|
|
TIMER: Exception > VMM -> 10167 | nVMM -> 7066 | Ratio -> 1.439
|
|
|
|
Detector 2 passes on stock KVM. No bare-metal Windows baseline was taken - the brief asked for
|
|
one, no physical Windows machine was available - so the only claim made here is the measured one:
|
|
the guest is under the threshold with `#DB` intercepted, by a margin of 1.1.
|
|
|
|
### Detector 1 is a world switch, and the only way to not pay it is to not exit
|
|
|
|
Measured: `cpuid 2054-2383 / lfence 262-310 ticks`, ratio 7.6-8.2, threshold 2.5. The exit itself
|
|
is ~1800 ticks, on the order of 400 ns, an ordinary Zen 4 world switch. Cross-CCD counter
|
|
placement, the exit fastpath, TSC offsetting and CPUID-leaf overrides were all measured and all
|
|
failed (table below).
|
|
|
|
`EXPERIMENTAL-0006-runtime-cpuid-passthrough` clears `INTERCEPT_CPUID` on every guest entry once
|
|
`cpuid_passthrough=1`, so `CPUID` runs on the silicon. Three things follow from raw CPUID, and the
|
|
patch handles each:
|
|
|
|
- **The brand string reverts to the host SKU.** The AMD Processor Name String MSRs
|
|
(`0xC0010030-35`) back `CPUID 0x80000002-4` directly and are writable and per-thread on Zen 4,
|
|
so the declared SKU is written on the pinned core at entry and restored in `svm_vcpu_put()`.
|
|
`THREAD_MISMATCH` stays clear; VMAware's own thread database resolves `7700X` to 16 threads.
|
|
- **Raw CPUID advertises RDPRU** (`0x80000008 EBX[4]`), which KVM masks out of its own CPUID and
|
|
intercepts with `kvm_handle_invalid_op()`, i.e. `#UD`. VMAware's `INTERRUPT_SHADOW` and
|
|
`SINGLE_STEP` both have an RDPRU variant gated on exactly that bit, and both report the `#UD` as
|
|
an "exception anomaly, hypervisor seems to be present with CPUID interception disabled". Two
|
|
full scans fired both checks before this was understood, and an exact-stub reproducer with a
|
|
`kvm_exit` trace showed the TF `#DB` after a native CPUID landing at the right RIP every time -
|
|
the single-step path was never at fault. The RDPRU intercept is cleared together with the CPUID
|
|
one; the guest then reads the pinned core's real `MPERF`/`APERF`.
|
|
- **Raw CPUID must only be enabled after boot.** Windows enumerates KVM's synthetic leaf-1 bits
|
|
(`x2apic`, `tsc-deadline`) during boot and hangs if they vanish mid-enumeration. The module
|
|
parameter is runtime-toggled for this reason.
|
|
|
|
The intercept is also withheld from any vCPU thread whose allowed-CPU mask is wider than one CPU
|
|
(`cpumask_weight(current->cpus_ptr) == 1`), so the brand override cannot leak onto a host core: the
|
|
patch verifies the 1:1 pinning itself rather than trusting the operator.
|
|
|
|
Measured with the passthrough on, VMAware HEAD debug, console session:
|
|
|
|
TIMER: Instruction > VMM -> 219 | nVMM -> 290 | Ratio -> 0.755
|
|
TIMER: Exception > VMM -> 10147 | nVMM -> 7551 | Ratio -> 1.344
|
|
VM detections: 1/85 ===== CONCLUSION: Running on bare metal =====
|
|
|
|
The `#DB` intercept is left in place. Clearing it too was measured (detector 2 drops to 0.96) and
|
|
gains nothing the score needs, while it removes the single-step re-injection KVM relies on for
|
|
its own NMI-window logic. An earlier note here blamed that clear for the `INTERRUPT_SHADOW` and
|
|
`SINGLE_STEP` detections; that was wrong, RDPRU was the cause both times.
|
|
|
|
Measured and dead before the passthrough:
|
|
|
|
| Attempt | Result |
|
|
| --- | --- |
|
|
| BetterTiming TSC compensation | no detection change, 6x slower boot (6.1 s -> 38.6 s CPU) |
|
|
| Hypervisor-Phantom CPUID override | ratio halved to ~810 ticks, **still detected**, and added `SINGLE_STEP` (10 -> 11) |
|
|
| Clear `INTERCEPT_CPUID` from boot | guest never boots |
|
|
| All four intercepts cleared | triple fault, `EFER=0`, instant |
|
|
| CPUID exit fastpath in `svm_exit_handlers_fastpath` | the fastpath runs *after* the world switch (`svm.c:4429`, called at `:4664`), so it cannot remove the exit that is being measured |
|
|
| Cross-CCD topology to grow the baseline | the measured ceiling falls ~1.7x short, and VMAware picks its own nearby core pair anyway |
|
|
| Both of the above combined | still short |
|
|
|
|
---
|
|
|
|
## CLEARED - DBVM (weight 150, VMAware HEAD only)
|
|
|
|
Added to VMAware after v2.8.1; the release build does not fire it, HEAD does, with
|
|
`DBVM: ICEBP failed to advance guest RIP`. Named after a hypervisor-backed debugger, but the sub-check
|
|
that fires is generic: it clears DR0-DR7 through `NtSetContextThread`, executes `F1 C3`
|
|
(`icebp; ret`) under SEH, and compares the exception context's RIP against `stub + 1`.
|
|
|
|
`ICEBP` (`INT1`) raises a trap-like `#DB`, so the frame RIP on hardware is the next instruction.
|
|
The first theory here - that SVM's `#DB` exception intercept catches it and `db_interception()`
|
|
re-injects it with RIP still on the `F1` byte - was wrong, and it was wrong in a way that only a
|
|
trace could show: on Zen 4 the exception intercept never fires for ICEBP at all, and every warm
|
|
reproducer read the correct `+1`. What the `kvm_exit` trace of a cold selftest shows instead:
|
|
|
|
kvm_exit: reason npf rip 0x402ec4 ... intr_info 0x80000301 <- ICEBP at 0x402ec4
|
|
kvm_inj_exception: #DB [reinjected]
|
|
|
|
The `#DB` was being delivered when it hit a nested page fault on an unmapped IDT/handler page.
|
|
The exit carries the pending `#DB` in `EXITINTINFO` with the saved RIP on the ICEBP, and
|
|
`svm_complete_interrupts()` re-queues it as a plain hardware exception, which the next `VMRUN`
|
|
injects at that RIP. So the guest sees the ICEBP's own address whenever the delivery touches a
|
|
page KVM has not mapped yet - deterministic in a fresh VM, sporadic in a running Windows guest
|
|
(page compaction and reclaim keep unmapping NPT entries), which is why the scanner hit it and the
|
|
warm probes did not.
|
|
|
|
`0004-KVM-SVM-intercept-ICEBP-and-skip-it-before-injecting-its-DB` enables SVM's dedicated
|
|
`INTERCEPT_ICEBP`, which fires before the `#DB` exists, skips the instruction with
|
|
`svm_skip_emulated_instruction()` and queues the `#DB` with the same DR6 payload
|
|
`db_interception()` uses. Once RIP has been advanced, an injection that is itself interrupted is
|
|
re-injected with the advanced RIP. This is the shape VMX already has (`is_icebp()` in
|
|
`vmx/vmcs.h`, skipped in `handle_exception_nmi()`).
|
|
|
|
Proof, at CPL0 with no Windows involved - a KVM selftest (`x86/icebp_test.c`, shipped as 5/5 of
|
|
the series) executes `icebp` under a `#DB` handler that records the frame RIP:
|
|
|
|
unpatched svm.c: 0x402ec4 != 0x402ec5 (db_rip != next_rip) FAIL
|
|
with 0004: PASS
|
|
|
|
and the two full scans with 0004 in place no longer fire `DBVM`.
|
|
|
|
---
|
|
|
|
## Method notes worth keeping
|
|
|
|
**Score from the console session.** OpenSSH on Windows puts you in session 0, the services
|
|
session. Every score this project took for its first two days came from there. Use a scheduled
|
|
task with `/it` and confirm `(Get-Process -Id $PID).SessionId` is 1 - `schtasks /run` reports
|
|
success regardless of where the task actually ran.
|
|
|
|
**One change per boot.** The 70-second silent power-off from an ACPI namespace failure has no
|
|
error message and no log line. The only way to attribute it is to have changed one thing.
|
|
|
|
**Read the debug line, not the count.** `[ DETECTED ]` with no debug line is not "cleared" -
|
|
some targets return silently (`Xen` at vmaware.hpp:9174, `BXPC` at :9186).
|
|
|
|
---
|
|
|
|
## Status
|
|
|
|
VMAware v2.8.1 debug: 1/85 GPU_CAPABILITIES Running on bare metal
|
|
VMAware HEAD debug: 1/85 GPU_CAPABILITIES Running on bare metal, likeliness 20%
|
|
|
|
with `0001-0004` in the modules, `EXPERIMENTAL-0006` built in and enabled after boot on a 16 vCPU
|
|
guest pinned 1:1 to one CCD, declaring a Ryzen 7 7700X. The remaining check needs a real GPU passed
|
|
through, which needs a spare card. `bench/timerprobe.c` reproduces both `TIMER` detectors and
|
|
`bench/`-adjacent probes for the others live in `docs/TESTING.md`.
|