THE SYMPTOM: My NVIDIA Tesla P40 would stop responding after a VM shutdown. No error messages, just a dead GPU that required a full host reboot to recover. WHAT I EXPECTED: A clean shutdown of a VM with GPU passthrough should leave the GPU in a ready state.

I assumed the host would handle power states gracefully. WHAT ACTUALLY HAPPENED: The GPU went into D3cold, a low-power state that it couldn't