Fix dangling-pointer walk in ResidencySet during DiscardCommandList

[vdwtanner]

Problem

Access violation in D3D12TranslationLayer::ResidencySet::Close during device shutdown when a Resource is destroyed while still tracked in an open per-CL residency set. Surfaces in release builds as:

00 D3D12TranslationLayer::ResidencySet::Close            [include/Residency.h:161]   <-- AV
01 D3D12TranslationLayer::ImmediateContext::Shutdown     [src/immediatecontext.cpp:378]
02 D3D12TranslationLayer::ImmediateContext::~ImmediateContext
03 std::_Optional_destruct_base<ImmediateContext>::~...
04 D3D9on12::Device::~Device                             [9on12device.cpp:569]
05 D3D9on12::DeviceInternal::DestroyDevice               [9on12deviceinternal.cpp:282]
06 d3d9!DestroyDeviceLHDDI

Root cause

ResidencySet::Set holds raw ManagedObject* with no ownership and no destroy-time notification. Close() walks Set and dereferences every entry to clear its CommandListsUsedOn bit, so it assumes every Resource backing an entry is still alive at Close-time.

That assumption holds for the submit paths (ExecuteCommandList, SubmitCommandQueueCommand, PreExecuteCommandQueueCommand), where Resources are kept alive by the deferred-deletion queue + GPU fence until well after the CL completes.

It does not hold for the teardown path. At ~ImmediateContext::Shutdown, a CL that was recorded but never submitted may have entries whose Resources were already released synchronously via a DDI callback (the D3D9 path D3D9on12::DestroyResource -> CleanupResourceBindingsAndRelease -> Resource::Release -> ~Resource destroys immediately when the refcount hits zero, with no deferred-deletion-queue intermediation). Close() then walks dangling pointers and AVs.

Fix

Per Jesse's review suggestion, the teardown path now routes through a new method on ResidencyManager that drops the per-CL set's tracking state without walking Set:

• ResidencyManager::DiscardResidencySet(ResidencySet*) -- new, the teardown entry point.
• ResidencySet::Discard() -- private, friended to ResidencyManager. Resets state without iterating.
• CommandListManager::DiscardCommandList() -- now calls GetResidencyManager().DiscardResidencySet(...) instead of Close().

The submit paths are unchanged: they still go through Close() and walk Set. Only the teardown discard path -- where the lifetime guarantee cannot hold -- skips the walk.

A short lifetime-contract comment on ResidencySet documents the invariant + the teardown carve-out for the next reader.

No API signature changes. No new locking. Two files, ~16 net lines.

Verification

Tested on x64 d3d9on12.dll against the failing CheckVideoProcessorCaps#MockD3D12Runtime TAEF test under AppVerifier:

appverif -enable Handles Heaps Leak Locks Memory SRWLock Threadpool TLS DirtyStacks -for Te.exe
WinDbgx -c "g" te.exe .\D3DFunc_12_TaefTests\D3DFunc_9on12_Core.dll /logOutput:low /Inproc /Name:Func_Video9on12::CheckVideoProcessorCaps#MockD3D12Runtime

Branch + Config	AV during D3D9 device-destroy	TAEF LeakTest	Process exit
Pre-fix master, Release	fires (2/2)	n/a (aborted)	trapped
This PR, Release	does not fire (2/2)	[Passed] (2/2)	clean exit

Test note: the verifying system has a hardware GPU, so CheckVideoProcessorCaps#MockD3D12Runtime itself reports [Skipped] ("9on12 Video Tests can only run on the Basic Display Driver"). The failure exercised reproduces during ClassCleanup regardless; the basic-display gating in the original repro is sufficient but not necessary to surface the failure.

Known issue (DBG builds only)

The existing #if TRANSLATION_LAYER_DBG trip-wire at Residency.h:74 (assert(!val) inside ~ManagedObject, checking that CommandListsUsedOn[i] is all false at Resource destruction) still fires on this repro under DBG builds. Empirically verified: 2/2 runs against d3d9on12.dll built from this branch.

The trip-wire fires from D3D9on12::DestroyResource during ClassCleanup -- before ~ImmediateContext::Shutdown -> DiscardCommandList ever runs -- so this fix's Close->Discard swap is downstream of the trip-wire's trigger. The bit-leak originates at recording time when Insert set the bit; the canonical clear path (Close walking Set) never runs in the teardown scenario.

This PR is intentionally scoped to fix the release-build AV described above. The DBG-only trip-wire is a separate concern with three options worth considering as a follow-up:

1. Gate ~ManagedObject's assert behind something narrower than TRANSLATION_LAYER_DBG (e.g., a flag that excludes the known-benign teardown bit-leak).
2. Clear the bit defensively in Discard() for entries we know are about to be destroyed.
3. Address the underlying bit-leak with a small targeted destroy-notification mechanism (the prior iteration of this PR; reverted in favour of the smaller fix here per review feedback).

Happy to spin a follow-up PR for whichever the team prefers.

[jenatali]

It seems like all this complexity (and extra locking) could be short-circuited by making CommandListManager::DiscardCommandList call a new method on the residency manager instead of Close that just doesn't try to touch the resources.

[vdwtanner]

Thanks Jesse, pivoted to your approach. Force-pushed the branch: the new ResidencyManager::DiscardResidencySet is called from CommandListManager::DiscardCommandList, no API signature changes, no extra locking. Also closing the sibling D3D9On12 / D3D11On12 PRs (#112 / #57) - under your design there's no API break so they're no longer needed.

[jenatali]

GitHub comments shouldn't be referencing internal bug IDs.

[jenatali]

Comments are still too verbose / referring to past state for my taste, but the change itself seems fine.