Skip to content

Extend server/discover probe timeout in in-memory tests to fix flaky Windows Debug CI hang (#1701)#1702

Merged
jeffhandley merged 3 commits into
mainfrom
pranavsenthilnathan/fix-task-cancellation-test-hang
Jul 24, 2026
Merged

Extend server/discover probe timeout in in-memory tests to fix flaky Windows Debug CI hang (#1701)#1702
jeffhandley merged 3 commits into
mainfrom
pranavsenthilnathan/fix-task-cancellation-test-hang

Conversation

@PranavSenthilnathan

@PranavSenthilnathan PranavSenthilnathan commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes the flaky Windows Debug CI test-host hang tracked in #1701, which surfaced most often as TaskCancellationIntegrationTests.TaskTool_CancellationToken_GetTaskShowsWorkingBeforeCancel.

The fix is a test-only change: ClientServerTestBase.CreateMcpClientForServer now sets DiscoverProbeTimeout = TestConstants.DefaultTimeout for the in-memory client. This extends the production default from 5 seconds to the shared 60-second test timeout, giving slow or overloaded CI agents enough time to complete the probe while preserving a bounded failure mode. Previously, the default timeout could be spuriously exceeded, triggering a protocol-fallback cascade that ends in a hang (details below).

Root cause

A July2026-capable client (the default) does not use the initialize handshake. Instead it calls server/discover to learn the server's capabilities, and only falls back to the legacy initialize handshake if the probe fails or times out. That probe is bounded by McpClientOptions.DiscoverProbeTimeout (default 5s).

On a slow CI agent, the in-memory server/discover round-trip could occasionally take longer than 5s. When that happened:

  1. The probe timed out and the client fell back to the initialize handshake, negotiating an older protocol version.
  2. The older protocol does not negotiate the tasks extension, so the server treated the call as an ordinary (non-task) tool invocation.
  3. TaskCancellationIntegrationTests registers a tool whose body is Task.Delay(Timeout.Infinite, cancellationToken). Without the tasks extension, the server ran that tool inline on the tools/call request instead of dispatching it as a cancellable background task.
  4. The inline tool never completes, so the server never sends the tools/call response, and the client's await CallToolRawAsync(...) blocks forever.
  5. Execution never reaches the test body's asserts — it's a true hang, not a failed assertion — so the 7-minute blame/hang-dump timeout eventually kills the test host, failing the whole run.

Because the trigger is purely environmental timing, the failure was intermittent and Windows-Debug-CI-specific.

How it was diagnosed (hang dump)

The blame-timeout hang dump was the key. Walking the managed thread stacks:

  • Client thread was parked inside CallToolRawAsyncSendRequestAsync, awaiting the JSON-RPC response for tools/call — i.e. the client had sent the request and was waiting for a reply that never came.
  • Server thread was inside the tool handler executing Task.Delay(Timeout.Infinite, …) synchronously on the request-handling path — proving the tool was running inline rather than as a background task.

That pairing (client awaiting a tools/call response while the server is blocked inside the tool on the same call) is only possible when the tasks extension was not negotiated. Confirming the negotiated protocol version was older than July2026 pointed directly at a server/discoverinitialize fallback, and the only thing that forces that fallback in the in-memory harness is the probe timeout expiring — i.e. CI slowness tripping the 5s DiscoverProbeTimeout.

The fix

protected async Task<McpClient> CreateMcpClientForServer(McpClientOptions? clientOptions = null)
{
    clientOptions ??= new McpClientOptions();

    // Disable the server/discover probe timeout to avoid CI slowness spuriously tripping it (issue #1701).
    // Tests that need a specific probe timeout should create their own client instead of using this helper.
    clientOptions.DiscoverProbeTimeout = TestConstants.DefaultTimeout;

    return await McpClient.CreateAsync(/* ... */);
}

Within ClientServerTestBase, the shared 60-second test timeout gives slow or overloaded CI agents substantially more time than the production 5-second default to complete the in-memory server/discover probe, while keeping the operation bounded. Applying it in the helper avoids per-caller churn and ensures callers that supply other client options receive the same protection, with no production code change.

Tests that intentionally exercise the probe-timeout fallback (July2026ProtocolFallbackTests) build their own transport and call McpClient.CreateAsync directly, so they bypass this helper and are unaffected.

Testing

  • dotnet build — clean (0 warnings / 0 errors).
  • Full in-memory test suite: 2174 passed, 0 failed, 4 skipped. (The external-process stdio families are excluded here only due to a pre-existing local TestServer.exe PATH issue unrelated to this change; they build and run in CI.)
  • The previously-flaky task-cancellation, protocol-fallback, and back-compat tests all pass.

@PranavSenthilnathan
PranavSenthilnathan marked this pull request as draft July 14, 2026 01:21
@PranavSenthilnathan PranavSenthilnathan changed the title Fix flaky task cancellation test hang on Windows Debug CI (#1701) [WIP] Fix flaky task cancellation test hang on Windows Debug CI (#1701) Jul 14, 2026
Comment thread src/ModelContextProtocol.Core/Server/McpServerImpl.cs Outdated
On slow Windows Debug CI, the default 5s DiscoverProbeTimeout can be spuriously exceeded when a July2026-capable client probes server/discover over the in-memory pipe. The client then falls back to the initialize handshake and negotiates an older protocol, dropping tasks-extension negotiation. For TaskCancellationIntegrationTests, whose tool runs Task.Delay(Timeout.Infinite), the fallback makes the server run the tool inline instead of as a background task, so the tools/call never returns and the test host hangs until the blame timeout.

Force DiscoverProbeTimeout = Timeout.InfiniteTimeSpan in ClientServerTestBase.CreateMcpClientForServer. The in-memory pipe server always answers server/discover immediately, so the probe never legitimately needs to time out. This is applied even when a caller supplies its own options so no test can reintroduce the flake. Tests that intentionally exercise the probe-timeout fallback (July2026ProtocolFallbackTests) build their own transport and bypass this helper, so they are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@PranavSenthilnathan
PranavSenthilnathan force-pushed the pranavsenthilnathan/fix-task-cancellation-test-hang branch from 42a99f0 to 0726434 Compare July 16, 2026 00:47
@PranavSenthilnathan PranavSenthilnathan changed the title [WIP] Fix flaky task cancellation test hang on Windows Debug CI (#1701) Disable server/discover probe timeout in in-memory tests to fix flaky Windows Debug CI hang (#1701) Jul 16, 2026
@PranavSenthilnathan
PranavSenthilnathan marked this pull request as ready for review July 16, 2026 01:09
Comment thread tests/ModelContextProtocol.Tests/ClientServerTestBase.cs Outdated
Co-authored-by: Stephen Halter <halter73@gmail.com>
Comment thread tests/ModelContextProtocol.Tests/ClientServerTestBase.cs Outdated
@PranavSenthilnathan PranavSenthilnathan changed the title Disable server/discover probe timeout in in-memory tests to fix flaky Windows Debug CI hang (#1701) Extend server/discover probe timeout in in-memory tests to fix flaky Windows Debug CI hang (#1701) Jul 24, 2026
@jeffhandley
jeffhandley merged commit f7c61fa into main Jul 24, 2026
10 checks passed
@jeffhandley
jeffhandley deleted the pranavsenthilnathan/fix-task-cancellation-test-hang branch July 24, 2026 23:46
@jeffhandley jeffhandley mentioned this pull request Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants