Remoting tests: 500 on discovery document, flaky under suite #4

New issue

Closed

opened 2026-07-11 21:03:38 +01:00 by notazof · 0 comments

notazof commented

2026-07-11 21:03:38 +01:00

Owner

Status

Flaky, pre-existing on main and fix/blog-detail. Two tests in src/Yavsc.Org.Tests/Mandatory/Remoting.cs fail when run as part of the full suite, but pass in isolation.

Observed

dotnet test src/Yavsc.Org.Tests/Yavsc.Org.Tests.csproj fails 2 tests:

System.Exception : Error connecting to
https://127.0.0.1:37433/.well-known/openid-configuration:
Internal Server Error
   at Yavsc.Org.Tests.Remoting.ObtainServiceToken()
   at Yavsc.Org.Tests.Remoting.ObtainResourceOwnerPasswordToken()

The test class is decorated [Collection("Yavsc Server")] and uses WebServerFixture (the collection-scoped fixture, not TestWebApplicationFactory). It hits the shared HTTPS host bound to a dynamic port (Addresses[0]), so the URL is correct.

The two tests pass in isolation:

dotnet test ... --filter "FullyQualifiedName~Remoting.ObtainServiceToken"
dotnet test ... --filter "FullyQualifiedName~Remoting.ObtainResourceOwnerPasswordToken"

This points to a shared-state dependency on a test that runs before Remoting in the full suite. Likely candidates:

WebServerFixture initialises a UserManager<ApplicationUser> and seeds a test user in BuildApp (or in EnsureUser). If a previous test has run a SaveChanges that detached/disabled the user, the password token request fails.
IdentityServer8's IClientStore is backed by the same in-memory database; if a previous test mutated the TestClientId/TestClientSecret, the discovery endpoint is fine but the token request is not — yet the test reports the discovery request failed, which is upstream of the token request.
The WebServerFixture is initialised lazily on first construction (xUnit collection semantics), so the Addresses array is captured once and shared across every test. If a test restarts the host (e.g. with Dispose), the next Remoting test reads a stale port.

Reproduction

dotnet test src/Yavsc.Org.Tests/Yavsc.Org.Tests.csproj

Result: 6 failed, 35 passed (the 4 ClientControllerCollectionTests failures are the bug documented in the sibling issue; the 2 Remoting failures are this bug).

dotnet test src/Yavsc.Org.Tests/Yavsc.Org.Tests.csproj \
  --filter "FullyQualifiedName~Remoting.ObtainServiceToken"

Result: 1 passed (no flake in isolation).

What we need

Identify the test that runs immediately before Remoting in alphabetical order. xUnit v3 does not guarantee an order by default, so this requires a run with [Trait("priority", ...)] or running with --logger "console;verbosity=detailed" and reading the test discovery order.
Once identified, check whether that test mutates the shared WebServerFixture state (in particular _sharedTestingUserPassword, _sharedTestClientSecret).
The fix is likely to make Remoting either (a) tolerant of an already-initialised fixture (re-read the secrets from the live Configuration rather than the captured copies) or (b) run as the first test in the collection by ordering it last alphabetically and prefixing with 0_ or by adding [CollectionOrder(1)] (xUnit v3 has ITestCollectionOrderer).

Recommendation

Start with reading the test execution order and the _sharedXxx fields of WebServerFixture after the first non-Remoting test runs. The 500 on discovery is the symptom of IdentityServer8 returning 500 — look at the server log output captured in the test result .html file (under src/Yavsc.Org.Tests/TestResults/Yavsc.Org.tests.html) for the actual stack trace.

## Status **Flaky, pre-existing on `main` and `fix/blog-detail`.** Two tests in `src/Yavsc.Org.Tests/Mandatory/Remoting.cs` fail when run as part of the full suite, but pass in isolation. ## Observed `dotnet test src/Yavsc.Org.Tests/Yavsc.Org.Tests.csproj` fails 2 tests: ``` System.Exception : Error connecting to https://127.0.0.1:37433/.well-known/openid-configuration: Internal Server Error at Yavsc.Org.Tests.Remoting.ObtainServiceToken() at Yavsc.Org.Tests.Remoting.ObtainResourceOwnerPasswordToken() ``` The test class is decorated `[Collection("Yavsc Server")]` and uses `WebServerFixture` (the collection-scoped fixture, not `TestWebApplicationFactory`). It hits the shared HTTPS host bound to a dynamic port (`Addresses[0]`), so the URL is correct. The two tests pass in isolation: ``` dotnet test ... --filter "FullyQualifiedName~Remoting.ObtainServiceToken" dotnet test ... --filter "FullyQualifiedName~Remoting.ObtainResourceOwnerPasswordToken" ``` This points to a shared-state dependency on a test that runs before `Remoting` in the full suite. Likely candidates: - `WebServerFixture` initialises a `UserManager<ApplicationUser>` and seeds a test user in `BuildApp` (or in `EnsureUser`). If a previous test has run a `SaveChanges` that detached/disabled the user, the password token request fails. - IdentityServer8's `IClientStore` is backed by the same in-memory database; if a previous test mutated the `TestClientId`/`TestClientSecret`, the discovery endpoint is fine but the token request is not — yet the test reports the discovery request failed, which is upstream of the token request. - The `WebServerFixture` is initialised lazily on first construction (xUnit collection semantics), so the `Addresses` array is captured **once** and shared across every test. If a test restarts the host (e.g. with `Dispose`), the next `Remoting` test reads a stale port. ## Reproduction ``` dotnet test src/Yavsc.Org.Tests/Yavsc.Org.Tests.csproj ``` Result: 6 failed, 35 passed (the 4 `ClientControllerCollectionTests` failures are the bug documented in the sibling issue; the 2 `Remoting` failures are this bug). ``` dotnet test src/Yavsc.Org.Tests/Yavsc.Org.Tests.csproj \ --filter "FullyQualifiedName~Remoting.ObtainServiceToken" ``` Result: 1 passed (no flake in isolation). ## What we need - Identify the test that runs immediately before `Remoting` in alphabetical order. xUnit v3 does not guarantee an order by default, so this requires a run with `[Trait("priority", ...)]` or running with `--logger "console;verbosity=detailed"` and reading the test discovery order. - Once identified, check whether that test mutates the shared `WebServerFixture` state (in particular `_sharedTestingUserPassword`, `_sharedTestClientSecret`). - The fix is likely to make `Remoting` either (a) tolerant of an already-initialised fixture (re-read the secrets from the live `Configuration` rather than the captured copies) or (b) run as the **first** test in the collection by ordering it last alphabetically and prefixing with `0_` or by adding `[CollectionOrder(1)]` (xUnit v3 has `ITestCollectionOrderer`). ## Recommendation Start with reading the test execution order and the `_sharedXxx` fields of `WebServerFixture` after the first non-`Remoting` test runs. The 500 on discovery is the symptom of IdentityServer8 returning 500 — look at the server log output captured in the test result `.html` file (under `src/Yavsc.Org.Tests/TestResults/Yavsc.Org.tests.html`) for the actual stack trace.

notazof closed this issue

2026-08-02 23:55:56 +01:00

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

notazof/yavsc#4

No description provided.

Rows
Columns