Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The active test run was aborted. Reason: Test host process crashed #451

Open
3 tasks
fanyang-mono opened this issue Jul 25, 2023 · 8 comments
Open
3 tasks

Comments

@fanyang-mono
Copy link
Member

fanyang-mono commented Jul 25, 2023

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=351450

Build leg reported

chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution

Pull Request

dotnet/runtime#89217

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The active test run was aborted. Reason: Test host process crashed",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

  • Feature changes/additions
  • Bug fixes
  • Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Report

Build Definition Step Name Console log Pull Request
921574 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75581
2620937 dotnet-interactive Test / Blame Log
2620694 dotnet-interactive Test / Blame Log
917698 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76752
2619106 dotnet-interactive Test / Blame Log
914993 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76725
914308 dotnet/roslyn Run Integration Tests Log
914177 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76692
913987 dotnet/roslyn Run Integration Tests Log
912714 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76695
2616802 dotnet-interactive Test / Blame Log
2616717 dotnet-interactive Test / Blame Log
2615694 dotnet-fsharp CmdLine@2 Log
2615469 dotnet-interactive Test / Blame Log
2614768 dotnet-interactive Test / Blame Log
909122 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75970
2614588 dotnet-fsharp CmdLine@2 Log
908618 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76649
908555 dotnet/roslyn Run Integration Tests Log
908425 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76646
2614116 dotnet-interactive Test / Blame Log
2614101 dotnet-interactive Test / Blame Log
2613990 dotnet-interactive Test / Blame Log
2613544 dotnet-interactive Test / Blame Log
2612493 dotnet-interactive Test / Blame Log
906308 dotnet/roslyn Run Integration Tests Log
2612433 dotnet-interactive Test / Blame Log
2612333 dotnet-interactive Test / Blame Log
905800 dotnet/roslyn Run Integration Tests Log
2611980 dotnet-interactive Test / Blame Log
2611872 dotnet-interactive Test / Blame Log
2611807 dotnet-interactive Test / Blame Log
905108 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#70455
905102 dotnet/roslyn Run Integration Tests Log
903012 dotnet/roslyn Run Integration Tests Log
900073 dotnet/roslyn Run Integration Tests Log
899709 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76517
899492 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75581
899447 dotnet/roslyn Run Integration Tests Log
Build Definition Test Pull Request
921691 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46110
921547 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45880
921685 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7136
921430 dotnet/sdk Microsoft.DotNet.Watch.UnitTests.BrowserLaunchTests.Microsoft.DotNet.Watch.UnitTests.BrowserLaunchTests.UsesBrowserSpecifiedInEnvironment dotnet/sdk#46100
921655 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45552
921527 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#45979
920975 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46087
921295 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46096
920675 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46021
921216 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#46064
921303 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution dotnet/sdk#46002
920211 dotnet/sdk Microsoft.NET.ToolPack.Tests.dll.3.WorkItemExecution dotnet/sdk#45999
921223 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46089
921100 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46045
921042 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46045
920824 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution dotnet/sdk#46086
920725 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#46034
920836 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45990
920760 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46085
920748 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#46084
919402 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.2.WorkItemExecution dotnet/sdk#46033
920701 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46026
920580 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#46055
919413 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46034
920410 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#46065
920549 dotnet/aspire Aspire.Hosting.Redis.Tests.WorkItemExecution
920462 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#46061
920317 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46066
920181 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46036
919906 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#46048
920131 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46059
920096 dotnet/aspire Aspire.Hosting.Redis.Tests.WorkItemExecution dotnet/aspire#7073
920061 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution dotnet/sdk#46002
919998 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#46019
919972 dotnet/aspire Aspire.Hosting.PostgreSQL.Tests.PostgresFunctionalTests.Aspire.Hosting.PostgreSQL.Tests.PostgresFunctionalTests.VerifyWithInitBindMount dotnet/aspire#7073
919859 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46023
919913 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46011
919794 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46049
919814 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45362
919735 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#42861
919723 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46048
919622 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46041
919642 dotnet/aspire Aspire.Oracle.EntityFrameworkCore.Tests-net9.0.WorkItemExecution dotnet/aspire#7121
919623 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.6.WorkItemExecution dotnet/sdk#46042
919630 dotnet/aspire Aspire.Microsoft.Data.SqlClient.Tests.WorkItemExecution dotnet/aspire#7073
919352 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#46023
919399 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46031
919353 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46024
919566 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#45986
919581 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46044
919569 dotnet/sdk EndToEnd.Tests.dll.WorkItemExecution dotnet/sdk#46014
919584 dotnet/aspire Aspire.EndToEnd.Tests.IntegrationServicesTests.Aspire.EndToEnd.Tests.IntegrationServicesTests.VerifyComponentWorks dotnet/aspire#7073
919372 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46026
919576 dotnet/aspire Aspire.Pomelo.EntityFrameworkCore.MySql.Tests.WorkItemExecution dotnet/aspire#7124
919342 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46021
919158 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.21.WorkItemExecution dotnet/sdk#45637
919274 dotnet/aspire Aspire.Hosting.Nats.Tests.WorkItemExecution dotnet/aspire#7129
919185 dotnet/aspire Aspire.Hosting.Nats.Tests.WorkItemExecution dotnet/aspire#7092
919085 dotnet/aspire Aspire.Hosting.Nats.Tests.WorkItemExecution dotnet/aspire#7127
919096 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution
919013 dotnet/sdk Microsoft.NET.Publish.Tests.GivenThatWeWantToPublishAnAotApp.Microsoft.NET.Publish.Tests.GivenThatWeWantToPublishAnAotApp.NativeAot_hw_runs_with_no_warnings_when_PublishAot_is_enabled dotnet/sdk#46000
918950 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#45999
918870 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46011
918847 dotnet/aspire Aspire.Hosting.Redis.Tests.RedisFunctionalTests.Aspire.Hosting.Redis.Tests.RedisFunctionalTests.RedisInsightWithDataShouldPersistStateBetweenUsages(useVolume: False) dotnet/aspire#7115
918817 dotnet/sdk Microsoft.NET.Publish.Tests.GivenThatWeWantToPublishAnAotApp.Microsoft.NET.Publish.Tests.GivenThatWeWantToPublishAnAotApp.NativeAot_hw_runs_with_no_warnings_when_PublishAot_is_enabled dotnet/sdk#46010
918651 dotnet/sdk Microsoft.NET.Publish.Tests.dll.2.WorkItemExecution dotnet/sdk#45932
917252 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45964
2621376 dotnet-roslyn workitem_103.WorkItemExecution
918736 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#42416
918546 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45986
918497 dotnet/sdk dotnet-watch.Tests.dll.6.WorkItemExecution dotnet/sdk#45873
917537 dotnet/sdk Microsoft.NET.Publish.Tests.dll.3.WorkItemExecution dotnet/sdk#45965
918658 dotnet/aspire Aspire.Hosting.Nats.Tests.WorkItemExecution
918629 dotnet/aspire Aspire.Hosting.Nats.Tests.WorkItemExecution dotnet/aspire#7026
918584 dotnet/sdk dotnet-watch.Tests.dll.6.WorkItemExecution dotnet/sdk#46003
918440 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#45998
918502 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.20.WorkItemExecution dotnet/sdk#42861
913213 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45861
918391 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.21.WorkItemExecution dotnet/sdk#45935
917907 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#45981
917952 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45991
917565 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45973
917886 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45944
917877 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45988
917669 dotnet/sdk dotnet-watch.Tests.dll.5.WorkItemExecution dotnet/sdk#45873
917843 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45369
917827 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45874
917740 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45977
917766 dotnet/aspire Aspire.Hosting.SqlServer.Tests.WorkItemExecution
917748 dotnet/aspire Aspire.Hosting.SqlServer.Tests.WorkItemExecution dotnet/aspire#6840
917544 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution dotnet/aspnetcore#59788
917355 dotnet/sdk dotnet-watch.Tests.dll.6.WorkItemExecution dotnet/sdk#45369
916462 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45944
917329 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7084
917246 dotnet/aspire Aspire.Hosting.Redis.Tests.RedisFunctionalTests.Aspire.Hosting.Redis.Tests.RedisFunctionalTests.RedisInsightWithDataShouldPersistStateBetweenUsages(useVolume: False)
917213 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45957
917133 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45935
917057 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45932
916895 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45951
916933 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45398
Displaying 100 of 383 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
23 166 422

Known issue validation

Build: 🔎
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.
Validation performed at: 2/8/2024 9:24:30 PM UTC

@andriipatsula
Copy link
Member

Hello @fanyang-mono, could you please update the "ErrorMessage" : "" by following the step by step documentation on how to create a known issue

@fanyang-mono
Copy link
Member Author

Updated.

@pavelsavara
Copy link
Member

pavelsavara commented Nov 7, 2023

This is duplicate of dotnet/runtime#89409, except here it's not only wasm debugger

@dougbu
Copy link
Member

dougbu commented Nov 23, 2023

@pavelsavara and @fanyang-mono, should this issue be closed in favour of dotnet/runtime#94560❔ if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

@pavelsavara
Copy link
Member

@pavelsavara and @fanyang-mono, should this issue be closed in favour of

You mean dotnet/runtime#89409

I think we should keep this one, because this one has some hits which are not just wasm debugger related.
Here we see other repo failures, on roslyn and aspnetcore CI.

if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

I have not investigated it myself yet. But got questions which could help to narrow it down

  • are there crash dump files on the machine ?
  • are there some hints on what's going on in the log files before crash ?
  • is this OOM or stack overflow in disguise ?
  • did we run out of disk space or number of inodes ?
  • hit some limit in docker container ?
  • is this always the same OS, version, image ?
  • could this be hardware issue ?

@fanyang-mono
Copy link
Member Author

This issue was originally created to track issues happened to wasm debugger tests (belonging to dotnet/runtime repo). However, the error message was generic enough to include issues from other repos, such as dotnet/roslyn and dotnet/aspnetcore. We need insights from owners from dotnet/roslyn and dotnet/aspnetcore to confirm that they are aware of these issue. And if they are okay to close this issue as well. We could close then.

@mitchdenny
Copy link
Member

We just got an instance of this in the dotnet/aspire repo. It wasn't a WASM related failure for us. Just looks like the test process crashed and took a few tests with it.

@nagilson
Copy link
Member

nagilson commented Dec 3, 2024

@marcpopMSFT was working on getting the test infrastructure to generate crash dumps when it times out to help with investigation. I think it was generating the dumps but not yet correctly uploading them to the artifacts.

We are waiting to hear from the infra team on an issue with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants