Skip to content

[Wasm][Perf] Wasm Wasm BDN perf tests are failing to build with top level stack overflow. #98334

New issue
New issue
Closed
Closed
[Wasm][Perf] Wasm Wasm BDN perf tests are failing to build with top level stack overflow.#98334
Assignees
kg
Labels
area-Infrastructure-mono

Description

@LoopedBard3
LoopedBard3
opened on Feb 13, 2024

Description

Wasm BenchmarkDotNet performance tests are failing to build and run in performance pipeline. Top level error is a Stack Overflow but that seems to be a downstream issue rather than the primary issue.

Reproduction Steps

General steps:

  • Build browser wasm from runtime with build.sh -ci -arch wasm -os browser -s mono+libs+host+packs -c Release /p:AotHostArchitecture=x64 /p:AotHostOS=linux
  • Build WBT to create dotnet-latest directory in /artifacts/bin: ./dotnet.sh build -p:TargetOS=browser -p:TargetArchitecture=wasm /nr:false /p:TreatWarningsAsErrors=true /p:Configuration=Release /p:ContinuousIntegrationBuild=true /t:InstallWorkloadUsingArtifacts ./src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj
  • Prepare runtime's artifacts for copying to performance repo:
mkdir -p <runtime>/artifacts/staging/built-nugets &&
      cp -r <runtime>/artifacts/bin/dotnet-latest <runtime>/artifacts/staging &&
      cp -r <runtime>/artifacts/bin/microsoft.netcore.app.runtime.browser-wasm <runtime>/artifacts/staging &&
      cp -r <runtime>/artifacts/bin/microsoft.netcore.app.ref <runtime>/artifacts/staging &&
      cp -r <runtime>/artifacts/packages/Release/Shipping/Microsoft.NET.Sdk.WebAssembly.Pack* <runtime>/artifacts/staging/built-nugets &&
      cp -r <runtime>/artifacts/packages/Release/Shipping/Microsoft.NETCore.App.Ref* <runtime>/artifacts/staging/built-nugets

mkdir -p <performance>/artifacts/bin/wasm/wasm-data && 
mkdir -p <performance>/artifacts/bin/wasm/dotnet && 
cp -r <runtime>/artifacts/staging/dotnet-latest/* <performance>/artifacts/bin/wasm/dotnet && 
cp -r <runtime>/artifacts/staging/built-nugets <performance>/artifacts/bin/wasm && 
cp <runtime>/src/mono/browser/test-main.js <performance>/artifacts/bin/wasm/wasm-data/test-main.js
  • Run the benchmarks_ci.py script for BDN testing: python3 <performance>/scripts/benchmarks_ci.py --csproj <performance>/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture x64 -f net9.0 --dotnet-versions 9.0.100-alpha.1.24070.3 --wasm --bdn-artifacts ./artifacts/BenchmarkDotNet.Artifacts --bdn-arguments='--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoWASM NoMono --wasmEngine ./.jsvu/bin/v8-12.0.267 "--wasmArgs=--expose_wasm --module" --wasmDataDir <performance>/artifacts/bin/wasm/wasm-data --logBuildOutput --generateBinLog' (replace with path to respective repo)

Using dotnet/performance scripts/benchmarks_local.py (on Linux):

  • Clone https://github.com/dotnet/performance
  • Maybe Optional: Install JSVU and V8
  • Change directory into performance/scripts and run (replacing repo and jsvu paths): sudo python3 benchmarks_local.py --commits ed9f475dde7f3cb0e05a7613e82d69bd1a42ce0e --repo-storage-path "/home/<user>/runtimes" --run-types WasmInterpreter --filter *Span.IndexerBench.CoveredIndex2* *WriteReadAsync* --wasm-engine-path /home/<user>/.jsvu/engines/v8/v8
  • This should install latest dotnet, clone the given commit to the repo-storage-path, build Wasm, and attempt to run using V8.

Expected behavior

BenchmarkDotNet builds and runs the tests successfully as demonstrated in the last successful run https://dev.azure.com/dnceng/internal/_build/results?buildId=2374634&view=results (...wasm wasm micro...)

Actual behavior

Full logs can be found in ... wasm wasm micro ... job in run https://dev.azure.com/dnceng/internal/_build/results?buildId=2375008&view=results

[2024/02/09 19:07:59][INFO] // start /home/helixbot/work/B1A209D8/p/dotnet/dotnet  build -c Release --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true /p:NuGetPackageRoot="/home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/packages" -bl:Job-SCUWQJ-build-no-restore.binlog in /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-SCUWQJ
[2024/02/09 19:07:59][INFO] MSBuild version 17.10.0-preview-24068-04+f9b862a13 for .NET
[2024/02/09 19:08:00][INFO]   Reporting -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/Reporting/Release/netstandard2.0/Reporting.dll
[2024/02/09 19:08:00][INFO]   BenchmarkDotNet.Extensions -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/BenchmarkDotNet.Extensions/Release/netstandard2.0/BenchmarkDotNet.Extensions.dll
[2024/02/09 19:08:06][INFO]   MicroBenchmarks -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/MicroBenchmarks/Release/net9.0/MicroBenchmarks.dll
[2024/02/09 19:08:09][INFO] CSC : warning CS8002: Referenced assembly 'MicroBenchmarks, Version=42.42.42.42, Culture=neutral, PublicKeyToken=null' does not have a strong name. [/home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-SCUWQJ/BenchmarkDotNet.Autogenerated.csproj]
[2024/02/09 19:08:09][INFO]   BenchmarkDotNet.Autogenerated -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-SCUWQJ/bin/net9.0/browser-wasm/Job-SCUWQJ.dll
[2024/02/09 19:08:10][INFO]   Optimizing assemblies for size may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink
[2024/02/09 19:08:10][INFO]   Optimizing assemblies for size. This process might take a while.
[2024/02/09 19:08:22][INFO]   BenchmarkDotNet.Autogenerated -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-SCUWQJ/bin/net9.0/browser-wasm/publish/
[2024/02/09 19:08:23][INFO]   Reporting -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/Reporting/Release/netstandard2.0/Reporting.dll
[2024/02/09 19:08:23][INFO]   BenchmarkDotNet.Extensions -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/BenchmarkDotNet.Extensions/Release/netstandard2.0/BenchmarkDotNet.Extensions.dll
[2024/02/09 19:08:23][INFO]   MicroBenchmarks -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/MicroBenchmarks/Release/net9.0/MicroBenchmarks.dll
[2024/02/09 19:08:23][INFO]   BenchmarkDotNet.Autogenerated -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-SCUWQJ/bin/net9.0/browser-wasm/Job-SCUWQJ.dll
[2024/02/09 19:08:23][INFO]   Optimizing assemblies for size may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink
[2024/02/09 19:08:23][INFO]   BenchmarkDotNet.Autogenerated -> /home/helixbot/work/B1A209D8/w/B46D0989/e/performance/artifacts/bin/for-running/MicroBenchmarks/Job-SCUWQJ/bin/net9.0/browser-wasm/publish/
[2024/02/09 19:08:25][INFO] Stack overflow.
[2024/02/09 19:08:25][INFO]    at System.RuntimeType.GetPropertyCandidates(System.String, System.Reflection.BindingFlags, System.Type[], Boolean)
[2024/02/09 19:08:25][INFO]    at System.RuntimeType.GetPropertyImpl(System.String, System.Reflection.BindingFlags, System.Reflection.Binder, System.Type, System.Type[], System.Reflection.ParameterModifier[])
[2024/02/09 19:08:25][INFO]    at System.Type.GetProperty(System.String, System.Reflection.BindingFlags)
[2024/02/09 19:08:25][INFO]    at PInvokeTableGenerator.IsFunctionPointer(System.Type)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:25][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
... (truncated (A lot taken out))
[2024/02/09 19:08:27][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:27][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:27][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:27][INFO]    at SignatureMapper.TypeToChar(System.Type, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:27][INFO]    at SignatureMapper.MethodToSignature(System.Reflection.MethodInfo, WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:27][INFO]    at IcallTableGenerator.<ProcessType>g__AddSignature|17_1(System.Type, System.Reflection.MethodInfo)
[2024/02/09 19:08:27][INFO]    at IcallTableGenerator.ProcessType(System.Type)
[2024/02/09 19:08:27][INFO]    at IcallTableGenerator.ScanAssembly(System.Reflection.Assembly)
[2024/02/09 19:08:27][INFO]    at Microsoft.WebAssembly.Build.Tasks.ManagedToNativeGenerator.ExecuteInternal(WasmAppBuilder.LogAdapter)
[2024/02/09 19:08:27][INFO]    at Microsoft.WebAssembly.Build.Tasks.ManagedToNativeGenerator.Execute()
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TaskBuilder+<ExecuteInstantiatedTask>d__25.MoveNext()
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.Build.BackEnd.TaskBuilder+<ExecuteInstantiatedTask>d__25, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]](<ExecuteInstantiatedTask>d__25 ByRef)
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TaskBuilder+<InitializeAndExecuteTask>d__23.MoveNext()
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.Build.BackEnd.TaskBuilder+<InitializeAndExecuteTask>d__23, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]](<InitializeAndExecuteTask>d__23 ByRef)
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TaskBuilder.InitializeAndExecuteTask(Microsoft.Build.BackEnd.Logging.TaskLoggingContext, Microsoft.Build.BackEnd.ItemBucket, System.Collections.Generic.IDictionary`2<System.String,System.String>, Microsoft.Build.BackEnd.TaskHost, Microsoft.Build.BackEnd.TaskExecutionMode)
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TaskBuilder+<ExecuteBucket>d__19.MoveNext()
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.Build.BackEnd.TaskBuilder+<ExecuteBucket>d__19, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]](<ExecuteBucket>d__19 ByRef)
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TaskBuilder+<ExecuteTask>d__18.MoveNext()
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.Build.BackEnd.TaskBuilder+<ExecuteTask>d__18, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]](<ExecuteTask>d__18 ByRef)
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TaskBuilder+<ExecuteTask>d__13.MoveNext()
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.Build.BackEnd.TaskBuilder+<ExecuteTask>d__13, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]](<ExecuteTask>d__13 ByRef)
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TargetEntry+<ProcessBucket>d__51.MoveNext()
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.Build.BackEnd.TargetEntry+<ProcessBucket>d__51, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]](<ProcessBucket>d__51 ByRef)
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TargetEntry+<ExecuteTarget>d__44.MoveNext()
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.Build.BackEnd.TargetEntry+<ExecuteTarget>d__44, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]](<ExecuteTarget>d__44 ByRef)
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TargetEntry.ExecuteTarget(Microsoft.Build.BackEnd.ITaskBuilder, Microsoft.Build.BackEnd.BuildRequestEntry, Microsoft.Build.BackEnd.Logging.ProjectLoggingContext, System.Threading.CancellationToken)
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.TargetBuilder+<ProcessTargetStack>d__23.MoveNext()
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.Build.BackEnd.TargetBuilder+<ProcessTargetStack>d__23, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].ExecutionContextCallback(System.Object)
[2024/02/09 19:08:27][INFO]    at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.Build.BackEnd.TargetBuilder+<ProcessTargetStack>d__23, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext(System.Threading.Thread)
[2024/02/09 19:08:27][INFO]    at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.Build.BackEnd.TargetBuilder+<ProcessTargetStack>d__23, Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
[2024/02/09 19:08:27][INFO]    at System.Threading.Tasks.TaskSchedulerAwaitTaskContinuation+<>c.<Run>b__2_0(System.Object)
[2024/02/09 19:08:27][INFO]    at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
[2024/02/09 19:08:27][INFO]    at System.Threading.Tasks.Task.ExecuteEntry()
[2024/02/09 19:08:27][INFO]    at Microsoft.Build.BackEnd.RequestBuilder+DedicatedThreadsTaskScheduler.<InjectThread>b__6_0()
[2024/02/09 19:08:27][INFO]    at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
[2024/02/09 19:08:30][INFO] // command took 31.21 sec and exited with 134

Regression?

Last known working runtime githash was a79c62d and first broken githash was ed9f475. Compare link: a79c62d...ed9f475

Known Workarounds

No response

Configuration

jsvu: Found specific V8 version: v12.0.267.

[2024/02/09 19:07:34][INFO] $ dotnet --info
[2024/02/09 19:07:35][INFO] .NET SDK:
[2024/02/09 19:07:35][INFO]  Version:           9.0.100-alpha.1.24070.3
[2024/02/09 19:07:35][INFO]  Commit:            f91d4ca399
[2024/02/09 19:07:35][INFO]  Workload version:  9.0.100-manifests.c9b9580e
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO] Runtime Environment:
[2024/02/09 19:07:35][INFO]  OS Name:     ubuntu
[2024/02/09 19:07:35][INFO]  OS Version:  22.04
[2024/02/09 19:07:35][INFO]  OS Platform: Linux
[2024/02/09 19:07:35][INFO]  RID:         linux-x64
[2024/02/09 19:07:35][INFO]  Base Path:   /home/helixbot/work/B1A209D8/p/dotnet/sdk/9.0.100-alpha.1.24070.3/
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO] .NET workloads installed:
[2024/02/09 19:07:35][INFO]  [wasm-tools]
[2024/02/09 19:07:35][INFO]    Installation Source: SDK 9.0.100-alpha.1
[2024/02/09 19:07:35][INFO]    Manifest Version:    9.0.0-ci/9.0.100-alpha.1
[2024/02/09 19:07:35][INFO]    Manifest Path:       /home/helixbot/work/B1A209D8/p/dotnet/sdk-manifests/9.0.100-alpha.1/microsoft.net.workload.mono.toolchain.current/WorkloadManifest.json
[2024/02/09 19:07:35][INFO]    Install Type:        FileBased
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO]  [wasm-tools-net8]
[2024/02/09 19:07:35][INFO]    Installation Source: SDK 9.0.100-alpha.1
[2024/02/09 19:07:35][INFO]    Manifest Version:    9.0.0-ci/9.0.100-alpha.1
[2024/02/09 19:07:35][INFO]    Manifest Path:       /home/helixbot/work/B1A209D8/p/dotnet/sdk-manifests/9.0.100-alpha.1/microsoft.net.workload.mono.toolchain.net8/WorkloadManifest.json
[2024/02/09 19:07:35][INFO]    Install Type:        FileBased
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO]  [wasm-experimental-net8]
[2024/02/09 19:07:35][INFO]    Installation Source: SDK 9.0.100-alpha.1
[2024/02/09 19:07:35][INFO]    Manifest Version:    9.0.0-ci/9.0.100-alpha.1
[2024/02/09 19:07:35][INFO]    Manifest Path:       /home/helixbot/work/B1A209D8/p/dotnet/sdk-manifests/9.0.100-alpha.1/microsoft.net.workload.mono.toolchain.net8/WorkloadManifest.json
[2024/02/09 19:07:35][INFO]    Install Type:        FileBased
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO]  [wasm-experimental]
[2024/02/09 19:07:35][INFO]    Installation Source: SDK 9.0.100-alpha.1
[2024/02/09 19:07:35][INFO]    Manifest Version:    9.0.0-ci/9.0.100-alpha.1
[2024/02/09 19:07:35][INFO]    Manifest Path:       /home/helixbot/work/B1A209D8/p/dotnet/sdk-manifests/9.0.100-alpha.1/microsoft.net.workload.mono.toolchain.current/WorkloadManifest.json
[2024/02/09 19:07:35][INFO]    Install Type:        FileBased
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO] Host:
[2024/02/09 19:07:35][INFO]   Version:      9.0.0-alpha.1.24066.33
[2024/02/09 19:07:35][INFO]   Architecture: x64
[2024/02/09 19:07:35][INFO]   Commit:       dbb335c6ba
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO] .NET SDKs installed:
[2024/02/09 19:07:35][INFO]   9.0.100-alpha.1.24070.3 [/home/helixbot/work/B1A209D8/p/dotnet/sdk]
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO] .NET runtimes installed:
[2024/02/09 19:07:35][INFO]   Microsoft.AspNetCore.App 9.0.0-alpha.1.24068.16 [/home/helixbot/work/B1A209D8/p/dotnet/shared/Microsoft.AspNetCore.App]
[2024/02/09 19:07:35][INFO]   Microsoft.NETCore.App 8.0.2 [/home/helixbot/work/B1A209D8/p/dotnet/shared/Microsoft.NETCore.App]
[2024/02/09 19:07:35][INFO]   Microsoft.NETCore.App 9.0.0-alpha.1.24066.33 [/home/helixbot/work/B1A209D8/p/dotnet/shared/Microsoft.NETCore.App]
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO] Other architectures found:
[2024/02/09 19:07:35][INFO]   None
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO] Environment variables:
[2024/02/09 19:07:35][INFO]   DOTNET_ROOT       [/home/helixbot/work/B1A209D8/p/dotnet/]
[2024/02/09 19:07:35][INFO] 
[2024/02/09 19:07:35][INFO] global.json file:
[2024/02/09 19:07:35][INFO]   Not found

Other information

Compare with commits that were merged between working and failing build: a79c62d...ed9f475

Activity

  1. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Feb 13, 2024
  2. ghost added
    area-Infrastructure-mono
    on Feb 13, 2024

  4. vargaz commented on Feb 13, 2024

    @vargaz
    vargaz
    on Feb 13, 2024
    Contributor

    @kg

  5. kg
    self-assigned this
    on Feb 13, 2024

  6. LoopedBard3 commented on Feb 13, 2024

    @LoopedBard3
    LoopedBard3
    on Feb 13, 2024
    MemberAuthor

    Added an alternative set of instructions to maybe help get a repro using the performance repos benchmarks_local.py to install dotnet, clone the correct runtime, build wasm, and try to run the BDN tests. I was able to replicate the error using this once some fixes were added (they are part of my branch that I linked in the steps).

  7. kg commented on Feb 13, 2024

    @kg
    kg
    on Feb 13, 2024
    Contributor

    We just changed the pinvoke table generator, so I'm certain that broke it. Will look into it once I can repro.

  8. kg commented on Feb 13, 2024

    @kg
    kg
    on Feb 13, 2024
    Contributor

    Your repro steps make it impossible for me to use a local build of the runtime, it seems like? Is there a way for me to adapt them to work like this old incantation that used a runtime from my local checkout:

    python3 ./scripts/benchmarks_ci.py -f net9.0 --dotnet-path /home/kate/Projects/dotnet-runtime-wasm/artifacts/bin/dotnet-latest --wasm --run-isolated --bdn-artifacts artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --logBuildOutput --wasmDataDir /home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm --wasmArgs \" --expose-wasm --module \" --filter *Span.IndexerBench.CoveredIndex2*"

    This worked the last time I had to iterate on the benchmarks.

  9. LoopedBard3 commented on Feb 13, 2024

    @LoopedBard3
    LoopedBard3
    on Feb 13, 2024
    MemberAuthor

    Please see comment below as I initially wasn't aware wasm could be directly used with the benchmarks_ci.py script.

    If you already have the artifacts built you can definitely just use the benchmarks_ci.py command instead. The benchmarks_local.py mostly acts as a wrapper around benchmarks_ci.py and builds the runtime. It is also possible to have benchmarks_local.py build a local repo if you want to skip manually doing the build steps between local changes. benchmarks_local.py should also print out the benchmarks_ci.py command for a given run on a line starting with "Running single benchmarks_ci.py for RunType.WasmInterpreter ...".

    If you build the wasm manually, the command line you put should work. The command that can be used from the /eng dir is build.sh -subset mono+libs -configuration Release -os browser -arch wasm -bl /p:AotHostArchitecture=x64 /p:AotHostOS=linux and the paths should be close to the command line you posted.

    If you want to try benchmarks_local.py with a local repo of the runtime, the command line would look something like:
    sudo python3 benchmarks_local.py --local-test-repo "<absolute path to runtime folder>/runtime" --run-types WasmInterpreter --filter *Span.IndexerBench.CoveredIndex2* *WriteReadAsync* --wasm-engine-path /home/<user>/.jsvu/engines/v8/v8. Essentially replace the --commits and --repo-storage-path arguments with --local-test-repo <path>/runtime. This should rebuild and test the runtime repo specified each invocation, but let me know if it does not seem to be doing that. Do note that each new run will overwrite the artifacts from the previous runtime repo build, they ca nbe saved by renaming the folder the build artifacts are saved to: ./runtime-testing-artifacts/WasmInterpreter-local-linux-x64

  10. LoopedBard3 commented on Feb 13, 2024

    @LoopedBard3
    LoopedBard3
    on Feb 13, 2024
    MemberAuthor

    It also looks like you may be able to call the command line you have directly, though you will have to update the --wasmDataDir to point to /src/mono/browser instead of /src/mono/wasm. I wasn't aware that for wasm runs you could bypass having to build runtime like that.

  11. kg commented on Feb 14, 2024

    @kg
    kg
    on Feb 14, 2024
    Contributor

    Build is failing, any idea why?

    [2024/02/13 16:57:17][INFO] // Found 1 benchmarks:
[2024/02/13 16:57:17][INFO] //   IndexerBench.CoveredIndex2: Job-NSPANJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [length=1024]
[2024/02/13 16:57:17][INFO] 
[2024/02/13 16:57:17][INFO] // Build Error: Standard output:
[2024/02/13 16:57:17][INFO] 
[2024/02/13 16:57:17][INFO]  Standard error:
[2024/02/13 16:57:17][INFO]    Determining projects to restore...
[2024/02/13 16:57:17][INFO]   Restored /home/kate/Projects/performance/src/benchmarks/micro/MicroBenchmarks.csproj (in 396 ms).
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102: Unable to find package Microsoft.NETCore.App.Ref with version (= 9.0.0-dev)
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 2073 version(s) in dotnet6 [ Nearest version: 6.0.2-servicing.1.22101.5 ]
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 1562 version(s) in dotnet7 [ Nearest version: 7.0.0-rtm.22511.4 ]
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 1458 version(s) in dotnet8 [ Nearest version: 8.0.0-rtm.23523.3 ]
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 704 version(s) in dotnet9 [ Nearest version: 9.0.0-preview.1.24072.8 ]
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 101 version(s) in dotnet-public [ Nearest version: 9.0.0-preview.1.24080.9 ]
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 76 version(s) in dotnet3.1 [ Nearest version: 3.1.1-servicing.19602.11 ]
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 1 version(s) in dotnet-eng [ Nearest version: 5.0.0-alpha.1.19618.1 ]
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 0 version(s) in benchmark-dotnet-prerelease
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 0 version(s) in dotnet-tools
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 0 version(s) in dotnet3.1-transport
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 0 version(s) in dotnet8-transport
[2024/02/13 16:57:17][INFO] /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj : error NU1102:   - Found 0 version(s) in dotnet9-transport
[2024/02/13 16:57:17][INFO]   Failed to restore /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/0591af58-7ba0-4040-8a67-2cb83f0b0715/BenchmarkDotNet.Autogenerated.csproj (in 1.3 sec).
[2024/02/13 16:57:17][INFO]   2 of 4 projects are up-to-date for restore.
[2024/02/13 16:57:17][INFO] 
[2024/02/13 16:57:17][INFO] // BenchmarkDotNet has failed to build the auto-generated boilerplate code.

  12. LoopedBard3 commented on Feb 14, 2024

    @LoopedBard3
    LoopedBard3
    on Feb 14, 2024
    MemberAuthor

    Can you try running again now that it has been some time? We occasionally hit errors similar to this when using a dotnet sdk that tries to pull in things not published yet. I was able to successfully build with sdk version: 9.0.100-preview.2.24113.9 from https://github.com/dotnet/installer?tab=readme-ov-file#use-dotnet---help-to-see-available-commands-or-visit-httpsakamsdotnet-cli which I downloaded yesterday.

  13. kg commented on Feb 17, 2024

    @kg
    kg
    on Feb 17, 2024
    Contributor

    I was able to manually build the autogenerated benchmarks csproj by doing this:

    pushd /home/kate/Projects/performance/artifacts/bin/for-running/MicroBenchmarks/66f85c28-d283-47bf-b618-33e97a9f61c3/
/home/kate/Projects/dotnet-runtime-wasm/artifacts/bin/dotnet-latest/dotnet build

    It's still not clear to me why I can't build using the normal ci script, it's still failing to find NETCore.App.Ref 9.0.0-dev when the script tries to build.

    Using the current rev of #98250 it doesn't hang, but the build fails due to a pinvoke-related issue. I will try to fix that so that once the PR lands, it should be possible to build benchmarks again. I'm still not sure why it broke.

  14. kg commented on Feb 21, 2024

    @kg
    kg
    on Feb 21, 2024
    Contributor

    #98250 should fix this now that it's landed in main. if it doesn't, let me know and I'll dig in immediately.

  15. LoopedBard3 commented on Feb 21, 2024

    @LoopedBard3
    LoopedBard3
    on Feb 21, 2024
    MemberAuthor

    Hm... we now seem to be hitting: ./p/dotnet/packs/Microsoft.NET.Runtime.WebAssembly.Sdk/9.0.0-ci/Sdk/WasmApp.Common.targets(350,5): error : WasmAssembliesToBundle item is empty. No assemblies to process [./performance/artifacts/bin/for-running/MicroBenchmarks/Job-KDLVIT/BenchmarkDotNet.Autogenerated.csproj]. Although it is not clear if this is after this change or if this started popping up earlier and we missed the new error. Let me see if I can determine where this error is coming from.

  16. LoopedBard3 commented on Feb 21, 2024

    @LoopedBard3
    LoopedBard3
    on Feb 21, 2024
    MemberAuthor

    I have been unable to repro this locally with the local tests running successfully. This means that this issue is not related to this change and that the fix for the issue seems to have worked 👍. I will do some more digging on this new failure and open a new issue if appropriate.

  17. LoopedBard3 commented on Mar 4, 2024

    @LoopedBard3
    LoopedBard3
    on Mar 4, 2024
    MemberAuthor

    This seems to have been properly fixed as outlined in dotnet/performance#3984.

  18. LoopedBard3
    closed this as completedon Mar 4, 2024
  19. ghost removed
    untriagedNew issue has not been triaged by the area owner
    on Mar 4, 2024
  20. github-actions
    locked and limited conversation to collaborators on Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-Infrastructure-mono

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions