Skip to content

buildtest: aggressive parallel algorithm causes OOM when benchmark tests run #679

@dnyw4l3n13

Description

@dnyw4l3n13

Issue

The buildtest script uses --parallel-algorithm aggressive when running dotnet test. When the repository contains benchmark tests (e.g. FunFair.Ethereum.DataTypes.BenchMark.Tests) that take 3-4 minutes to complete, the aggressive parallelism causes OutOfMemoryException crashes in other test assemblies running concurrently.

Error

Out of memory.
Aborted (core dumped)

✗ ######## Tests Failed ########

Stack trace (from first failure in FunFair.Ethereum.Client.Tests):

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Threading.Tasks.Task.AssignCancellationToken
   at Microsoft.DotNet.Cli.Commands.Test.IPC.NamedPipeServer.WaitConnectionAsync
   at Microsoft.DotNet.Cli.Commands.Test.TestApplication.WaitConnectionAsync

Context

  • Source repository: funfair-tech/funfair-server-ethereum
  • The DataTypes.BenchMark.Tests take ~4 minutes and consume significant memory
  • With --parallel-algorithm aggressive, all test assemblies (including benchmarks) start at the same time
  • This exhausts available memory, causing random OOM crashes in other assemblies depending on timing

Observed behaviour

The failure is non-deterministic — different test assemblies OOM on different runs:

  1. First run: FunFair.Ethereum.Client.Tests OOMs
  2. Second run: FunFair.Ethereum.Blocks.Stream.Tests OOMs

Proposed fix

Either:

  1. Exclude benchmark test assemblies from the pre-commit test run (they are not appropriate for every commit anyway), or
  2. Use a less aggressive parallel algorithm (e.g. --parallel-algorithm sequential or no parallel flag) when benchmark tests are present, or
  3. Add a flag to skip benchmarks: only run *.Tests assemblies, not *.BenchMark.Tests

Label: AI-Work

Metadata

Metadata

Assignees

No one assigned

    Labels

    AI-WorkWork for an AI Agent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions