nunit-console | NUnit Console runner and test engine | Unit Testing library

That loader loads nunit projects. You want the VSProjectLoader.

Install with

The plugin set the result to UNSTABLE because the option, by default, failedTestsFailBuild is set to false.

You can control the behavior applies of NUnit, setting failedTestsFailBuild to true. When you call from a scripted or declarative pipeline.

The issue is the GUI doesn't reflect all the options available for this plugin. There is a PR opened to include this option inside the freestyle pipeline, you can vote up or ask the status of this PR.

To change to an error you need to catch the unstable result and set it to failure using a plugin or a scripted or declarative pipeline.

By default, the NUnit Console will run multiple test assemblies in parallel. Add --agents=1 to force the two assemblies to run sequentially, under a single agent.

Just to clarify some of the other things you tried as well...

[NonParallelizable] is used to prevent the parallelization of different tests within a single assembly. By default, tests within an assembly do not run in parallel, so adding this attribute when you haven't specifically added [Parallelizable] at a higher level will have no effect.

[Apartments(Apartment.STA)] can be added as an assembly-level attribute, and does not have to be added per test, as mentioned in the comments. Check out the docs here: https://docs.nunit.org/articles/nunit/writing-tests/attributes/apartment.html

You can give it a try to rewrite your test to use TaskCompletionSource with CancellationToken.

If you don't need the EventArgs then the rewritten version of the test would look like this:

I tried to run tests using latest dotMemory Unit 3.1, it works

dotMemoryUnit.exe "c:\Program Files\dotnet\dotnet.exe" -- test "path\to\the\solution.sln"

For your version it would be

dotMemoryUnit.exe -targetExecutable="c:\Program Files\dotnet\dotnet.exe" -returnTargetExitCode -- test "E:\MyProject\bin\Release\MainTests.dll"

Have a look at project file in GitHub sources, which you've attached. They reference nunit.engine.api project, so you should add this as well. NUnit.Engine.Api is a Nuget package of this reference

Unfortunately, the only way to pass settings from dotnet test into NUnit is to use a .runsettings file. There's no way for NUnit to create custom command line arguments for the dotnet test tool - although we'd love there to be!

Take a look at the sample .runsettings file here. The specific bit you'll need:

The --test command-line option is used to construct a filter, which determines which tests are run. It doesn't affect order - no command-line options have to do with order. NUnit applies the created filter to the tests as it examines them, deciding one test at a time whether it should be executed.

Neither the order of the options nor the order in which NUnit examines the tests has any connection to the order in which they are executed. The execution order is determined by:

1. Any OrderAttributes you use in your tests.
2. If no such attributes are used, the order is unspecified. (*)

You can specify [Order(n)] on any fixture or method. Those items with an OrderAttribute execute first, starting with the lowest value of n. If you are running tests in parallel, the order doesn't guarantee that following tests will not start while the first test is running. It's up to you to ensure you don't run such tests in parallel.

See the docs as well: https://github.com/nunit/docs/wiki/Order-Attribute

*Note: some people use the alphabetical order of tests. Some versions of NUnit, in some environments use that ordering. It's not guaranteed by NUnit, so it's not a good idea to rely on it.

Either use a tool that can parse it or write your own.

A web search for a "NUnit Results Viewer" should give you some choices for existing tools.

You can see the format spec here: https://github.com/nunit/docs/wiki/Test-Result-XML-Format