Parameterised Tests

Additions to file name

Every parameterised case should have a unique file name with the parameters appended to the file name. This happens automatically for NUnit; xUnit and MSTest require the use of UseParameters() (see below).

The appending format is _ParamName=ParamValue repeated for each parameter.

A test with two parameters param1 + param2, and called twice with the values value1a+ value2a and value1b+ value2b would have the following file names:

MyTest.MyMethod_param1=value1a_param2=value2a.verified.txt
MyTest.MyMethod_param1=value1b_param2=value2b.verified.txt

Invalid characters

Characters that cannot be used for a file name are replaced with a dash (-).

UseParameters()

Verify.NUnit and Verify.Fixie automatically detect the method parameters. So UseParameters() is not required unless using custom parameters.

UseParameters() is used to control what parameters are used when naming files. The usual usage is to pass though all parameters (in the same order) that the test method accepts:

[Theory]
[InlineData("Value1")]
[InlineData("Value2")]
public Task UseParametersUsage(string arg)
{
    var somethingToVerify = $"{arg} some text";
    return Verify(somethingToVerify)
        .UseParameters(arg);
}

^{snippet source | anchor}

If not all parameters are required, a subset can be passed in. In this scenario, the parameters passed in will match with the method parameter names from the start. For example the following will result in a file named ParametersSample.UseParametersSubSet_arg1=Value1_arg2=Value2.verified.txt

[Theory]
[InlineData("Value1", "Value2", "Value3")]
public Task UseParametersSubSet(string arg1, string arg2, string arg3)
{
    var somethingToVerify = $"{arg1} {arg2} {arg3} some text";
    return Verify(somethingToVerify)
        .UseParameters(arg1, arg2);
}

^{snippet source | anchor}

If the number of parameters passed to UseParameters() is greater than the number of parameters in the test method, an exception will be thrown.

NUnit

TestCase

[TestCase("Value1")]
[TestCase("Value2")]
public Task TestCaseUsage(string arg) =>
    Verify(arg);

^{snippet source | anchor}

TestFixtureSourceUsage

When using a TestFixtureSource the the name provided by NUnit will be as the TestMethodName.

[TestFixtureSource(nameof(FixtureArgs))]
public class TestFixtureSourceUsage(string arg1, int arg2)
{
    [Test]
    public Task Test() =>
        Verify(
            new
            {
                arg1,
                arg2
            });

    static object[] FixtureArgs =
    [
        new object[]
        {
            "Value1",
            1
        },
        new object[]
        {
            "Value2",
            2
        }
    ];
}

^{snippet source | anchor}

Produces TestFixtureSourceUsage(Value1,1).Test.verified.txt and TestFixtureSourceUsage(Value2,2).Test.verified.txt.

xUnit

InlineData

[Theory]
[InlineData("Value1")]
[InlineData("Value2")]
public Task InlineDataUsage(string arg)
{
    var settings = new VerifySettings();
    settings.UseParameters(arg);
    return Verify(arg, settings);
}

[Theory]
[InlineData("Value1")]
[InlineData("Value2")]
public Task InlineDataUsageFluent(string arg) =>
    Verify(arg)
        .UseParameters(arg);

^{snippet source | anchor}

MemberData

[Theory]
[MemberData(nameof(GetData))]
public Task MemberDataUsage(string arg)
{
    var settings = new VerifySettings();
    settings.UseParameters(arg);
    return Verify(arg, settings);
}

[Theory]
[MemberData(nameof(GetData))]
public Task MemberDataUsageFluent(string arg) =>
    Verify(arg)
        .UseParameters(arg);

public static IEnumerable<object[]> GetData()
{
    yield return
    [
        "Value1"
    ];
    yield return
    [
        "Value2"
    ];
}

^{snippet source | anchor}

Complex MemberData

xUnit only exposes parameter information for certain types. For unknown types the information cannot be retrieved from the xUnit context, and instead the text for the parameter value needs to be explicitly specified. This is done by calling NameForParameter().

public class ComplexParametersSample
{
    [ModuleInitializer]
    public static void Initialize()
    {
        VerifierSettings.NameForParameter<ComplexData>(_ => _.Value);
        VerifierSettings.NameForParameter<ComplexStructData>(_ => _.Value);
    }

    [Theory]
    [MemberData(nameof(GetComplexMemberData))]
    public Task ComplexMemberData(ComplexData arg)
    {
        var settings = new VerifySettings();
        settings.UseParameters(arg);
        return Verify(arg, settings);
    }

    [Theory]
    [MemberData(nameof(GetComplexMemberData))]
    public Task ComplexMemberDataFluent(ComplexData arg) =>
        Verify(arg)
            .UseParameters(arg);

    [Theory]
    [MemberData(nameof(GetComplexMemberData))]
    public Task ComplexMemberNullableData(ComplexData arg)
    {
        var settings = new VerifySettings();
        settings.UseParameters(arg);
        return Verify(arg, settings);
    }

    [Theory]
    [MemberData(nameof(GetComplexMemberData))]
    public Task ComplexMemberNullableDataFluent(ComplexData arg) =>
        Verify(arg)
            .UseParameters(arg);

    public static IEnumerable<object[]> GetComplexMemberData()
    {
        yield return
        [
            new ComplexData
            {
                Value = "Value1"
            }
        ];
        yield return
        [
            new ComplexData
            {
                Value = "Value2"
            }
        ];
    }

    public class ComplexData
    {
        public string Value { get; set; } = null!;
    }

    [Theory]
    [MemberData(nameof(GetComplexMemberStructData))]
    public Task ComplexMemberStructData(ComplexStructData arg)
    {
        var settings = new VerifySettings();
        settings.UseParameters(arg);
        return Verify(arg, settings);
    }

    [Theory]
    [MemberData(nameof(GetComplexMemberStructData))]
    public Task ComplexMemberStructDataFluent(ComplexStructData arg) =>
        Verify(arg)
            .UseParameters(arg);

    [Theory]
    [MemberData(nameof(GetComplexMemberStructData))]
    public Task ComplexMemberNullableStructData(ComplexStructData arg)
    {
        var settings = new VerifySettings();
        settings.UseParameters(arg);
        return Verify(arg, settings);
    }

    [Theory]
    [MemberData(nameof(GetComplexMemberStructData))]
    public Task ComplexMemberNullableStructDataFluent(ComplexStructData arg) =>
        Verify(arg)
            .UseParameters(arg);

    public static IEnumerable<object[]> GetComplexMemberStructData()
    {
        yield return
        [
            new ComplexStructData("Value1")
        ];
        yield return
        [
            new ComplexStructData("Value2")
        ];
    }

    public struct ComplexStructData
    {
        public ComplexStructData(string value) =>
            Value = value;

        public string Value { get; set; } = null!;
    }
}

^{snippet source | anchor}

VerifierSettings.NameForParameter() is required since the parameter type has no ToString() override that can be used for deriving the name of the .verified. file.

Fixie

Fixie has no build in test parameterisation. Test parameterisation need to be implemented by the consuming library. See Attribute-Based Parameterization for an example.

Verify.Fixie requires some customisation of the above example.

Inside ITestProject.Configure call VerifierSettings.AssignTargetAssembly(environment.Assembly);
Inside IExecution.Run wrap test.Run in using (ExecutionState.Set(testClass, test, parameters))

Example implementation:

public class TestProject :
    ITestProject,
    IExecution
{
    public void Configure(TestConfiguration configuration, TestEnvironment environment)
    {
        VerifierSettings.AssignTargetAssembly(environment.Assembly);
        configuration.Conventions.Add<DefaultDiscovery, TestProject>();
    }

    public async Task Run(TestSuite testSuite)
    {
        foreach (var testClass in testSuite.TestClasses)
        {
            foreach (var test in testClass.Tests)
            {
                if (test.HasParameters)
                {
                    foreach (var parameters in test
                                 .GetAll<TestCase>()
                                 .Select(_ => _.Parameters))
                    {
                        using (ExecutionState.Set(testClass, test, parameters))
                        {
                            await test.Run(parameters);
                        }
                    }
                }
                else
                {
                    using (ExecutionState.Set(testClass, test, null))
                    {
                        await test.Run();
                    }
                }
            }
        }
    }
}

^{snippet source | anchor}

Resulting usage:

[TestCase("Value1")]
[TestCase("Value2")]
public Task TestCaseUsage(string arg) =>
    Verify(arg);

^{snippet source | anchor}

MSTest

DataRow

[TestClass]
public partial class ParametersSample
{
    [DataTestMethod]
    [DataRow("Value1")]
    [DataRow("Value2")]
    public Task DataRowUsage(string arg)
    {
        var settings = new VerifySettings();
        settings.UseParameters(arg);
        return Verify(arg, settings);
    }

    [DataTestMethod]
    [DataRow("Value1")]
    [DataRow("Value2")]
    public Task DataRowUsageFluent(string arg) =>
        Verify(arg)
            .UseParameters(arg);
}

^{snippet source | anchor}

Overriding text used for parameters

UseTextForParameters() can be used to override the substitution text used for {Parameters}.

{Directory}/{TestClassName}.{TestMethodName}_{Parameters}_{UniqueFor1}_{UniqueFor2}_{UniqueForX}.verified.{extension}

[Theory]
[InlineData("Value1")]
[InlineData("Value2")]
public Task UseTextForParameters(string arg)
{
    var settings = new VerifySettings();
    settings.UseTextForParameters(arg);
    return Verify(arg + "UseTextForParameters", settings);
}

[Theory]
[InlineData("Value1")]
[InlineData("Value2")]
public Task UseTextForParametersFluent(string arg) =>
    Verify(arg + "UseTextForParametersFluent")
        .UseTextForParameters(arg);

^{snippet source | anchor}

Results in:

TheTest.UseTextForParameters_Value1.verified.txt
TheTest.UseTextForParameters_Value2.verified.txt
TheTest.UseTextForParametersFluent_Value1.verified.txt
TheTest.UseTextForParametersFluent_Value2.verified.txt

Ignore parameters for verified filename

By default, Verify expects every parameterised case to have a unique file name with the parameters appended to the file name. This behavior can be overridden by using IgnoreParametersForVerified(). In this case, the verified file name does not contain the parameter values, meaning it is the same for each testcase.

[Theory]
[InlineData("One")]
[InlineData("Two")]
public Task IgnoreParametersForVerifiedWithArgs(string arg)
{
    var settings = new VerifySettings();
    settings.IgnoreParametersForVerified(arg);
    return Verify("valueIgnoreParametersForVerifiedWithArgs", settings);
}

[Theory]
[InlineData("One")]
[InlineData("Two")]
public Task IgnoreParametersForVerifiedFluent(string arg) =>
    Verify("valueIgnoreParametersForVerifiedFluent")
        .IgnoreParametersForVerified(arg);

^{snippet source | anchor}

Results in:

NamerTests.IgnoreParametersForVerified_arg=One.received.txt
NamerTests.IgnoreParametersForVerified_arg=Two.received.txt
NamerTests.IgnoreParametersForVerified.verified.txt

And for the second test:

NamerTests.IgnoreParametersForVerifiedFluent_arg=One.received.txt
NamerTests.IgnoreParametersForVerifiedFluent_arg=Two.received.txt
NamerTests.IgnoreParametersForVerifiedFluent.verified.txt

Hashing parameters

Parameters can be hashed as an alternative to being stringified. This is useful when the parameters are large and could potentially generate file names that exceed allowances of the OS.

Hashing parameter is achieved by using UseParameters in combination with HashParameters. Alternatively UseHashedParameters can be used as a wrapper for those two method calls.

Overriding text used for parameters is respected when generating the hash.

XxHash64 is used to perform the hash.

MSTest

[TestClass]
public partial class ParametersHashSample
{
    [DataTestMethod]
    [DataRow("Value1")]
    [DataRow("Value2")]
    public Task UseHashedParametersUsage(string arg)
    {
        var settings = new VerifySettings();
        settings.UseHashedParameters(arg);
        return Verify(arg, settings);
    }

    [DataTestMethod]
    [DataRow("Value1")]
    [DataRow("Value2")]
    public Task UseHashedParametersUsageFluent(string arg) =>
        Verify(arg)
            .UseHashedParameters(arg);

    [DataTestMethod]
    [DataRow("Value1")]
    [DataRow("Value2")]
    public Task HashParametersUsage(string arg)
    {
        var settings = new VerifySettings();
        settings.UseParameters(arg);
        settings.HashParameters();
        return Verify(arg, settings);
    }

    [DataTestMethod]
    [DataRow("Value1")]
    [DataRow("Value2")]
    public Task HashParametersUsageFluent(string arg) =>
        Verify(arg)
            .UseParameters(arg)
            .HashParameters();
}

^{snippet source | anchor}

NUnit

[TestFixture]
public class ParametersHashSample
{
    [TestCase("Value1")]
    [TestCase("Value2")]
    public Task UseHashedParametersUsage(string arg)
    {
        var settings = new VerifySettings();
        settings.UseHashedParameters(arg);
        return Verify(arg, settings);
    }

    [TestCase("Value1")]
    [TestCase("Value2")]
    public Task UseHashedParametersUsageFluent(string arg) =>
        Verify(arg)
            .UseHashedParameters(arg);

    [TestCase("Value1")]
    [TestCase("Value2")]
    public Task HashParametersUsage(string arg)
    {
        var settings = new VerifySettings();
        settings.UseParameters(arg);
        settings.HashParameters();
        return Verify(arg, settings);
    }

    [TestCase("Value1")]
    [TestCase("Value2")]
    public Task HashParametersUsageFluent(string arg) =>
        Verify(arg)
            .HashParameters();

    [TestCase("Value1")]
    [TestCase("Value2")]
    public Task HashParametersOmitPassingParameters(string arg)
    {
        var settings = new VerifySettings();
        settings.HashParameters();
        return Verify(arg, settings);
    }

    [TestCase("Value1")]
    [TestCase("Value2")]
    public Task HashParametersOmitPassingParametersFluent(string arg) =>
        Verify(arg)
            .HashParameters();
}

^{snippet source | anchor}

Note that NUnit can derive the parameters without explicitly passing them.

xUnit

public class ParametersHashSample
{
    [Theory]
    [InlineData("Value1")]
    [InlineData("Value2")]
    public Task UseHashedParametersUsage(string arg)
    {
        var settings = new VerifySettings();
        settings.UseHashedParameters(arg);
        return Verify(arg, settings);
    }

    [Theory]
    [InlineData("Value1")]
    [InlineData("Value2")]
    public Task UseHashedParametersUsageFluent(string arg) =>
        Verify(arg)
            .UseHashedParameters(arg);

    [Theory]
    [InlineData("Value1")]
    [InlineData("Value2")]
    public Task HashParametersUsage(string arg)
    {
        var settings = new VerifySettings();
        settings.UseParameters(arg);
        settings.HashParameters();
        return Verify(arg, settings);
    }

    [Theory]
    [InlineData("Value1")]
    [InlineData("Value2")]
    public Task HashParametersUsageFluent(string arg) =>
        Verify(arg)
            .UseParameters(arg)
            .HashParameters();
}

^{snippet source | anchor}

Globally

public static class ModuleInitializer
{
    [ModuleInitializer]
    public static void Init() =>
        VerifierSettings.HashParameters();
}

^{snippet source | anchor}