Benchmarkdotnet: Can ParamsSource be evaluated after GlobalSetup?

[chriga]

I have to initialize several objects that are generating parameters. Therefore my idea was to initialize these objects in the GlobalSetup area.

    private IProjectInfo _generatedProject1;
    private IProjectInfo _generatedProject2;

    [GlobalSetup]
    public void BenchmarkSetup()
    {
        var loader = new ProjectLoader();
        _generatedProject1 = loader.GetSample();
        _generatedProject2 = loader.GetBigSample();
    }

    [ParamsSource(nameof(Projects))]
    public IProjectInfo Project;

    public IEnumerable<IParam> Projects()
    {
        yield return new ProjectParameter { Project = _generatedProject1};
        yield return new ProjectParameter { Project = _generatedProject2};
    }

    [Benchmark]
    public IProject OpenProject()
    {
        var project = OpenProject(Project);
        return project;
    }

    public class ProjectParameter : IParam
    {
        public IProjectInfo Project { get; set; }
        public string ToSourceCode() => "";
        public object Value => Project;
        public string DisplayText => Project.Name;
    }

But unfortunately the parameters are built before, so I get an exception. Is there any workaround?

[adamsitnik]

Parameters are generated before the call to Setup method(s), to make it possible for the setup to use their values.

You could do sth like:

private Dictionary<string, IProjectInfo> _projectsByName;

private IProjectInfo _project;

[ParamsSource(nameof(Projects))]
public string CurrentProjectName;

public IEnumerable<IParam> Projects()
{
    yield return "generatedProject1"
    yield return "generatedProject2";
}


[GlobalSetup]
public void BenchmarkSetup()
{
    var loader = new ProjectLoader();
    _projectsByName = new Dictionary<string, IProjectInfo>
    {
        { "_generatedProject1", loader.GetSample() },
        { "_generatedProject2", loader.GetBigSample() },
    }

    _currentProject = _projectsByName[CurrentProjectName];
}

[Benchmark]
public IProject OpenProject()
{
    var project = OpenProject(_currentProject);
    return project;
}

[chriga]

Hi Adam,

thanks for your input. Your idea seems to work with some modification:

private Dictionary<string, IProjectInfo> _projectsByName;

[ParamsSource(nameof(Projects))]
public string CurrentProjectName;

public IEnumerable<string> Projects()
{
    yield return "generatedProject1"
    yield return "generatedProject2";
}

[GlobalSetup]
public void BenchmarkSetup()
{
    var loader = new ProjectLoader();
    _projectsByName = new Dictionary<string, IProjectInfo>
    {
        { "generatedProject1", loader.GetSample() },
        { "generatedProject2", loader.GetBigSample() }
    }
}

[Benchmark]
public IProject OpenProject()
{
    var project = OpenProject(_projectsByName[CurrentProjectName]);
    return project;
}

Original I wanted that the project names are used from loaded projects. But I can get over it.