Scientist.NET Documentation

Scientist.NET uses two primary abstractions to facilitate safe refactoring:

• Control: The existing, proven implementation. It is defined using the experiment.Use() method. The result of the Use block is what is ultimately returned to the caller of Scientist.Science<T>.
• Candidate: The new, refactored implementation being tested. It is defined using the experiment.Try() method. Scientist monitors the candidate for performance, correctness (result matching), and exceptions, but its output is not returned to the caller.

If no Try blocks are declared within an experiment, the Scientist machinery is bypassed and the control value is returned directly.

If you only care about side effects (like timing or ensuring a new code path doesn't throw exceptions) rather than value equality, you can instruct Scientist.NET to ignore the results. This is useful when using the Enabled method to incrementally roll out code.

To disregard values entirely while still logging exceptions, use Ignore((x, y) => true) or the more efficient Compare((x, y) => true).

To use Scientist.NET in your project, install the Scientist package using the NuGet package manager.

Install-Package Scientist

While it is generally recommended to test one alternative at a time to ensure isolation and clear reporting, you can test multiple candidates by providing names to Try blocks. This allows you to compare several different implementations within a single experiment block.

public bool CanAccess(IUser user)
{
    return Scientist.Science<bool>("widget-permissions", experiment =>
    {
        experiment.Use(() => IsCollaborator(user));
        experiment.Try("api", () => HasAccess(user));
        experiment.Try("raw-sql", () => HasAccessSql(user));
    });
}

Scientist uses the == operator to compare control and candidate values by default. If you need custom comparison logic (e.g., comparing specific properties of an object), use the experiment.Compare method within the science block.

public IUser GetCurrentUser(string hash)
{
    return Scientist.Science<IUser>("get-current-user", experiment =>
    {
        experiment.Compare((x, y) => x.Name == y.Name);

        experiment.Use(() => LookupUser(hash));
        experiment.Try(() => RetrieveUser(hash));
    });
}

Use experiment.AddContext(string identifier, object value) to attach metadata to an experiment. This data is stored in an internal dictionary and is accessible via the Contexts property on the Result object during publishing.

public IUser GetUserByName(string userName)
{
    return Scientist.Science<IUser>("get-user-by-name", experiment =>
    {
        experiment.AddContext("username", userName);

        experiment.Use(() => FindUser(userName));
        experiment.Try(() => GetUser(userName));
    });
}

// Accessing context in a publisher:
public class MyResultPublisher : IResultPublisher
{
    public Task Publish<T, TClean>(Result<T, TClean> result)
    {
        foreach (var kvp in result.Contexts)
        {
            Console.WriteLine($"Key: {kvp.Key}, Value: {kvp.Value}");
        }
        return Task.FromResult(0);
    }
}

To prevent result publishing from delaying your experiments, wrap your IResultPublisher implementation in a FireAndForgetResultPublisher. This delegates the publishing task to a separate thread.

Scientist.ResultPublisher = new FireAndForgetResultPublisher(new MyResultPublisher(onPublisherException));

By default, Scientist runs tasks synchronously. To prevent experiments from doubling the latency of your method calls, use Scientist.ScienceAsync<T>(name, concurrencyLimit, callback) to run candidates in parallel. This ensures the total time taken is roughly the time of the slowest task.

await Scientist.ScienceAsync<int>(
	"ExperimentName",
	3, // number of tasks to run concurrently 
	experiment => {
        experiment.Use(async () => await StartRunningSomething(myData));
        experiment.Try(async () => await RunAtTheSameTimeAsTheControlMethod(myData));
        experiment.Try(async () => await AlsoRunThisConcurrently(myData));
	});

To avoid storing large or complex objects in your results, use experiment.Clean(selector) to extract a specific piece of data (like an ID or a string) from the observed value. The cleaned value is then available in result.Control.CleanedValue and result.Candidate.CleanedValue.

public IUser GetUserByEmail(string emailAddress)
{
    return Scientist.Science<IUser, string>("get-user-by-email", experiment =>
    {
        experiment.Use(() => OldApi.FindUserByEmail(emailAddress));
        experiment.Try(() => NewApi.GetUserByEmail(emailAddress));
        
        experiment.Clean(user => user.Login);
    });
}

// In the publisher:
// result.Control.Value = <IUser object>
// result.Control.CleanedValue = "user name"