Docs: Question about language

[gyuwon]

I have a question about the language. This document defines the terms fake, mock, and stub. Where does the definition of them come from? Many programmers use these terms as defined in the xUnit pattern, and they differ from the definitions in this document. I'm not saying the xUnit pattern is correct or this document is wrong, but because many programmers already follow the definition of the xUnit pattern, the language definition of this document needs reasonable grounds.

Please refer to these documents

• http://xunitpatterns.com/Test%20Double%20Patterns.html
• https://martinfowler.com/articles/mocksArentStubs.html

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 51947dc8-9ae8-21fa-13db-a7ffb3f9bc0e
• Version Independent ID: be661bf5-6f6e-cd18-344b-253edf5751ae
• Content: Best practices for writing unit tests
• Content Source: docs/core/testing/unit-testing-best-practices.md
• Product: dotnet-core
• GitHub Login: @jpreese
• Microsoft Alias: wiwagn

[jpreese]

Thanks for the feedback @gyuwon. In what ways do the xUnit pattern definitions differ from the ones in the best practices document? If the current wording makes it seem that way, we should probably reconsider the phrasing.

It essentially boils down to (and I believe the documents that you link follow this same approach):

_Mocks_ - Test double that you assert against (was something called, was it called how you expected it to be called, etc)
_Stub_ - Test double that is used to instantiate another object, instead of using a dependency (input). Can also be used as a form of output, for example the response from a web service call.
_Fake_ - Either of the above. If you have a in memory file system, but a much lighter implementation, the class itself should be called a FakeFileSystem. Then, in your tests, you could name it how its being used. Asserted against would be a mock, satisfying a depedency, a stub.

[gyuwon]

At first, this is the definition in the current version(947d54f8) of the best practice document.

Fake - A fake is a generic term which can be used to describe either a stub or a mock object. Whether it is a stub or a mock depends on the context in which it's used. So in other words, a fake can be a stub or a mock.
Mock - A mock object is a fake object in the system that decides whether or not a unit test has passed or failed. A mock starts out as a Fake until it is asserted against.
Stub - A stub is a controllable replacement for an existing dependency (or collaborator) in the system. By using a stub, you can test your code without dealing with the dependency directly. By default, a fake starts out as a stub.

Let me explain the difference.

Stub

The xUnit pattern defines stub as a tool used to set indirect inputs up. 'Predefined indirect input' is the main purpose of using stubs (we can use spies for indirect output). Therefore, they are not arguments for public APIs but answers from systems that SUT(system under test) depends on. Martin Fowler used the expression 'canned answers'.

Mock

According to the xUnit pattern, mocks check whether SUT correctly uses systems that it depends on. It means mocks verify indirect inputs and indirect outputs. I think the definition of mock in the best practice document is not much different from the xUnit pattern.

Fake

The xUnit pattern defines fake as a 'much lighter-weight implementation'. It means fake is a fully-functioning replacement of some component, but not fit in the production environment. I use Microsoft.EntityFrameworkCore.InMemory as a fake of the Azure SQL database. So in the xUnit pattern, fake is not a stub neither a mock. Fake is a kind of test double but not itself.

[jpreese]

Ok so it seems like we're OK with the Mock definition.

For stub, you call out that they cannot be used for a public API. What type of test double would at that point then? I would consider a stub both a "canned answer" as well as input to instantiate objects or pass as parameters. I think the stub definition highlights that stating that it's a way to not have to deal with a dependency directly. You can stub in any dependency you want to get the system under test up and running (input), or use it as a canned response (output).

For fake, speaking directory to Microsoft.EntityFrameworkCore.InMemory, I would consider that a Fake. But again, it comes down to how it's used. In their (testing documentation)[https://docs.microsoft.com/en-us/ef/core/miscellaneous/testing/in-memory] it looks like they're passing in the options variable to the BloggingContext. In this case, the InMemory fake is being used as a stub.

So that's definitely a difference. Stating that a Fake can't be a stub.

I personally prefer the approach described in the documentation, it simplifies things, but its a slightly different approach to defining the terms, so I'm open to further discussion.

[gyuwon]

According to the xUnit pattern, if I can prepare all answers for all possible cases of input(but why should I do that?) so it can be a working implementation of a component, that is a stub and also a fake because it not only provides canned answers but also fully functions. but if the test double provides answers only for single(or few) cases, then it's a stub but not a fake because it does not fully function. Although fake can be used as stub, it's not a generic term. The most generic term is just 'test double'. Consider following code.

public interface IRepository
{
    Order GetOrder(Guid orderId);
}

// Cannot be a replacement of a component,
// Simple and fast.
class RepositoryStub : IRepository
{
    public Order CannedAnswer { get; set; }

    public Order GetOrder(Guid orderId)
        => CannedAnswer?.Id == orderId ? CannedAnswer : default;
}

public void TestCaseUsingStub(Order order)
{
    // Arrange
    var stub = new RepositoryStub { CannedAnswer = order };
    var id = order.Id;

    // Act
    ...

    // Assert
    ...
}

// Can be a replacement of a component
// Not suitable to production and more complex and slower than stub.
class RepositoryFake : DbContext, IRepository
{
    public RepositoryFake(string databaseName)
        : base(new DbContextOptionsBuilder()
            .UseInMemoryDatabase(databaseName)
            .Options)
    {
    }

    public DbSet<Order> Orders { get; set; }

    public Order GetOrder(Guid orderId) => Orders.Find(orderId);
}

public void TestCaseUsingFake(string databaseName, Order order)
{
    // Arrange
    var fake = new RepositoryFake(databaseName);
    fake.Orders.Add(order);
    fake.SaveChanges();
    var id = order.Id;

    // Act
    ...

    // Assert
    ...
}

Sometimes stub is enough and sometimes we need fake.

And let's see some code in the best practice document.

var stubOrder = new FakeOrder();
var purchase = new Purchase(stubOrder);

purchase.ValidateOrders();

Assert.True(purchase.CanBeShipped);

stubOrder is a direct input through public API. Therefore it is not a stub but a fixture. Stub is a kind of a fixture but they are not same.

You can be prefer some approach for test double terms or you don't agree the xUnit pattern and I'm not saying that's bad. But then you'd better provide reasonable grounds to say they are 'the most common types' or bound the scope of the language to just the best practice document. Defining language is a complex and sensitive issue.

*I'm not good at English so if you feel that my expressions are rude please don't hesitate to tell me.

[jpreese]

Nope, I don't believe it's rude. Just a discussion 👍

So FWIW a lot of the information presented comes from Roy Osherove's school of thought when it comes to unit testing. I've read his books, watched his lectures, and agree with pretty much everything he says when it comes to unit testing. Obviously I wouldn't consider his approach or xUnit's approach wrong, just a different way of thinking.

In exciting news, I found a clip where he talks about this exact discussion that we're having, and he even mentions the xUnit patterns https://youtu.be/fAb_OnooCsQ?t=1537 . I've already included the timestamp so it should take you exactly to that point.

I think both schools are valid, but Roy simplifies the concepts and makes them (I feel) easier to understand.

Maybe a solution could be to sidestep the whole concept of a "fake". The point I really wanted to get across is that mock is way overused, and is often used incorrectly. That is the heart of that section. I don't want the community using the terms "stub" and "mock" incorrectly.

[BillWagner]

This is a great conversation. I prompted me to do some more research on these teams.

I wonder if the right answer for these docs is to reference existing content that covers this area quite well. You've both referenced a few sources above. Others include:

Martin Fowler - Test double
Martin Fowler - Mocks aren't stubs

Peter Provost has a series as well:

Stubs
Shims
Mocks

thoughts?

[jpreese]

My gut says to simplify the Language section and just highlight the Mocks vs. Stubs debate. It seems like we agree with the Mock definition, and there's a slight difference with stub, but I may be able to generalize it further.

I'm OK with dropping defining "fake" as it potentially takes away from the meat of the section (mocks vs. stubs). Using that same logic, I don't necessarily want to include articles linking to Shims, test doubles, etc. "Mocks aren't stubs" would be a good one. I would just personally prefer to keep the section focused on comparing and contrasting mocks and stubs.

[gyuwon]

I couldn't agree more with that mocks are special and different from other kinds of test double. Overusing mocks increases uncertainty and often leads us to make tight coupling between production code and test code.

Mockists Are Dead. Long Live Classicists.

Since the xUnit pattern is still popular, If the document specifies the source of other terms or the scope of the language, the reliability will be higher. Of course I respect Roy Osherove's opinion too.

[kgilmer]

IMHO the section about mocks vs stubs in the documentation, while relevant, takes on more importance than seems necessary. Also, how is the statement "By renaming the class to FakeOrder, you've made the class a lot more generic,..." qualified? TBH it seems like someone's pet issue made maybe a bigger impact than is appropriate.

[thomasvanveen]

Is it bad practice, using the DisplaName Property of the FactAttribute to describe, what a test does, instead of using very long unreadable Method Name?

[yannduran]

@specialwork, no it wouldn't be bad practice to do that. It's exactly why DisplayName exists.