It feels like everywhere I turn (in my circles anyway), people are talking about Ports and Adapters, and I thought I would jump on the bandwagon. You may have also heard this architecture called Onion Architecture, or Hexagonal Architecture. All of these are essentially the same idea, under a different name.
When we started solving our problems with a 3 tier architecture, we were trying to solve the problem of isolating our business logic from our persistence and from our presentation. But it wasn't a silver bullet, due to coupling back up the stack of implementation concerns, and the idea has been refined over the last decade or so. The TDD/BDD movement lead us down a more prescriptive approach, as well as CQRS patterns, and basically more and more abstractions. But those who were implementing TDD/BDD had lost the core of the messaging. Specifically, isolate your tests from each other, not your methods from their required dependencies.
Mocking frameworks allowed us to test our code with heavy white-box testing, but by coupling our tests to our implementation and making them volatile to change. The intention of TDD was to isolate tests from each other, and be implementation agnostic, not to test isolated chunks of code. We might want to test that our system spits something out into a persistence layer, but we should be more concerned that given valid input, we get the correct success response, and a subsequent request returns the expected values back to us. We also want to focus on business logic, complex rules and interactions, and not presentation or persistence concerns.
What we ended up with was - at worst-case - a test that checked the tested 2 lines of code did what the two lines of code did (often though duplication of the logic in the test itself) or a test with 90% of it's code there to mock out all 8 (or 20) dependencies. Worse, we would be mocking
FizzMapper, and other core parts of the behaviour we are looking at testing. The net result is we test the plumbing and mock the implementation details out. Your experience may vary, but I have seen more than one code base heading into or already in a state similar to this.
The Silver Bullet
Ok, there is no such thing, but there is certainly the reverse, a Blunt Knife if you will. Some techniques, once other techniques are learned, often fall into the bad idea category, the 'doing it this way is almost always the wrong way' type of approaches, and I'm thinking the mock everything approach falls into this category. But until you see another way, it is hard to see the light.
This is where Ports and Adapters comes in. The idea here is that we define our system and its behaviours at the centre of our architecture. We then expose ports (think DTOs and interfaces, in a loose shared .DLL sense of the terms) that are the places where we can build user experience on top of the system functionality. We define adapters that fit into these ports, such as our UI. We can even have multiple versions of the adapters we plug in, such as an MVC front-end, and a WPF Desktop or Windows Store application adapter, or perhaps a developer Rest API adapter.
These ports and the only place we plug anything(as adapters) into our system, so we only need to write our unit tests to test our system for specific behaviours by calling into these ports. Our test methods now become adapters themselves.
The other side of the Ports and Adapters conversation is the persistence, or outgoing communications, such as logging, external service calls and your database repositories. Where these services plug in are also defined as ports, but instead of your adapter calling into a specified interface, your adapter is an implementation of it. These ports now become the ones you need to Fake ,Stub or Mock in your tests, not internal components used by a specific class, such as mappers or builders. Think about how this approach actually isolates your persistence layer implementation details from your domain specific logic?
So how do we test? Well, lets redefine what we mean by Unit Test, and write Unit Tests for our architecture. Unit Test, means testing a Unit of Behaviour in isolation from other tests. This means we need to not use the real implementations of your database adapters and logging frameworks and third party APIs, we can stub or fake these as required. We also won't include our presentation adapters since our test is acting as this type of adapter and calling the ports where the behaviour lives.
You can still do your traditional black-box testing on internal components, especially some of your purely functional ones, the same way you have been. Ill call these developer tests. In theory these can be useful for TDD, but can be easily deleted later on once unit tests have been written.
You will also want to make sure you are testing your adapters. You know what your system expects from your adapter, what their interface is for use, so test that when your application calls out to functionality, that it works as expected.
You will also want to continue with your existing Integration and Automated UI testing as you do now, although hopefully you can see that since you will have more confidence in the business logic being tested, the breadth of these two types of tests can be simpler and more generalised. Think about Martin Fowler's Test Pyramid and avoiding the Ice Cream Cone anti-pattern
There is a more detailed write-up on the Hexagonal Architecture Pattern by Alistair Cockburn for you to read, and searching around for Ports and Adapters, Hexagonal Architecture or Onion Architecture will find a multitude of resources out there.
Lastly, for a good introduction to this pattern in an iterative example, and very funny talk, check out the video Architecture the Lost Years by Robert Martin (Uncle Bob), and also his Clean Architecture article for more details across this pattern.