We want to implement fully stood up acceptance level tests against all of our message handlers. Do you all recommend using the NServiceBus.AcceptanceTesting package for this? It seems like the best tool for job given that it is being used internally, but I’m wondering if this package is really just meant for internal testing. Even though it is a nuget package, I have found just a hand full of pages from Google searches describing how to implement tests using the package. I’m totally comfortable digging into the source if I need to… I just don’t want to head down the wrong path if there are better options out where.
I’m aware of the NServiceBus.Testing package… but how much of that gets mocked out when testing a handler. Open to other suggestions too… our main goal is being able to test our handlers as close to what a production deployment would look like.
1 Like
hi @RyanFerretti,
the NServiceBus.AcceptanceTesting package is currently designed for internal use only. It’s on nuget because we need it on nuget. It is not supported for usage in end-users projects. We don’t provide, yet, any end-to-end/integration testing framework.
One thing I’d like to better understand is: why do you feel the need of an integration/acceptance testing framework? In theory we could say:
- we test Core
- we test Transports and Persistence
- we have integration and acceptance tests
You should be able to rely on those assumptions and simply unit test in isolation your autonomous components exercising them with input messages and testing results.
What am I missing from the picture?
.m
In our development life cycle, we do traditional unit testing and what we call acceptance tests that are fully deployed websites that use selenium/chrome as a test driver. Having a test that hits the complete running application means that we know all of the config is setup correctly as well as the code is acting as asserted. We don’t use acceptance tests to cover every possible code path… but something simple like having all of your unit tests passing and your app doesn’t start because of some autofac error is what we are trying to catch.
Since NServiceBus handlers have no ui, they are a perfect candidate to completely automate the QA effort… so this is what led me to the AcceptanceTesting package. If you think that the NServiceBus.Testing package gets us pretty close to what we need… I guess my question is what does that package not do that we would need to figure out how to test or assume the risk.
If I understand correctly what you’re trying to achieve it can be summarized as:
- handlers can be tested.
- sagas can be tested.
- whatever part of Core, for example Pipeline behaviors or serializers, can be tested.
There is no way to test if 2 endpoints are correctly set up to talk to each other, e.g. are they using the correct serializers. Are transports configured correctly? and so on.
Am I heading in the right direction?
yes… so if NServiceBus.Testing can handle can handle your 3 bullet points that is great.
I’m not worried about testing that 2 endpoints are correctly setup to talk to each other… I would say that falls into the core of NServiceBus and I can trust that works. I think for my case we can do 95% of our testing using the NServiceBus.Testing package.
Would it be worth trying to build a simple smoke test to cover the other 5% using the AcceptanceTesting package that really just spins up a handler to make sure it starts up without testing any logic… thinking of something similar to https://github.com/Particular/NServiceBus/blob/develop/src/NServiceBus.AcceptanceTests/Core/Config/When_startup_is_complete.cs
Just thinking out-loud here, thanks for your help so far!
@RyanFerretti, intrigued by your scenarios I started investigating options to run integration tests so to test real endpoints. Here are some thoughts and a sample: Exploring NServiceBus Integration Testing options
.m