What's the difference between unit, functional

Question

What is the difference between unit, functional, acceptance, and integration testing (and any other types of tests that I failed to mention)?

Answer 1

Depending on where you look, you'll get slightly different answers. I've read about the subject a lot, and here's my distillation; again, these are slightly wooly and others may disagree.

Unit Tests

Tests the smallest unit of functionality, typically a method/function (e.g. given a class with a particular state, calling x method on the class should cause y to happen). Unit tests should be focussed on one particular feature (e.g., calling the pop method when the stack is empty should throw an InvalidOperationException). Everything it touches should be done in memory; this means that the test code and the code under test shouldn't:

• Call out into (non-trivial) collaborators
• Access the network
• Hit a database
• Use the file system
• Spin up a thread
• etc.

Any kind of dependency that is slow / hard to understand / initialise / manipulate should be stubbed/mocked/whatevered using the appropriate techniques so you can focus on what the unit of code is doing, not what its dependencies do.

In short, unit tests are as simple as possible, easy to debug, reliable (due to reduced external factors), fast to execute and help to prove that the smallest building blocks of your program function as intended before they're put together. The caveat is that, although you can prove they work perfectly in isolation, the units of code may blow up when combined which brings us to ...

Integration Tests

Integration tests build on unit tests by combining the units of code and testing that the resulting combination functions correctly. This can be either the innards of one system, or combining multiple systems together to do something useful. Also, another thing that differentiates integration tests from unit tests is the environment. Integration tests can and will use threads, access the database or do whatever is required to ensure that all of the code and the different environment changes will work correctly.

If you've built some serialization code and unit tested its innards without touching the disk, how do you know that it'll work when you are loading and saving to disk? Maybe you forgot to flush and dispose filestreams. Maybe your file permissions are incorrect and you've tested the innards using in memory streams. The only way to find out for sure is to test it 'for real' using an environment that is closest to production.

The main advantage is that they will find bugs that unit tests can't such as wiring bugs (e.g. an instance of class A unexpectedly receives a null instance of B) and environment bugs (it runs fine on my single-CPU machine, but my colleague's 4 core machine can't pass the tests). The main disadvantage is that integration tests touch more code, are less reliable, failures are harder to diagnose and the tests are harder to maintain.

Also, integration tests don't necessarily prove that a complete feature works. The user may not care about the internal details of my programs, but I do!

Functional Tests

Functional tests check a particular feature for correctness by comparing the results for a given input against the specification. Functional tests don't concern themselves with intermediate results or side-effects, just the result (they don't care that after doing x, object y has state z). They are written to test part of the specification such as, "calling function Square(x) with the argument of 2 returns 4".

Acceptance Tests

Acceptance testing seems to be split into two types:

Standard acceptance testing involves performing tests on the full system (e.g. using your web page via a web browser) to see whether the application's functionality satisfies the specification. E.g. "clicking a zoom icon should enlarge the document view by 25%." There is no real continuum of results, just a pass or fail outcome.

The advantage is that the tests are described in plain English and ensures the software, as a whole, is feature complete. The disadvantage is that you've moved another level up the testing pyramid. Acceptance tests touch mountains of code, so tracking down a failure can be tricky.

Also, in agile software development, user acceptance testing involves creating tests to mirror the user stories created by/for the software's customer during development. If the tests pass, it means the software should meet the customer's requirements and the stories can be considered complete. An acceptance test suite is basically an executable specification written in a domain specific language that describes the tests in the language used by the users of the system.

Conclusion

They're all complementary. Sometimes it's advantageous to focus on one type or to eschew them entirely. The main difference for me is that some of the tests look at things from a programmer's perspective, whereas others use a customer/end user focus.

Answer 2

Unit Testing - As the name suggests, this method tests at the object level. Individual software components are tested for any errors. Knowledge of the program is needed for this test and the test codes are created to check if the software behaves as it is intended to.

Functional Testing - Is carried out without any knowledge of the internal working of the system. The tester will try to use the system by just following requirements, by providing different inputs and testing the generated outputs. This test is also known as closed-box testing or black-box.

Acceptance Testing - This is the last test that is conducted before the software is handed over to the client. It is carried out to ensure that the developed software meets all the customer requirements. There are two types of acceptance testing - one that is carried out by the members of the development team, known as internal acceptance testing (Alpha testing), and the other that is carried out by the customer or end user known as (Beta testing)

Integration Testing - Individual modules that are already subjected to unit testing are integrated with one another. Generally the two approachs are followed :

1) Top-Down
2) Bottom-Up

Answer 3

This is very simple.

1. Unit testing: This is the testing actually done by developers that have coding knowledge. This testing is done at the coding phase and it is a part of white box testing. When a software comes for development, it is developed into the piece of code or slices of code known as a unit. And individual testing of these units called unit testing done by developers to find out some kind of human mistakes like missing of statement coverage etc..

2. Functional testing: This testing is done at testing (QA) phase and it is a part of black box testing. The actual execution of the previously written test cases. This testing is actually done by testers, they find the actual result of any functionality in the site and compare this result to the expected result. If they found any disparity then this is a bug.

3. Acceptance testing: know as UAT. And this actually done by the tester as well as developers, management team, author, writers, and all who are involved in this project. To ensure the project is finally ready to be delivered with bugs free.

4. Integration testing: The units of code (explained in point 1) are integrated with each other to complete the project. These units of codes may be written in different coding technology or may these are of different version so this testing is done by developers to ensure that all units of the code are compatible with other and there is no any issue of integration.

Answer 4

The important thing is that you know what those terms mean to your colleagues. Different groups will have slightly varying definitions of what they mean when they say "full end-to-end" tests, for instance.

I came across Google's naming system for their tests recently, and I rather like it - they bypass the arguments by just using Small, Medium, and Large. For deciding which category a test fits into, they look at a few factors - how long does it take to run, does it access the network, database, filesystem, external systems and so on.

http://googletesting.blogspot.com/2010/12/test-sizes.html

I'd imagine the difference between Small, Medium, and Large for your current workplace might vary from Google's.

However, it's not just about scope, but about purpose. Mark's point about differing perspectives for tests, e.g. programmer vs customer/end user, is really important.

Answer 5

Some (relatively) recent ideas against excessive mocking and pure unit-testing:

• https://www.simple-talk.com/dotnet/.net-framework/are-unit-tests-overused/
• http://googletesting.blogspot.com/2013/05/testing-on-toilet-dont-overuse-mocks.html
• http://codebetter.com/iancooper/2011/10/06/avoid-testing-implementation-details-test-behaviours/
• http://cdunn2001.blogspot.com/2014/04/the-evil-unit-test.html
• http://www.jacopretorius.net/2012/01/test-behavior-not-implementation.html
• Why Most Unit Testing is Waste

Answer 6

http://martinfowler.com/articles/microservice-testing/

Martin Fowler's blog post speaks about strategies to test code (Especially in a micro-services architecture) but most of it applies to any application.

I'll quote from his summary slide:

• Unit tests - exercise the smallest pieces of testable software in the application to determine whether they behave as expected.
• Integration tests - verify the communication paths and interactions between components to detect interface defects.
• Component tests - limit the scope of the exercised software to a portion of the system under test, manipulating the system through internal code interfaces and using test doubles to isolate the code under test from other components.
• Contract tests - verify interactions at the boundary of an external service asserting that it meets the contract expected by a consuming service.
• End-To-End tests - verify that a system meets external requirements and achieves its goals, testing the entire system, from end to end.