As the development style is bending more towards the agile and scrum approaches, the need to test more often and at a minimal level has become a necessity of the industry. In this article, we will dig deep into different types of software testing.
Manual vs. Automated Types of Software Testing
Software testing is crucial for ensuring the quality and reliability of applications, and it can be categorized into manual and automated testing. Manual testing involves human testers executing test cases without the use of automation tools. This approach is essential for exploratory testing, usability testing, and scenarios requiring human judgment. Testers can adapt on the fly, providing immediate feedback on user experience and interface issues. However, manual testing can be time-consuming and prone to human error, making it less suitable for repetitive tasks or large-scale applications.
On the other hand, automated testing utilizes specialized tools to run tests automatically, enhancing efficiency and accuracy. Automated testing is particularly beneficial for regression testing, performance testing, and large-scale projects where consistency is key. By enabling frequent test execution, automation helps identify defects early in the development cycle, ultimately reducing time-to-market. While the initial setup of automated tests may require a higher investment in time and resources, the long-term benefits often outweigh the costs, especially in agile development environments where rapid changes are common. Balancing both methods can lead to a robust testing strategy, leveraging the strengths of each approach.
The Importance of Self-Testing Code
As Martin Fowler said, “At the core of my philosophy to testing is that we want to write self-testing code, meaning we have a suite of automated tests that be easily run against the codebase. We should be confident that this suite will catch almost all bugs in the software, so that when this suite ‘goes green’ (passes), we can release that version of the software into production.” Self-software testing code or Automated software Testing behavior is one of the biggest advances in development methods in the last few decades. This developer-friendly practice has vast benefits in terms of increasing productivity, improving quality, and keeping software from becoming brittle. The fact that so many developers are now doing it of their own free will speaks for its effectiveness.
What types of software testing should you have?
There are different types of software testing, contingent on what they are trying to verify, the extent of the codebase they cover, and their role in the software process. A popular way to look at your test suite is the “Test Pyramid” (Mike Cohn came up with this concept in his book Succeeding with Agile). It’s a visual metaphor telling you to think about different layers of testing. It also tells how much testing to do in each layer.
Write lots of small and fast unit tests, some more integration tests, and very few high-level tests that test your application from end to end. For most regular projects it is sufficient and provides the best possible coverage. People usually get confused by the names of layers in this pyramid.
General Metaphor for Testing
Remember, this pyramid is a general metaphor for testing; Unit Testing is component-level testing, Service Testing is integration testing (testing multiple modules together) and UI Testing is the end-to-end complete application testing. This pyramid is ignored in applications that are more critical, such as avionics software, or industrial applications such as SCADA (ex: concrete batching software). Because here, we need to make sure that our software works properly with the hardware. Otherwise, the aircraft might crash and people will die.
Example
For example, the hoppers above a mixer in a plant will not release the correct amount of ingredients for us to make the concrete that we need, resulting in a dead batch. Unit Testing is the backbone of most of your applications’ test suites. You cannot guarantee the quality of software without unit testing. As someone said, “You can’t be Agile without Unit Testing.”.
What Types of Software Testing Should You Implement?
Unit testing is a method where components or units of software are continuously tested to determine their compliance with the system specifications. Unit Testing also includes Data and usage procedure testing. A unit is simply the smallest piece of code that is logically isolated in a system. The unit test itself is a short piece of code that is designed to verify the behavior of a particular unit to produce a pass or fail result.
Unit Testing: The Backbone of Types of Software Testing
When developers write Unit Tests, they often get confused between Functional and Unit Testing (Types of Software Testing). What is the size of a unit? How can you tell if you’re writing the correct unit tests? Your system can add employees to the database, does testing this functionality be considered a Unit Test?
- A functional test is when you test one complete functional requirement. e.g. Writing a single test for a User Story can be considered as a Functional Test.
- As for the unit testing, there is no predefined size of it. It is a wide term.
Michael Feathers, the author of Working Effectively with Legacy Code, concurs that there is no “definition of unit test”.
List of Indicators
He instead offers a list of indicators that one shouldn’t find in a unit test.
A unit test should not:
- Talk to the Database (until its sole purpose is to test the database).
- Communicate across the network.
- Touch the file system.
- A special environment (e.g. editing in config files) is required to run it.
- It can’t run at the same time as any of your other unit tests
If your code is properly refactored, then 1 Method = 1 Unit Test is often acceptable.
System Under Test (SUT): Types of Isolation Techniques
In unit testing which is a type of software testing, your SUT (System Under Test) should be fully isolated from other pieces of software.
If it depends on another method, a service, a database, or maybe data from an external source then your unit test is not ideal. Imagine you’re testing an invoice class’s create method. The invoice creates method needs to invoke some functions on the order and customer classes. Ideally, your test shouldn’t use the real order or customer classes here, because a fault in the order class would cause the invoice class’s tests to fail. Instead, you use Test-doubles for the dependents.
Figure 1: A unit test typically replaces external collaborators/dependents with test doubles
Note: The SUT is whatever class or method(s) we are writing the unit test for.
Types of Test Doubles in Software Testing
A Test Double is any object/component that we install in place of the actual component to easily run the test.
There are a few different kinds of test doubles,
- Fake: An actual implementation of the piece of code, that is not suitable for production.
- Dummy: Used as a placeholder when an argument needs to be filled in.
- Stub: Provides the SUT with fake data.
- Spy: Records details about how it is used and will return the information to the test.
- Mock: Defines an expectation of how and for which parameters it will be used. If the requirement is not met, it will cause a test to automatically fail.
The concern of Test Doubles
As a software engineer, you will mainly be more concerned with mocks and stubs. Later on, maybe spies. A lot of people use the words Mock and Stub interchangeably. Though at the top they might look the same at the core they have different responsibilities. The need for Stub/Mocks can be well understood by looking into “Indirect Input” and “Indirect Output” of your SUT.
Indirect Input
Not all SUT inputs are the output of the test. Some of the indirect inputs come from other SUT-call components in the form of return values, modified parameters, or thrown exceptions. Here, we don’t care about what is happening in the dependent component. We care only about what it returns as a result.
Indirect Output
Not all SUT outputs are visible to the test. In the form of method calls or messages, such indirect outputs are sent to other components. We have to be able to observe the calls that the SUT makes to the dependent component to evaluate the indirect outputs. In addition, we need to be able to control the values returned if we need the test the progress beyond that stage.
Stubs and Mocks with Indirect Inputs and Outputs
Stubs
A Test Stub is an object that acts as a control point (component) to deliver indirect inputs to the SUT when the Test Stub’s methods are called. It retains predefined data and uses it during testing to answer calls. It is used when we don’t want or can’t involve items that would be answered with real data or have unwanted side effects. The internal structure of this object is not important for our test.
Example
For example, an entity that needs to fetch certain data from the database to respond to a method call might be an example. We create a stub instead of the individual object and decide what data should be returned.
<?php
public function averageGrades(Student student)
{
return averageStub(student.grades);
}
Public function averageStub(grades){
...
return 8;
}
We can use multiple stubs to test the different execution paths in our SUT. If the method we are testing calls a function that returns information that affects the execution path in our SUT, we need to use stubs.
Mocks
Mocks are different; here, we care about the function we’re calling, not just what it returns. A Mock Object serves as an observation point for the indirect outputs of the System Under Test (SUT). With mocks, the focus is on what the function received from the SUT.
When we don’t use the actual code or lack a clear way to validate execution, we rely on mocks. It’s challenging to check the return value or system state change in such cases.
For example, if we’re testing a method that calls an external API to conduct a background check on employee information, we need to ensure our SUT correctly calls the API method. This scenario is ideal for a mock, as it verifies that our SUT behaves properly and invokes the API with the right parameters at the right time.
Structure of Stubs and Mocks
Tests written with mocks usually follow an initialize -> set goals/expectations -> exercise -> verify test patterns. Whereas, initialize -> exercise -> verify would be followed by the pre-written stub. An easy way to remember the difference between mocks and stubs is to remember that state is verified by stubs, and behavior is verified by mocks.
Unit Testing & Laravel
As for now, we have learned the basics of Unit Testing. You might not have understood everything yet. Unit Testing is a wide concept and it requires practice to be skilled. PHPUnit is the testing framework for PHP. Like for other languages, JUnit and XUnit. JUnit for Java and XUnit for NET. Before, we had to do everything with PHPUnit, because that is what was available. Writing mocks with PHPUnit was painful but with the invention of Laravel; PHP’s most famous framework, testing has become a lot easier.
Laravel Mocks
Laravel has been built by keeping testing in mind. Support for PHPUnit testing is included out of the box, and a phpunit.xml file has already been set up for use. By default Laravel’s ‘’tests’’ directory contains two sub-directories, “Feature” and “Unit” for feature (functional) and unit tests respectively. Making tests in Laravel is easy. You only need to run the PHP artisan make test –unit command and it’ll create an empty Unit Test in the tests>Unit directory.
<?php
namespace Tests\Unit;
use PHPUnit\Framework\TestCase;
class ExampleTest extends TestCase
{
/**
* A basic test example.
*
* @return void
*/
public function testBasicTest()
{
$this->assertTrue(true);
}
}
Laravel offers many built-in methods and assertions to make testing pain-free.
Mockery
To make mocking easy, Laravel uses Mockery. It is a simple mock object framework for PHP which you can be used with PHP Unit. You can read more about mocking in Laravel by going to this link: https://laravel.com/docs/8.x/mocking Here, we can see that Laravel gives many built-in mock objects for Notifications, Emails, Queues, and File Uploads, etc. which make our lives easier. To start writing unit tests in Laravel.
Things to Remember in the selection of types of software testing
This article is about the general idea of types of Software Testing. You might not have learned everything but what you’ve learned will help you get started with unit testing. Unit testing (a type of software testing) is practical knowledge. You can’t excel in it by just reading different books and articles. I recommend you start writing Unit Tests for your application today. You might not be able to write a perfect test at once but with practice and skills, you’ll gradually succeed in it.
Further Reading about Types of Software Testing:
- Testing Guide – Martin Fowler
- The Practical Test Pyramid – Martin Fowler
- Test Double – Martin Fowler
- Test Double – xUnit Patterns
- Mocks Aren’t Stubs – Martin Fowler