Automated testing in CI/CD pipelines: types and stages

In the software development life cycle, automated testing integrated in the continuous integration and continuous delivery (CI/CD) pipeline is critical to ensuring software quality. Automated testing helps find errors quickly, reduces manual work, and improves test accuracy and coverage. In the CI/CD pipeline, automated tests are usually performed after the build is completed and before the code is deployed, including unit tests, UI tests, and integration tests, which can be run in parallel to speed up execution. Integration testing is especially important in a CI/CD pipeline because it tests the interactions between application components and validates the overall functionality of the system.

# In the previous article, we discussed various use cases for automated test cases for agile teams. One scenario is when a team wants to integrate testing with every build and implement continuous integration as part of the build process.

In this article, we will discuss integration testing in continuous integration/continuous delivery platforms.

Let’s start with the basics.

What is automated testing?

Software testing refers to executing tests based on a set of test cases and comparing actual results with predicted results. There are several steps and testing techniques to follow during this process. Testing is essential to ensure product quality. This process is typically done manually by teams of testers. However, in automated testing, the process is automated using software tools and techniques. Here, manual steps are no longer required but scripts are automatically created and test cases run. Automated testing eliminates human error, coverage, and many other issues. It also saves time, improves the convenience of conducting tests, and increases efficiency and effectiveness.

Automated Testing in the CI/CD Pipeline

Automated testing is a core part of the CI/CD pipeline because fast-running tests provide early feedback to developers. Problems or errors detected early have the potential to be corrected sooner. As a result, published products will be more accurate and error-free. This improves the overall quality of the product, thereby earning customer satisfaction. In short, the advantages of automated testing in CI/CD pipelines are as follows:

• In line with the CI/CD concept of “build fast, fail fast”

• Reduce the amount of manual work, thus saving time and reducing the possibility of errors.

• As the number of test cases increases, the test results become more accurate and can cover a wider area.

• Get immediate feedback on any issue

• Multiple test results can be generated and compared to ensure quality and consistency

Types of Automated Testing Used in CI/CD Pipeline

• Unit Testing: This is the low-level testing done after coding and review of the module . Test cases are designed to test individual components. The purpose is to ensure that every component works as expected under any circumstances.

• Integration testing: Integration refers to testing the interaction of components within an application. This testing is done after all modules have been unit tested. The main goal is to test module interfaces and check if there are any errors in parameter passing when one module calls a function of another module.

• System Testing: Testing is designed to validate a fully developed system and ensure that it conforms to the requirements specification document. Typically, at this stage, the software is ready for use by potential users. These types of system testing are called alpha (performed by a development team within an organization), beta (performed by a selected group of users/customers), or acceptance testing (performed by users/customers to determine acceptance of a delivered system).

What are continuous integration and continuous delivery?

In short, continuous integration allows development teams to integrate their code into a shared repository. This helps maintain code quality and identify potential issues with local versions of your code early.

Continuous delivery is also often called "continuous deployment". Everything that the development team is constantly merging is constantly being deployed to the live environment.

Since most developers work in parallel, constantly integrating their code into a repository means that the master branch is constantly updated with new features. To ensure that code quality doesn't suffer due to changes happening so quickly, tests must run at the same pace.

Not surprisingly, manual testing in this environment is not the best way to achieve this goal. Automated testing is the key to successful testing in your CI/CD pipeline.

9 Continuous Delivery Phases

• Development: Developers build code based on project requirements or feature requests.

• Write tests: Once you have written the code, you need to write tests. At this point, these tests are typically developer-written unit tests.

• Local testing: Then do local testing to check that all tests pass and make sure the code doesn't break. Typically, the percentage is set to the pass rate that the tests being run need to meet.

• Rebase and conflict resolution: In a real development scenario, there will be multiple people merging their code. Developers need to ensure that their branches are always updated. Updating a branch with the latest merged code is called "rebasing". Once repositioned, some conflicts may arise that need to be resolved. Afterwards, run the test again against the rebased code.

• Commit: Once the tests pass, the code is ready to commit all changes.

• Build: The developed source code is then assembled to build a deployment artifact that can be run on an instance, such as a server where the environment is local to the environment. This code is now ready to be deployed to a different testing environment.

• UAT: The code is then deployed to a test server and testers start testing the feature. These tests can be automated or manual.

• Merge: If the commit under test is approved by the testers, it will be merged into the master branch.

• Production deployment: Once the code is merged, it is deployed to production.

Every build a developer codes needs to go through the above process.

Where does automated testing fit in the CI/CD pipeline?

Ideally, automated testing occurs once the build phase is complete and the code can be deployed. Unit tests, UI tests, and integration tests can all be run at this stage. These tests help ensure that the code meets quality standards.

This phase can last from a few minutes to a few hours, depending on how the automation is architected.

Tests can be run in parallel to execute them faster. If the code fails during the testing phase, the build can be rejected without investing any further manual testing time.

Tools for CI/CD

• Jenkins: Jenkins is an open source tool for continuous integration. It's free to use, and jobs can be configured through the interface and scripts.

• Travis CI: This tool is free for open source projects and hosted on GitHub.

• Gitlab: Gitlab is a version control tool with its own cloud-based approach to CI. It is supported on multiple platforms and has both free and paid versions.

• Bamboo: Bamboo is a CI tool for Jira. If your organization uses Jira, it would be helpful to check out this tool. It also supports automatic merging on ticket approval.

Best practices for CI/CD pipelines to get the most out of test automation

• Incremental changes: always recommended Follow a feature-by-feature approach. If a feature is really large, it's better to break it down into smaller and faster-to-test features. This is important for automation because if something goes wrong, it's easier to identify the root cause. If your commitment is too big, figuring out the cause of the problem will be a daunting task.

• Determine what can be automated: It’s common for teams to quickly dive in and say “let’s automate everything,” but this is a common mistake. We have to understand the purpose of automation and identify the test cases that should be automated.

• Parallel testing: Tests should be run in parallel to make testing more efficient and timely. It can significantly reduce the time required to run tests, giving results faster. But it's not enough to just execute these tests in parallel; it's also important to scale the size of the server running the tests to really increase test speed.

Conclusion

Automated testing is an important part of successfully deploying projects while maintaining quality standards. Ensuring that tests are run at every stage provides good transparency into code quality. Errors can be caught early and any delays that may have been caused by them can be addressed promptly. Having a CI/CD pipeline for integrated testing helps speed up the testing and deployment process.

The above is the detailed content of Automated testing in CI/CD pipelines: types and stages. For more information, please follow other related articles on the PHP Chinese website!