Selenium and Appium with Python: Build robust and scalable test automation frameworks using Selenium, Appium and Python (English Edition)

Chapter 1

Testing Process and Role of Automation

Introduction

The definition of software testing is simple and has not changed since its origin. The actual developed software is in sync with the expected software, intended to be developed as defined in the business specification

The goal of software testing is to find potential errors in the developed software. Today, modern software testing is done using different automation tools, although a couple of years ago, the scripts were written manually and manual testers were doing the validations manually. Nonetheless, there remains a need for manual test engineers in high-sensitive software domains, for specific high-critical testing, which requires manual intervention.

Various questions might arise in our minds when we think of software testing. Where did it all start? How has software testing evolved? What is the standard testing process? What invoked the need for automation? In the field of software testing, knowing where and why it originated, will help understand the scenarios, different software testing types, their significance in the process, and how to implement them during real-time projects.

In this chapter, we will go over some of the majorly used concepts in software testing. This will provide you with great skills and a strong grip on the core concepts, while performing testing in real-world projects, as well as provide you with a great amount of confidence facing the software testing interviews.

Structure

In this chapter, we will discuss the following topics:

Significance of software testing

Core concepts of software testing

Types of software testing

Software Development Life Cycle (SDLC)

Defect/Bug life cycle

Automation in software testing

Objectives

By the end of this chapter, the reader will be able to understand the importance of software testing in the software industry, along with core concepts and terminologies used in software testing, different testing types, and their definitions. The reader will also have a complete understanding of the process of software testing and the defect cycle within the software development process and models, followed by testing automation significance.

Significance of software testing

Software Testing is a mandatory step in the software lifecycle. The completion of this step provides confidence and guarantees that the product is of a standard to be pushed to production for customers. It is significant because these software applications are bound to have errors, and identifying them in the early stage saves a great sum of money and time.

There are so many incidents that occurred in the past, that were caused due to software glitches. One such incident is the multiple Boeing 747 Max crashes in recent times. It was concluded that the key factor causing the crash was the Maneuvering Characteristics Augmentation System (CAS). It is assumed that maybe something went wrong during the testing of the updated system. The tragedy could have been avoided with complete comprehensive testing; the testers do impact the world. No major product launch happens without testing; any product, or even an app on the phone you are using right now, is delivered to you after testing. Testing is very crucial to identify any bugs or errors in the system early in the stage, so that they can be fixed before being delivered to customers. Moreover, apart from quality, this ensures dependability, security, and performance, which can benefit in cost and time saving, and thus customer satisfaction. As evident from the example of Boeing 747 Max mentioned previously, the company lost its reputation and dependability, the cost for the company in compensation, and the settlement was around 3-4 billion dollars.

A few major reasons that make software testing crucial in the software development process are as follows:

Software testing indicates the bugs and defects in the product that may have occurred during the development phase.

Software testing provides a smooth user experience for the customer. Thus, the company can gain their trust and confidence.

Software testing ensures that the application’s performance is intact for updates or the addition of new features to existing applications.

The application’s continuous software testing over time creates a platform for the developer to improve the development process. This prevents repeating the same error that occurred before, thereby reducing the coding cycles.

Software testing makes the process cost-efficient, by capturing early defects.

Software testing is a continuous process of delivering a clean product. Here are some major benefits of testing for companies and customers:

Better business optimization by reducing cost: It is very critical to the project, to figure out at which level of the software development, the bug was identified. The later the stage that the bug was raised, the more is the cost for the company.

Security: The testing phase case significantly finds the vulnerabilities in the software, which can prevent hackers from hacking the system. This ensures that the customer data or any significant information is safe.

Performance and efficiency of the application: This is closely related to the reputation of the company. During the testing phase, the performance of the application can be identified, which ensures that in the long run, the customers are satisfied.

Reputation: For any industry, the consumers are the most important part. As consumers, we too depend on companies that produce reliable products. The basis for this comes from how much time is spent on testing the product, to a point of its result in customer satisfaction.

User satisfaction: Customers are the highest priority for any industry. Customer satisfaction is directly proportional to the customer’s flawless experience with the product, which is tied to how much testing was performed to match the customer’s expectations.

Support to the development process: Regression testing of a product and tracking the testing over time, helps developers consider these potential error scenarios in the upcoming development cycle.

Core concepts of software testing

Some of the core concepts of software testing are as follows:

Software testing, quality assurance, and quality control: The terms software testing, quality assurance, and quality control are used closely with each other. Although they are thought to be the same, there are subtle differences among them all and these terms serve different purposes. Software testing is a process used to find possible bugs, defects, correctness, completeness, and quality of the developed software. Software testing is the core of the testing process, as shown in Figure 1.1, and it finalizes the software application by closing the gap between developed software and the business requirement, before the software is released to production for customer use. Figure 1.1 illustrates these concepts of software testing:

Figure 1.1: Concepts of software testing

Quality assurance is process-oriented, where testing and analysis are done to prevent error in the system before it happens. As shown in Figure 1.1, Quality Assurance is the outermost layer of testing which prevents the possibility of errors. This method provides assurance, that all the business requirements will be achieved or delivered as per the expectations of the consumer of the product. A high-quality software application cannot simply be achieved by testing the application. The defects occur because something somewhere did not work as expected; testing can find these defects. It will, however, not be able to prevent them from happening. Thus, the process that allowed these defects in the first place, needs to be identified and re-engineered.

Quality Assurance makes the development process more effective by taking into consideration the development techniques, pre-design, management, and project analysis, and so on. Quality Assurance comes under the category of Verification, which is a static analysis technique, where testing is done without executing the code.

Some examples of quality assurance in the software development process are as follows:

Software development planning.

The team resources are involved from the early stage of product development so that they are trained and have comprehensive knowledge of the application.

Requirement documentation.

Reviews and walkthroughs throughout the process.

Internal and external audits.

Quality control is a Product-Oriented approach, that acts as a bridge between software testing and quality assurance. As shown in Figure 1.1, it lies in the middle, to determine if the application has any defects, as well as to make sure that the software meets all the business requirements. It also mainly focuses on the final product. Quality Control ensures that standards are followed, and is performed after the software feature is developed, during the testing phase of the software development life cycle. It is also considered under the category of Validation, which is a dynamic analysis technique, where testing is done by executing the code.

Some examples of quality control in the software development process are as follows:

Inspections and Peer Code Review

Debugging Requirement documentation

Software Testing Process

The software testing process is as follows:

Test Strategy: As the name suggests, a test strategy is a high-level approach to how testing of the application will be carried out in the given cycle. This is documented by considering the test environment, test data, tools, risk analysis entry criteria, exit criteria, and so on.

Test Plan: This is a comprehensive document that lists all the activities to be performed in the testing. The test plan records requirements in scope, and testing is to be performed during different stages of development, customer sign-off, and so on.

Test Scenario: It is the description of the use case, an outcome of business specification, and it determines all technical aspects and objectives.

Test Case: This is a more specific step of testing linked to a given scenario. It comprises test steps, test data, the execution result of each step, and so on.

Test Execution: The test cases are executed using specific test data to determine the application behaves as expected and meets the specified requirement. The goal is to detect defect or issues in the system that could impact application functionality, security, performance, or other quality attributes.

Test closure: It is the final phase of the testing process ensuring testing objectives are complete and the application is reliable, stable, and ready for release to production. The phase involves analyzing test results, compiling test reports, and communicating the findings back to stakeholders.

Types of software testing

Let us look at the broad classification diagram shown in Figure 1.2, before deep diving into different types of testing. In a wide sense, the testing of an application can be categorized as Manual and Automation testing. Manual testing can be further classified into Functional and Non-Functional testing, both of which have subcategories based on the definition:

Figure 1.2: Manual Testing and its types

Let us now look at the definition of manual testing in detail.

Manual testing

This type of testing is done by manual testing engineers and performed manually by executing and validating the test cases interacting with the application. All the test cases and execution test results can be stored and tracked in a test case management tool or a spreadsheet.

Advantages of manual testing

The advantages of manual testing are as follows:

Mimicking the customer experience is only possible via Manual testing.

It is comparatively cheaper than Automation testing and requires less maintenance.

Highly efficient in low-critical applications, where investing time and money in automation does not provide business value.

Best testing solution during early stages of application development.

Production day release testing for new features is more efficient with manual testing.

Disadvantages of manual testing

The disadvantages of manual testing are as follows:

Considered risky and highly susceptible to human error.

Inefficient and unreliable when a huge number of test cases validation is required during pre-production release and time-bound testing.

Logging, reporting, and documenting execution results is extremely time-consuming.

Performance testing of the application is not possible or inaccurate with manual testing.

Functional and non-functional testing

When testing is done considering the business requirement of the application, it is called functional testing. This is a type of black box testing where only the output is validated against expected results that are based on the given input. Here, the internal flow or program structure is not considered.

On the other hand, Non-Functional testing assesses the application behavior which is not critical to the functionality of the application but contributes to the consumer experience.

Non-Functional testing includes Penetration testing, where the application is tested for security vulnerabilities, Performance testing, Usability, and Compatibility testing, where the application’s cross-platform support, Exploratory testing, and Accessibility testing are performed.

Unit testing

When testing is done to validate the smallest module or block of code of the application, which could be a function or a procedure, it is called Unit testing. Unit testing of the application is performed during its development, by developers or by testers. Unit testing is done by passing an argument to a function, and validating that the expected value is returned or executed by the function. The benefit of Unit testing is that it divides the whole application into the smallest of pieces, and validates its correctness, thereby helping to identify and fixing an error in the very early stages.

White-Box, Black-Box, and Grey-Box Testing

White-Box is a type of testing that is performed considering the inner component of the application. On the other hand, Black-Box is a type of testing that, like Functional testing, is performed considering only the expected result, based on provided input, and not the internal structure of the application. The grey-box testing is a software testing type that is used to test the behavior of the application and the internal component of the application.

Integration testing

This type of testing validates whether different modules that make the application, are working well together. This can also be done to validate two applications working together as expected.

User acceptance testing

Also called ‘end user testing’ or ‘application testing’, acceptance testing is performed by testers or consumers in the production environment. This testing output determines if the application or feature is ready to be released.

Alpha and beta testing

Both Alpha and Beta testing are kinds of User Acceptance testing, where Alpha testing is done before the release in order to find bugs in the application, and beta testing is performed by real users in the real environment.

System testing

In System testing, all the components of the application are tested as a single unit. This ensures that the developed software meets the business requirements. It is high-level testing performed after Unit and Integration testing.

End-to-end testing

This involves the complete testing of the application’s workflow, from start to finish in a real-world scenario, to validate the application for integration and data integrity.

Sanity testing

It is a subset of regression testing, performed on the application to ensure that the newly added feature or bug fixes have not broken any existing functionality.

Smoke testing

It is the testing performed to make sure that the core functionalities of the application are intact.

Software Development Life Cycle (SDLC)

Software testing is one of the phases in the development process. Software testing should happen in sync with the model followed for development. Let us discuss the flow of testing the application. There are multiple models for software testing, and the predominantly used one in the modern industry is the Agile methodology. Even though there are many models such as Agile, Spiral, V-Model, Rapid action, Big Bang, Waterfall, and so on, we will look at 3 famous models including Agile, to see how the models evolved.

Waterfall model

A majorly used model around 10 years ago, the Waterfall model is the earliest Software Development Life Cycle (SDLC) used in software development. It follows a distinctive sequential approach to performing Software testing. Software testers perform one step at a time, in each of the software development phases, starting from documenting the requirements, to maintenance after deployment. The next phase can only begin when the previous phase is completed, and hence it is also called the Linear-Sequential life cycle model. Refer to Figure 1.3 for an illustration of the Waterfall model:

Figure 1.3: Waterfall Model

As shown in Figure 1.3, the waterfall model comprises of the following broadly classified phases:

Requirement gathering and analysis is the first step in any software development process, where the business requirement and specifications are gathered and documented. The testers understand the requirements and come up with a testing strategy.

In the System design phase, the software and hardware requirements along with the development approach are finalized. This phase is where the testing team is finalizing the testing requirements.

This is followed by the actual implementation of the logic in the Implementation phase.

Then, all the required testing of the logic including integration (the link between two systems in software) and end-to-end testing (complete flow of a scenario), are performed. This is the phase where defects are logged and fixed, and there is continuous coordination between testers and developers.

Upon completion of testing and fixing all the defects, the logic is deployed to production in the deployment phase.

This is followed by maintenance of the deployed version and supporting customers in the last phase.

Waterfall model application

There is no ambiguity in this model. Neither is there going to be any requirement changes during the mid-phases. Moreover, the business specifications are clear and well-documented. There are no pending items from the previous phase, and there exists a clear understanding of the product development among the resources. This model is ideal for short projects with a small set of requirements.

Advantages of the Waterfall model

The advantages of the Waterfall model are as follows:

The simple and linear approach of this model is easy to understand and use.

It is also easy to manage since the deliverables in each phase are clear among the team.

Disadvantages of the waterfall model

The disadvantages of the Waterfall model are as follows:

The major disadvantage of the model is that it is time-consuming. The entire development process of navigating through each phase can take months and no working software is available until a later phase.

Until the development process is going on, the testing team is waiting idle.

If there is a change/update to the requirements, it increases uncertainty. Hence, in this model, significant changes to the business specification cannot be made.

Within each phase, the progress is not measurable.

The model can end up with a pile of errors in the testing phase because the integration and error risks are not considered due to integrations.

Iterative (and incremental) model

This is a model which was developed to address the shortcomings of the waterfall model. In this model, the software development process starts with the initial planning of the complete requirement in an Iterative (repetitive) and Incremental (Occurring in a short period) manner, with a new subset of complete requirements from the initial planning. This allows developers and the software development process to accept changes and improvements and can also accommodate changes to the requirements. The Iterative and Incremental Model is depicted in the following Figure 1.4:

Figure 1.4: Iterative and incremental Model

Iterative and incremental model application

While the major requirements are outlined and planned, minor changes to the specification and improvements can be added during the process. Testing is done in each iteration, giving confidence in the new addition. It accommodates the Integration testing during the iterations.

Advantages of the Iterative and Incremental Model

The advantages of the Iterative and Incremental model are as follows:

A working model of the application in the early phase of development allows early feedback and enhancements.

Testing is done in each iteration, resulting in the early detection of errors.

Changes to software, hardware, and system design requirements can also be updated during the process along with business specifications.

Within each iteration, the progress is measurable and can be improved upon.

It is easier to prioritize the requirements and manage risk, where high-risk or high-priority requirements can be taken in early iterations.

There is learning in each iteration that can be applied to the next iteration, which increases the efficiency of the development process.

It has a lower risk in comparison to Waterfall Model.

Disadvantages of the iterative and incremental model

The disadvantages of the iterative and incremental model are as follows:

The model is still not susceptible to major changes or improvements to the requirements, which can incur higher costs.

The number of Iterations depends on several factors; thus, the model does not provide enough control over the time of completion.

Not ideal for smaller projects.

Agile methodology – SCRUM

Agile is the most widely used and accepted software development model. This again relies on an Iterative and incremental model, with a focus on deliverability and adaptability with more control over time, thus leading to customer satisfaction. In Agile, instead of developing the software iteratively and deploying at the end of all iterations, the product is broken down into incremental builds. Hence, each iteration has a product deployment. Agile teams are designed to be able to change, improve, stabilize, and recover faster in case of a failure. In Agile, a team can prioritize the requirements and adapt in real-time, and thus Agile teams deliver.

Let us discuss Scrum Agile briefly. Scrum is the most widely accepted framework of Agile, which has stages of development called Sprints. Each sprint is of 2-4 weeks, although a week does not necessarily start on a Monday. In a real project, every day starts with a small meeting called daily scrum managed by the scrum master for a progress update, blockers, and resolving issues to plan a productive working day. Usually, all the teams in a company have an all-day meeting, once a quarter, to plan out each team’s goal, and dependencies called epics. Epics will have sub-stories and tasks that are required to be achieved. The refinement meeting helps the team pick up the Sub stories that are in individual sprints, considering the efforts and complexity. The story points are given to these stories which determine the time it could take to complete. If the story points cross certain limit, the story is broken down into two sub-stories which can be completed in two sprints. The beginning of a sprint starts with sprint planning to set the expectation. Here, each team member picks up these stories based on bandwidth, which is called team capacity and is calculated based on people’s availability in each sprint. At the end of the sprint, there is a Sprint review meeting to look at the finished work and get feedback from stakeholders and customers, based on which product owner can add, remove, or update the backlog items. At the end of each sprint, there is also a Retrospective meeting where the team looks back at the last sprint and opens a platform to answer questions and comments on what went well, which team should continue to work, what could be improved upon, appreciation of team member and so on. This helps the team to keep