The Pros and Cons of Different Test Tools – FitNesse

Some testing tools are defined by their capabilities others by the way they use syntax or the formatting of test scripts that make them popular. FitNesse is one of those old test automation tools, that while not offering too much in the way if innovation or capabilities, stands out because of its unique approach to test script design that makes it readable to non-coders and easy to work with from a data perspective in the way it manages it’s different test inputs and outputs.

FitNesse was built around the idea of BDD testing, similar to other tools we’ve looked at like Cucumber. However, whereas most tools make use of the Gherkin syntax for their BDD approach, FitNesse takes more of a wiki-based data-driven approach that operates similarly to the way an Excel spreadsheet would work. This makes it easy for people who are used to working with data in Wikis or spreadsheets to be able to easily replicate automation scripts.

The tool has been around for a long time and although it has continued to develop, its older architecture is definitely not up to date with many latest development trends. It’s also arguably not as updated with new features as other tools meaning that it’s usage is dwindling. However, its approach to BDD and script writing does mean that it has a wide following especially when integrating with some legacy systems. There are even companies who may have stopped using FitNesse as a test automation code, who model their testing frameworks on the FitNesse BDD approach because of how easy it is to understand and work with.

Here are some key features and concepts associated with FitNesse:

• Wiki-Based Interface: FitNesse provides a web-based, wiki-style interface that allows testers, developers, and business stakeholders to collaboratively create and manage test cases and documentation. This makes it easier to maintain test documentation alongside the actual tests.

• Acceptance Testing: FitNesse is often used for acceptance testing, where non-technical stakeholders (such as product owners or business analysts) can write and understand test cases in plain English. This helps bridge the communication gap between technical and non-technical team members.

• Fixture and Test Tables: Test cases in FitNesse are created using tables with rows and columns. These tables consist of fixtures (code that interacts with the system under test) and test data. The fixture code is responsible for executing test actions and verifying results.

• Extensible and Customizable: FitNesse is designed to be highly extensible. You can write custom fixtures in various programming languages (e.g., Java, .NET, Python) to interact with your application. This flexibility allows you to automate tests for a wide range of software systems.

• Integration with Version Control: FitNesse can be integrated with version control systems like Git, enabling you to track changes to test cases and collaborate effectively with team members.

• Test Execution and Reporting: FitNesse provides reporting features that show the results of test runs, including pass/fail status and detailed logs. This helps in identifying and diagnosing issues in the application.

• Continuous Integration (CI) Integration: FitNesse can be integrated into CI/CD pipelines, allowing you to automatically execute tests whenever code changes are pushed to the repository. This helps ensure that tests are run consistently and continuously.

• Test Data Management: FitNesse allows you to manage and manipulate test data easily, which is crucial for executing a wide range of test scenarios.

• Plugins and Extensions: There are numerous plugins and extensions available for FitNesse, which can enhance its functionality and integrate it with other tools and frameworks.

• Support for Multiple Languages: FitNesse supports multiple programming languages for writing fixtures, making it accessible to development teams with various technology stacks.

So, what makes the scripts of FitNesse so unique? Well, below I break down how the scripting works and why many companies that have gotten used to this scripting approach continue to rely on FitNesse for their test automation needs

A FitNesse test script is typically represented as a table within a FitNesse wiki page. The table consists of rows and columns, with each row serving a specific purpose. Here's a breakdown of how a basic FitNesse test script looks:

Test Table: A FitNesse test script starts with a test table enclosed in double vertical bars (||). The table defines the structure of the test and consists of multiple rows.

Action Fixture Row: The first row of the table specifies the action fixture. The action fixture is responsible for interacting with the system under test (SUT). It typically contains the name of the fixture class and any arguments it requires. The fixture class contains the automation code that performs actions on the SUT.

Arguments Row: If the action fixture requires arguments, they are specified in a separate row just below the action fixture row. The arguments are separated by vertical bars (|).

Input Data Row(s): Following the action fixture and its arguments, you can have one or more rows for providing input data to the test. The number of columns in these rows should match the number of arguments specified earlier. Input data is used by the fixture to perform actions on the SUT.

Output Data Row: After specifying input data, you can have a row to specify expected output data or results. This row helps in verifying that the actions performed by the fixture yield the expected outcomes.

Assertion Row: The assertion row is used to define the criteria for checking whether the test passes or fails. It typically contains assertions that compare the actual results produced by the fixture with the expected results specified in the output data row.

Comment Row(s): You can include comment rows within the table to provide additional context or documentation for the test script. Comment rows start with a ! symbol.

In this example, the FitNesse test script uses an action fixture named "ActionFixtureName" to perform an action on the SUT. It specifies input data, expected output data, and assertions to verify the test's success or failure. Keep in mind that the specific syntax and structure of FitNesse test scripts may vary depending on the fixture being used and the conventions followed by your team or organization.  

Below is a brief history of FitNesse: ·      

• 2001: FitNesse's development began in 2001 when Ward Cunningham (the inventor of the wiki) and Robert C. Martin (also known as Uncle Bob) started exploring ways to apply the principles of wiki collaboration to software testing.

• 2002: FitNesse was first introduced to the public as an open-source tool for acceptance testing in 2002. It was initially created in Java and designed to facilitate collaboration between developers and non-technical stakeholders.

• 2004: FitNesse gained popularity within the agile development community for its ability to support behavior-driven development (BDD) practices and foster collaboration among cross-functional teams.

• 2005: A significant development occurred in 2005 when the FitNesse project was moved to a Google Code repository, making it more accessible and open to contributions from the software development community.

• 2006: FitNesse continued to evolve with improvements in its user interface and support for various programming languages beyond Java, including .NET and others.

• 2010s: FitNesse continued to be actively developed and maintained throughout the 2010s. It received updates, bug fixes, and contributions from developers and testers worldwide.

• 2020s: FitNesse remains a relevant tool for acceptance testing and continues to be used in agile and BDD environments, although other testing frameworks and tools also gained popularity.

Below is a list of pros and cons of using FitNesse:

Pros:      

• Ease of Use: FitNesse's wiki-based interface makes it user-friendly, especially for non-technical stakeholders who can write and understand test cases in plain English.

• Collaboration: It promotes collaboration among team members, including developers, testers, product owners, and business analysts, by providing a platform for creating and maintaining test documentation.

• Acceptance Testing: FitNesse is well-suited for acceptance testing, allowing teams to define and automate tests that align with business requirements and user expectations.

• Versatility: FitNesse supports various programming languages for writing fixtures, making it adaptable to different technology stacks and applications.

• Continuous Integration: It can be integrated into CI/CD pipelines, enabling automated test execution as part of the development workflow.

• Test Data Management: FitNesse offers robust test data management capabilities, allowing testers to create and manipulate test data easily.

• Extensibility: You can extend FitNesse's functionality by writing custom fixtures and integrating them with other tools and frameworks.

• Active Community: FitNesse has an active and supportive user community, which means you can find resources, tutorials, and solutions to common problems online.

Cons:

• Limited Documentation: While FitNesse has documentation and community resources, it may not be as extensively documented as some other testing tools, which can make it challenging for new users to get started.

• Learning Curve: Despite its user-friendly interface, writing and configuring fixtures may require a learning curve, especially for those new to FitNesse or test automation in general.

• Maintenance Overhead: Maintaining a large number of FitNesse tests and fixtures can become complex, especially in larger projects, and may require a dedicated effort to keep tests up to date.

• Test Execution Speed: Depending on the complexity of the tests and the technology stack, FitNesse tests can be slower to execute compared to other testing tools.

• Limited Test Reporting: While FitNesse provides basic reporting capabilities, more advanced reporting and analytics may require additional plugins or custom development.

• Dependency on Fixtures: FitNesse relies heavily on fixtures to interact with the system under test, and if fixtures are not well-maintained or become outdated, it can impact the reliability of tests.

• Not Ideal for All Testing Types: FitNesse is best suited for acceptance testing and may not be the best choice for unit testing or certain types of performance and load testing.

• Integration Challenges: Integrating FitNesse with certain technologies and platforms may require additional effort, and not all systems are equally compatible with it.

When to use FitNesse:      

• If you’ve already been using FitNesse for a long time on an older application and have a team familiar with its approach.

• You have a lot of data-driven test cases that fit easily into a table format, so the conversion to the FitNesse test script approach is quite easy.

• You have a UAT-heavy test approach with a lot of testing and automation driven from business users.

It may not have the greatest feature-set, but its simplicity of operation and being wiki and data-based makes it incredibly easy to use,  especially for business users. Yes, scripts require a lot of maintenance, but for many data-driven applications where the data changes more than the actual test execution steps, the FitNesse approach can still save a lot of time and be the best approach to getting a high test coverage.