THE BACKGROUND

Our client, a leading player in the contract intelligence sector, provides a sophisticated software platform for contract management, analysis, and optimisation. Their platform assists organisations in streamlining contract processes, extracting insights, and mitigating risks.

THE CHALLENGES

The client faced a series of testing challenges that hindered their development and software quality assurance processes, such as:

1). SLOW EXECUTION OF AUTOMATED TESTS

• The client's nightly regression tests took up to an hour to execute around 150 tests (and even longer if failures encountered).

• Adding new tests impacted the execution time and the availability of timely results.

• No new tests had been added for 6 months, meaning coverage had suffered.

2). TEST RECOVERY FROM FAILURE

• The tests were dependent on prior test success, causing a single failure to cascade into subsequent test failures.

3). READABILITY AND MAINTAINABILITY OF PAGE OBJECTS

• Page objects were generated through a record and playback tool, leading to naming and XPath inconsistencies.

• Object conventions were challenging to understand, making test creation and maintenance difficult.

4). OBJECT FRAGILITY

• Absolute XPath usage for object creation resulted in brittle object identification, leading to false failures after DOM changes.

5). CODE DUPLICATION

• The tests were written with extensive code duplication, lacking reusability.

• Any changes to code needed replication across multiple test cases.

6). STATIC DELAYS

• Static delays were added to account for slow page responsiveness, causing unnecessary waiting during test execution.

7). DATABASE SCRIPT SECURITY CONCERNS

• Some tests used database calls for test data setup, posing security risks.

8). LACK OF TRACEABILITY AND REPORTING

• The absence of any test case management prevented progress tracking, proper quality measurement, and future test planning.

9). DEFECT MANAGEMENT

• The lack of a structured defect management process hindered bug tracking and resolution.

10). UAT TIME TO EXECUTE

• UAT execution was taking up to 6 weeks to execute and was more extensive than required. This was because of a lack of faith in system testing activities, mostly due to low visibility of test outcomes.

THE SOLUTIONS

1). HEADLESS BROWSERS

• Headless browsers were introduced for faster test execution, leading to a 75% reduction in execution time.

2). TEST SUITE RESTRUCTURING AND DEPENDENCY MANAGEMENT

• The test suite was rearranged to manage dependencies and isolate failures, preventing cascading failures.

• Test recovery was added by introducing automated retry on failure with new browser session.

3). PAGE OBJECT REDESIGN FOR READABILITY AND MAINTAINABILITY

• Page objects were re-designed with standardised naming and relative XPath usage for improved readability and maintainability.

4). REUSABLE CODE BLOCKS AND METHODS

• Reusable code blocks, functions, and methods were implemented to eliminate code duplication.

5). DYNAMIC WAITS AND RESPONSIVE HANDLING

• Replaced static delays with dynamic waits to enhance test execution efficiency.

6). SECURE TEST DATA MANAGEMENT

• Database scripts were replaced with secure data management techniques.

• Added seeded data to database to ensure continued compatibility.

7). TEST CASE MANAGEMENT AND REPORTING TOOL

• Introduced a test case management and reporting tool via spreadsheets to enhance traceability and quality measurement.

8). DEFECT TRACKING AND LIFECYCLE MANAGEMENT

• Established a structured defect management process, from discovery and triage to resolution.

9). ADDED NEW TESTS FOR NEW FEATURES

• Used a Shift Left approach to ensure new features had test cases written alongside development of application code instead of adding to backlog.

• Reduced the feedback loop from original weeks to hours with outcomes and automation baked into process.

• Increased the system Ttest cases by more than double while maintaining execution time under 50% of original.

10). UAT AUTOMATION

• Approximately 160 automated tests were created for UAT to provide confidence in delivered features of each release.

• Automation also seeded data for extended UAT, which was previously a manual process.

THE OUTCOMES AND BENEFITS

The comprehensive solutions implemented resulted in:

1). SIGNIFICANT TIME SAVINGS

• Automated test execution time was reduced by 75%, enabling quicker feedback and feature releases.

2). ENHANCE SOFTWARE QUALITY

• A reduced test fragility, improved code quality and secure data handling contributed to higher software quality.

3). EFFICIENT MAINTENANCE AND SCALABILITY

• Readable page objects, reusable code, and dependency management improved test maintainability and scalability.

4). STREAMLINED TESTING WORKFLOW

• Enhanced test traceability, reporting, and defect management streamlined the overall testing process.

5). GREATER TEST COVERAGE

• New features tested alongside development, adding confidence to product quality and delivery of new features.

THE CONCLUSION 

By addressing a range of testing challenges through innovative solutions, our software testing consultancy successfully transformed the client's testing process.

The implementation of headless browsers, page object redesign, reusability, and robust test and defect management led to accelerated test execution, improved software quality, fewer test failures, faster execution times and streamlined testing workflows.

This case study exemplifies the potential of strategic testing enhancements in overcoming challenges and driving excellence within the realm of contract intelligence and contextualised contract data reconciliation.