Automation and Regression Testing for the Basic Payment Scheme

The Basic Payment Scheme (BPS) is a system in the Department of Food, Agriculture and the Marine (DAFM), that processes €1.5bn in direct payments to farmers in Ireland every year. Version 1 took over the support of the current system and redeveloped it to support the needs of new payment schemes under the Common Agricultural Policy (CAP) in 2015.

The BPS was introduced in 2015 and replaced the Single Payment Scheme, with all entitlements held under the Single Payment Scheme expired on 31 December 2014. A new set of entitlements were allocated in 2015 to those eligible for an allocation under the Basic Payment Scheme.

The BPS and a Greening Payment allow the Government to pay all eligible farmers and qualify other farmers for a further payment under the Young Farmer Top-up or the Protein Aid Scheme.

Challenge

New Demand

As the new CAP happened, there was a huge demand for new functionality, support for changing business processes, bug fixes, upgrade of libraries and technologies. On the BPS project, Version 1 followed an Agile process with 4 weeks sprints that required several releases per year.

To reduce pressure, DAFM suggested the introduction of automation testing.

Solution

Behaviour Driven Development

The BPS introduced the idea of Behaviour Driven Development in DAFM. The first step involved developing automation tests for existing functionalities, followed by 3 amigos sessions, where Version 1 could extract correct scenarios from users to test new functionalities.

While progressing the test scenarios before implementation, Version 1 had several meetings with users to confirm that the scenarios were correct. These meetings resulted in collaboration to generate scenarios as well progressing as the adaptation of the Gherkin language. This end-to-end testing resulted in the increased efficiency of the development team and assisted with regression testing.

The BPS team now categorises four different automated tests: smoke tests, regression tests, sprint tests, and functional tests. Depending on the type of test, they are executed hourly, daily, or weekly to validate the application’s validity, functionality, and stability end to end. This allows Version 1 to detect any possible errors introduced during development. BPS has also recently started automation on LPIS/GIS maps features which can be drawn on the map or the division of parcels as part of the end-to-end testing.

The BPS Team uses databases to search for cases before executing any scenario. Doing this ensures the correct scenario for the herd and the user. This is important because, depending on the role, users can perform different operations. BPS will be working on an alternative solution that uses stored data, which is calculated every day, to reduce the time needed for the initial step to find the correct herd. This method uses stored data to help users find cases for specific scenarios, on a proof-of-concept application.

A Dedicated Testing Team

DAFM has now organised a dedicated testing team. This was due to initially relying on users and the wider team to test new functionality as well as performing regression tests. As the introduction of automation was recognised as beneficial for avoiding bugs and creating reports, Version 1 implemented new functionality that enabled screenshots per step. This is to emulate the reports that users themselves are generating, resulting in 3 Amigo sessions and cucumber scenarios widely spread over DAFM. This ensures functionality with JIRA so users can document scenarios for any new functionality or bugs.

As part of the automation solution, Version 1 developed 2 approaches for users and the proof of concepts:

• Version 1 prepared a VM to project executions from Jenkins so users could customise Jenkins and see scenarios running over the VM.
• Version 1 built a scenario builder so users can select, run, or write new scenarios with existing steps and run it.

Real Differences, Delivered.

• The cost of product maintenance has been reduced, as users are alerted when a bug is found which is a major benefit to the customer in the long run.
• BDD as part of automation has helped DAFM to define requirements.
• Automation tests scenarios have helped new joiners to learn about business rules.
• Developing the automation tests as part of the development has helped to test the functionality and think about what is needed to be done before implementing anything.
• The quality and efficiency of the product is improved because test automation executes, tests and delivers results much faster than manual testing and test all possible scenarios.
• Using open-source tools for automating the application validation (such as Selenium, Cucumber, Jenkins, etc.) has been beneficial to the client in terms of cost saving on the project.
• Using automation helps to identify data that can be used to test specific scenarios.
• All these tests will make our transition easier to use new technologies for CAP 2023.