Problem Statement
The client was trying to implement aggressive release capabilities, where current releases were often taking more than a year. Following challenges were identified:
- Manual testing was a bottleneck for frequent releases.
- Business knowledge residing with individuals.
- Segregated testing for respective application components.
Solution Proposed
We worked with the customer to create a test strategy for CI driven build-test-release approach, integrated with automated testing at different layers and components. To establish continuous quality engineering, tests were planned for four different environments:
- Dev environment – After developers check-in their codes, automated code quality and smoke tests are executed to ensure code stability.
- Nightly build environment – All check-in from different development teams are merged and built. The build is deployed as a nightly batch for automated regression validation. It ensured defects are found even before the functional testing.
- Release environment – The feature-complete ready for release build is deployed for this testing. Automated smoke, regression and end-to-end testing scripts are executed by the CI framework. Exploratory tests are executed for newly added features.
- Production environment – The new features are released to production and shared with selected customers for Beta testing. Final release to production for all customers is done after 2 weeks of release beta testing release.
Following are the tools used for delivering continuous quality engineering:
- Framework tools – Jenkins for CI/CD, Jfrog artifactory for artifact storage and Maven & shell scripts for build deployment.
- Testing Tools – Selenium for functional testing, Dynatrace for APM in production, Jmeter and Accunetix for non-functional testing, SonarQube for code quality.