A Strategic Tool to Enable Velocity and Quality
By: Sidharth Sabat & Ankush Rathore
Part1: Test Principles, Test Practices and Test Operating Model
Executive Overview
Over the last couple of years, Microsoft Industry Solution Delivery (ISD) has on an average observed the percentage mix of migrated applications types at 50% for Lift and Shift (Rehost), 30% for Workload Migration (Re-platform and Refactor) and 20% for Clean Deployment (Rearchitect and Rebuild).
However, the percentage mix of the applications for an organization is based on several criteria’s like Application Portfolio Complexity (Complex, Medium and Simple), Level of Functional Automation, Cloud Operational Maturity and the overarching Compliance and Security readiness and requirements.
A recent Large Transformation and Migration program undertaken by ISD, reveled the percentage mix to be heavily skewed towards Workload Migration. It was observed the Workload Migration stood at almost 95% as compared to the Lift and Shift Migrations which constituted a little less than 5% of the App portfolio. This case study serves as the premise for this whitepaper, and aims at sharing key Testing Principles, Practices, and the Operating Model to achieve Velocity and Quality.
Background
Migrating applications to cloud brings-in unique challenges in terms of technology and management. And, when the migration involves workloads for a large organizations where the no. of workloads runs into hundreds and thousands the challenges grow multifold due to additional complexities of alignment, communication, managing multiple stakeholders, navigating the organization ecosystem and other dependencies.
Although a cloud migration effort involves changes to servers and other components related to infrastructure and minimal configuration update to the application, the business value is derived based on how the effectively and efficiently the application works on the cloud. To drive the business value, every migration program needs to build comprehensive cloud migration test strategy targeting the unique needs of the program.
The approach elaborated below is not “The Only Approach” recommended by ISD, however the authors are keen to share the key practices and principles with the community which proved to hit the sweet spot in achieving program objectives of quality migration at scale.
There is no silver bullet or a one size fits all when it comes to defining a test strategy to ensure the (1) quality of the migration process, (2) migrating applications to cloud at scale and (3) ensuring smooth operations on the cloud. The program team needs to carefully weigh in the objectives and challenges to channelize the effort in building a strategy to address majority of the objectives and challenges.
Migration to cloud in-principle involves multiple applications belonging to multiple teams within an organization. These application teams are generally accustomed to an existing process and often are resistant to change. Hence building an overarching testing strategy for the entire program ensures, a consistent and repeatable process is followed across different application and platform teams whose applications are a part of the migration journey to the cloud.
It is quite natural for an application team to push for a comprehensive testing of their application prior to production deployment, however simultaneously it directly challenges the program velocity. The balance here lies in identifying and testing ‘Just Enough’ and ‘Just In Time’ across all areas to aid velocity and bringing in efficiency in handovers between dependent teams to achieve leanness in operation and desired velocity.
Test Principles
The balance between testing and migration velocity can be achieved by driving focus on below principles:
- Repeatable Use across various Application teams
- Migration of legacy workloads to Azure cloud involves applications from various app teams, who are accustomed to using different software development methodologies. It is of utmost importance to have a consistent and repeatable testing process to align application team and the migration team to the program objective.
- Lightweight Testing
- Migrating workloads to Azure cloud from different application team simultaneously requires huge co-ordination between various platform, security, compliance and operation team to meet compliance needs, hence keeping the testing process lightweight brings the required optimization to the program
- Risk Based Approach
- Migration of workloads to Azure cloud from on-prem hardly requires any change to the application functionality, hence following a risk based approach to identify minimum yet fit to purpose test scenarios can ensure meeting the quality goals with minimum effort.
- Minimum effort to operationalize
- While defining the test process it is necessary to employ expertise to bring in minimum variance to the testing process in order to keep the effort required for acceptance and incorporation by the application and platform teams
Test Practices
Listed below are the recommended key testing practices which should be considered for building the test operating model (discussed in next section) for creating an overarching test strategy across migration workloads:
- Practice #1:
- Align test strategy across migration treatment-types i.e. Rehost, Re-platform and Refactor
- Practice #2:
- Fit to purpose testing to aid migration of scale. Identify 20% of the critical scenarios for testing to minimize risk and optimize speed.
- Practice #3:
- Categorize test types into mandatory and non-mandatory testing
- Practice #4:
- Build decision trees for non-mandatory test types for faster decision making
- Practice #5:
- Build lean questionnaire to gather early information from application and platform team to identify test types and scope
- Practice #6:
- Templatization of test artifacts to ensure consistency and establishing a rhythm in evidencing quality goals met for application cutover
- Practice #7:
- Build decision trees to determine the involvement of dependent teams like test data masking, service virtualization and environment determination
Test Operating Model
The Test Operating Model brings consistency to the migration journey and helps in the alignment of the different applications team to the program objectives. The Operating Model can be categorized into four major stream of work as follows:
Readiness:
Focusses on activities related to discovery of current testing capabilities for the application under consideration and determine a high level testing scope
- Participate in Discovery Interview
- Co-ordinate collection and storing of test information via test questionnaire
- Upload and maintain high level understanding prior to application migration plan
- Participation in the design and planning discussions
- Identify scope of testing
- Engage dependent team (across Application, Middleware and Infrastructure Dependencies)
- Governance:
Focusses on coaching application teams, standardization of test artifacts, removing blockers, monitor progress, quality assurance and process improvement
- Co-ordinate planning activities with respective application team
- Perform lightweight assessment to determine non-mandatory tests inclusion
- Review the test plan
- Coach application teams on test process
- Monitor progress and govern planned test activities
- Provide resources to address resource constraints within corresponding application team
- Gather feedback and work towards improvement of the process and standardization of test artifacts
- Delivery
Focusses on activities required to complete test planning, test execution, test reporting and test evidencing on appropriate environment for application cutover
- Finalize, review and approve the test plan
- Align the testing timeline to the migration window provided
- Complete test case are uploaded to the testing tools
- Execute test cases on non-prod environment
- Defect management, Test Status Report and Evidencing
- Test execution on the prod environment during cutover
- Test execution of any multi-cloud test scenarios post cutover.
- Dependent Service
Focusses primarily on the dependent activities involving teams which need to be engaged based on application requirement. Listed below are few of the dependent services which are common across industries however these may vary based on the organizational environment
- Involve test data masking team to endorse and provide the masking scripts based on the application requirement
- Involve service virtualization team to stub integration point in case the application cannot be integrating with the systems to conduct integration testing
- Involve release team in case the application needs integration with the real time systems to conduct testing.
- Involve required security team to provide guidance, conduct security scans and security testing to ensure the compliance need for the assets.
Summary
This whitepaper is the first in series of few proposed publications, and was drafted with an objective to share the Test Principles, Test Practices and Test Operating Model that served as Key Enablers for Velocity and Quality in the referential Case Study, and helped in driving business synergies across various applications and Infrastructure teams. The authors are equally keen to learn and hear feedback from the Community on what other principles and practices worked for their programs.
