DevOps

DevOps (a clipped compound of “software DEVelopment” and “information technology OPerationS”) is a term used to refer to a set of practices that emphasize the collaboration and communication of both software developers and information technology (IT) professionals while automating the process of software delivery and infrastructure changes.

It aims at establishing a culture and environment, where building, testing, and releasing software can happen rapidly, frequently, and more reliably.

DevOps isn’t a framework or methodology in and of itself. It doesn’t stand alone. DevOps adopts and leverages multiple frameworks and philosophies such as Agile, Lean and ITSM. DevOps is benefitted tremendously from the work the Agile community has done, showing how small teams, operating with high-trust, small batch sizes with smaller, more frequent software releases, can dramatically increase the productivity of development organizations. DevOps applies Lean principles such as increasing flow and reducing waste to the IT value stream. DevOps requires agile service management processes to remove bottlenecks and achieve faster lead and cycle times. By adopting and adapting practices from multiple frameworks we generate more productivity and economic value for the business.

This series of Blog Posts will address the various myths associated with DevOps and will attempt to address them.

Agile Testing

The Differences between Testing in Traditional and Agile Approaches

Testers must understand the differences between testing in traditional lifecycle models (e.g., sequential such as the V-model or iterative such as RUP) and Agile lifecycles in order to work effectively and efficiently. The Agile models differ in terms of the way testing and development activities are integrated, the project work products, the names, entry and exit criteria used for various levels of testing, the use of tools, and how independent testing can be effectively utilized.

Testers should remember that organizations vary considerably in their implementation of lifecycles. Deviation from the ideals of Agile lifecycles may represent intelligent customization and adaptation of the practices. The ability to adapt to the context of a given project, including the software development practices actually followed, is a key success factor for testers.

Testing and Development Activities

One of the main differences between traditional lifecycles and Agile lifecycles is the idea of very short iterations, each iteration resulting in working software that delivers features of value to business stakeholders. At the beginning of the project, there is a release planning period. This is followed by a sequence of iterations. At the beginning of each iteration, there is an iteration planning period. Once iteration scope is established, the selected user stories are developed, integrated with the system, and tested. These iterations are highly dynamic, with development, integration, and testing activities taking place throughout each iteration, and with considerable parallelism and overlap. Testing activities occur throughout the iteration, not as a final activity.

Testers, developers, and business stakeholders all have a role in testing, as with traditional lifecycles.

  • Developers perform unit tests as they develop features from the user stories.
  • Testers then test those features. Business stakeholders also test the stories during implementation.
  • Business stakeholders might use written test cases, but they also might simply experiment with and use the feature in order to provide fast feedback to the development team.

In some cases, hardening or stabilization iterations occur periodically to resolve any lingering defects and other forms of technical debt. However, the best practice is that no feature is considered done until it has been integrated and tested with the system. Another good practice is to address defects remaining from the previous iteration at the beginning of the next iteration, as part of the backlog for that iteration (referred to as “fix bugs first”).

However, some complain that this practice results in a situation where the total work to be done in the iteration is unknown and it will be more difficult to estimate when the remaining features can be done. At the end of the sequence of iterations, there can be a set of release activities to get the software ready for delivery, though in some cases delivery occurs at the end of each iteration.

When risk-based testing is used as one of the test strategies, a high-level risk analysis occurs during release planning, with testers often driving that analysis. However, the specific quality risks associated with each iteration are identified and assessed in iteration planning. This risk analysis can influence the sequence of development as well as the priority and depth of testing for the features. It also influences the estimation of the test effort required for each feature.

In some Agile practices (e.g., Extreme Programming), pairing is used. Pairing can involve testers working together in twos to test a feature. Pairing can also involve a tester working collaboratively with a developer to develop and test a feature. Pairing can be difficult when the test team is distributed, but processes and tools can help enable distributed pairing.

Testers may also serve as testing and quality coaches within the team, sharing testing knowledge and supporting quality assurance work within the team. This promotes a sense of collective ownership of quality of the product.

Test automation at all levels of testing occurs in many Agile teams, and this can mean that testers spend time creating, executing, monitoring, and maintaining automated tests and results. Because of the heavy use of test automation, a higher percentage of the manual testing on Agile projects tends to be done using experience-based and defect-based techniques such as software attacks, exploratory testing, and error guessing. While developers will focus on creating unit tests, testers should focus on creating automated integration, system, and system integration tests. This leads to a tendency for Agile teams to favour testers with a strong technical and test automation background.

One core Agile principle is that change may occur throughout the project. Therefore, lightweight work product documentation is favoured in Agile projects. Changes to existing features have testing implications, especially regression testing implications. The use of automated testing is one way of managing the amount of test effort associated with change.

However, it’s important that the rate of change not exceed the project team’s ability to deal with the risks associated with those changes.

Project Work Products

Project work products of immediate interest to Agile testers typically fall into three categories:

  • Business-oriented work products that describe what is needed (e.g., requirements specifications) and how to use it (e.g., user documentation)
  • Development work products that describe how the system is built (e.g., database entity-relationship diagrams), that actually implement the system (e.g., code), or that evaluate individual pieces of code (e.g., automated unit tests)
  • Test work products that describe how the system is tested (e.g., test strategies and plans), that actually test the system (e.g., manual and automated tests), or that present test results (e.g., test dashboards)

In a typical Agile project, it is a common practice to avoid producing vast amounts of documentation. Instead, focus is more on having working software, together with automated tests that demonstrate conformance to requirements. This encouragement to reduce documentation applies only to documentation that does not deliver value to the customer. In a successful Agile project, a balance is struck between increasing efficiency by reducing documentation and providing sufficient documentation to support business, testing, development, and maintenance activities. The team must make a decision during release planning about which work products are required and what level of work product documentation is needed.

Typical business-oriented work products on Agile projects include user stories and acceptance criteria. User stories are the Agile form of requirements specifications, and should explain how the system should behave with respect to a single, coherent feature or function.

A user story should define a feature small enough to be completed in a single iteration. Larger collections of related features, or a collection of sub-features that make up a single complex feature, may be referred to as “epics”. Epics may include user stories for different development teams. For example, one user story can describe what is required at the API-level (middleware) while another story describes what is needed at the UI-level (application). These collections may be developed over a series of sprints. Each epic and its user stories should have associated acceptance criteria.

Epics can be easily broken down into user stories by slicing them vertically and they comply to INVEST criteria. Each Epic and user story contains acceptance criteria with necessary business rules and mockups attached to them.

Typical developer work products on Agile projects include code. Agile developers also often create automated unit tests. These tests might be created after the development of code. In some cases, though, developers create tests incrementally, before each portion of the code is written, in order to provide a way of verifying, once that portion of code is written, whether it works as expected. While this approach is referred to as test first or test-driven development, in reality the tests are more a form of executable low-level design specifications rather than tests.

Typical tester work products on Agile projects include automated tests, as well as documents such as test plans, quality risk catalogs, manual tests, defect reports, and test results logs. The documents are captured in as lightweight a fashion as possible, which is often also true of these documents in traditional lifecycles. Testers will also produce test metrics from defect reports and test results logs, and again there is an emphasis on a lightweight approach.

In some Agile implementations, especially regulated, safety critical, distributed, or highly complex projects and products, further formalization of these work products is required. For example, some teams transform user stories and acceptance criteria into more formal requirements specifications. Vertical and horizontal traceability reports may be prepared to satisfy auditors, regulations, and other requirements.

In some regulated agile projects, more inspection of the documentation that is being produced during the project execution may be required.

For example: Federal drug administration (FDA) projects require lot of audit reports, strategy reports and compliance reports that need to be produced during the project execution.

Agile Transformation in An Agile Manner :: Form an Agile Team

Transforming the way of work to agile will be a one most effort consuming work your organization will take up. The team composition and roles need to be well thought out to ensure relatively smooth transformation. (The work anyway will be volatile enough!). Some of the key criterion to keep in mind while forming teams are

  1. Members who have influence – For a change that is all pervasive, it is important to choose stakeholders who can influence the change
  2. Members Who Are Excited – Having members who are excited about the change will make the initial part easier
  3. Diverse Skills Scrum is pervasive. So to adopt scrum we need changes in organizational process , technology , org structure , skill building – its essential to have people who can manage these diverse areas of impact
  4. Have a Sponsor – Choosing a sponsor who can not only influence but is also actively engaged with the team.He should be the leader who is engaged enough that he encourages others to be drive. Some of the most successful agile transformations have the sponsor as a product owner. This then ensures that his vision gets implemented. 
  5. Consider Chapter teams – Chapter teams are the horizontal teams who take up a specific item on the backlog and then drive it through the organizations. They usually end up having a sub blog for their own area of improvement.We discussed this sub log structure in earlier post
  6. “Two in a Box Coach team” At times it is helpful to have a combination of coaches – Internal and External coaches working together. External coaches are able to bring outside perspective and they are in a better position to influence senior management in their capacity as “expert consultant” . At the same time, internal coaches are better able to understand pulse of organization and carve out action items that actually work 🙂

Driving an Agile Transformation in Agile Way :: Build a Well Planned Product Backlog

Like every scrum project, an agile transformation needs to have its own Product backlog. This backlog will list all items that need to be accomplished in order to transform organizational way of work to agile.

Some Key Items to consider

  1. Consider using backlog with parent child structure – Agile transformation is a complex piece of work. In the initial stage all the items identified will be very large in size and each of those then needs to be decomposed in items that are manageable in size. Often the top level items are quite independent in nature and will be executed by different sets of people in your organization . For a recent transformation I was a part of some of the initial items in the backlog were Increase Scrum awareness ,Create a supporting organization structure that enables scrum and Increase automation . Each of these items needed to be implemented by completely different teams. Managing the backlog was becoming a difficult task and we ended up creating a sub backlog for each of the items in the parent backlog
  2. Ensure that each item has a defined/measurable acceptance criterion. For any large initiatives you will have negative stakeholders – Those stakeholders who are skeptical and don’t agree on value brought in by the transformation. It then helps to have measurable success criterion – defined and agreed upon before you start the work.
  3. Deliberate on Business value of each item – With stakeholders and team members being used to a plan driven way of operating, the product backlog transforms into action driven plan rather than a list of items where each item adds a discrete value. It helps to consciously call out the business value attached with each item. Over time this becomes a habit but it needs attention and focus initially while the team is getting used agile way of working

Driving an Agile Transformation in Agile Way :: Plan Short Sprints

As we discussed before, many of agile transformations start with a detailed, prescriptive plan mandating action items…..This a beautiful plan considering a lot of perspectives, it is rarely a short plan! This does not allow stakeholders to realize the value that is delivered. As a result after a while stakeholders, most of whom already have a lot of responsibilities start to get the dejected…

A good to way to conqueror this can be to design the transformation as a series of sprints which together deliver the large enterprise goal, but each sprint individually delivers at least a small, tangible piece of that large goal. These smaller goals allow stakeholders to taste success and retain their enthusiasm. It also helps to spread the word and convince those who are on fence.

I usually suggest that these sprints follow the scrum events and artefacts. It allows the team to collaborate in a structured manner. I have found that often stakeholders who are worried about one more topic for long meetings. They are really happy at the prospect of one short and time boxed meeting everyday. Once they see the benefits of shorter sprints , timeboxes and demonstrated values, they become the biggest supporters. In fact the opposing party stakeholder I mentioned in earlier article, went on to be the biggest agile champion in that organization.

For this short sprints to succeed its really important to have goals that can be measured and have a product backlog that is well refined. More about that in my next post…