Organisations struggle to get DevOps teams to accept application architecture guidance because DevOps methodologies conflict with application architecture methodologies. Adapting the working practices of application architecture to align with DevOps practices eases that conflict.
Key considerations
Ensure your application architects interact effectively with DevOps team members by understanding each of their individual roles.
Match the cadence of your application architecture work to that of your DevOps teams by changing your application architecture practices to provide ongoing, principle-based guidance, and embracing the feedback from this process.
Introduce architecture requirements and promote consistency by having your application architects monitor and influence the backlog of each team. Pay special attention to areas of cross-team impact such as interfaces and reuse opportunities.
DevOps friendly application architecture
Software development using agile and DevOps is a significant departure from software development using traditional practices. The practices of application architecture and of development, once well-aligned, have diverged as DevOps has gained prominence. This divergence often leads to a conflict between application architecture teams and DevOps teams, causing DevOps teams to resist application architecture guidance.
This situation cannot continue. Over half of organizations now use DevOps, and agile accounts for more development work than waterfall does. Technology leaders must understand and resolve the conflict to gain DevOps teams’ acceptance of application architecture guidance, and help to ensure the success of their organization’s DevOps initiatives.
Here we discuss three ways of achieving this (see Figure 1).
Practices for DevOps-Friendly Application Architecture
Figure 1. Three Ways to Help Ensure the Success of DevOps Initiatives
Interact Effectively by Understanding Each Team Member’s Individual Role
DevOps affects not only the way development teams work, but the roles of the individuals on the teams. Existing relationships between application architects and development team members, which are based on the responsibilities and authorities of waterfall teams, will no longer function. Application architects must understand the new roles and redefine their relationships accordingly. They must also adapt to new working styles and cultures, which can vary from team to team. In organizations that have adopted the product model, application architects may even find themselves members of a “tribe” that consists of multiple DevOps team, product management staff and technical specialists. This human-oriented challenge can be a novel one for some application architects, and introduces new demands for support of application leaders.
In traditional software development practices, application architects might interact only with development team technical leads and perhaps project managers. In a DevOps practice, a new set of roles and responsibilities means that application architects must create new working relationships. The composition of DevOps teams and the mix of roles and responsibilities can vary widely. On a typical DevOps team using Scrum, for example, there are six roles that application architects need to understand (see Figure 2).
DevOps Team Roles
Figure 2. Six Typical DevOps Team Roles to Understand
Despite the name, the ScrumMaster is not the master of the team but rather its coach, facilitator, guardian and guide. As its guardian, the ScrumMaster will aim to remove impediments that the team faces, and protect it from interruptions and disruptions. Application architects must therefore ensure that ScrumMasters don’t perceive them as a source of interference. Simply respecting the time and commitments of team members, and working with the ScrumMaster to schedule interactions, can help greatly. As facilitators and guides, ScrumMasters can be valuable allies who ensure that all voices are heard, including those of application architects.
The product owner represents the customer (often the business) and ensures the inclusion of valuable functionality in the application. They have the best understanding of the backlog items and can best help application architects to identify cross-team impacts. They also own the backlog, which must explicitly include application architecture requirements. Application leaders must therefore encourage application architects to establish trust with product owners in order to advise on and influence the backlog.
Product owners may initially perceive application architects as threats who are introducing requirements that conflict or compete with functional stories. Application architects can overcome this perception by focusing conversations on two aspects of customer satisfaction:
1. The long-term aspect — The customer must be satisfied not just with the new features the software immediately brings, but with ongoing non-functional requirements such as the reliability, availability, performance and security of the system.
2. The enterprise aspect — The software is a part of a system of systems that must interoperate to allow the customer to achieve their business objectives.
Every member of a Scrum team besides the ScrumMaster and product owner is a developer. This is not to say that they all have identical skills, competencies or experience, but rather that their place in the team is not constrained by those. Application architects may need to work with any or all of these developers, and will need to adapt their communication to each individual. While doing so, application architects can seek a developer advocate to help ensure application architecture is represented continuously on the team.
DevOps teams typically include a member concerned with the infrastructure or platform underlying the application. In some cases, they will create a specific technology stack for the application. More often, they will create an instance of a standard stack or employ a platform provided as a service in a private or public cloud. Platform staff may be reluctant or unable to make changes to the platform itself. The application architect must work with the platform staff to make the most effective use of the platform’s capabilities, often extending them rather than modifying them. In some cases, architects may lead this work themselves. They may even develop the extension along with the product team and contribute it to the platform after proving its value in production. As the platform evolves, the application architect can continue to influence it as a contributor to the platform’s development backlog.
Application architects often do not work closely with operations or support staff, but will need to develop relationships with them. Operations staff will help identify, realize and promote requirements such as reliability, maintainability, performance, resiliency, operability and security. Support staff will provide insights into the end-user experience and as-built effectiveness of the application. Closing the feedback loop from the user to the architect is one of the most valuable benefits of DevOps, so be sure to take advantage of it.
Many DevOps organizations use groups such as communities of practice, guilds and dojos to bring together people who are interested in a given subject. Application architects can participate in such groups to build relationships, to better understand the concerns and perspectives of DevOps team members, and to communicate architecture-related information.
Match Cadences by Changing Practices to Provide Ongoing, Principle-Based Guidance
Teams using agile methods such as Scrum often cite the Agile Manifesto in arguments against architecture input — particularly the principle that “The best architectures, requirements, and designs emerge from self-organizing teams.” Applications built using agile evolve to their final form over a series of iterations. Feedback from users and the lessons of the team can take the application in directions not originally envisioned. Detailed, prescriptive application architecture and comprehensive upfront design is therefore wasteful and constraining.
It is a mistake, though, to say that application architecture guidance is unnecessary. Without it, teams may take wrong turns in their journey toward the best solution that they could have avoided by following proven application architecture patterns. Worse, applications that emerge without guidance may be difficult to maintain, operate, integrate, enhance and reuse. The new role of the application architect is to provide application architecture input that helps DevOps teams recognize their application’s place in the enterprise — advocating for rather than specifying “the application in the large.”
Emergent architecture, though often seen as a barrier to deliberate architecture, is an opportunity to engage agile teams in the creation of applications that align well with your application architecture. Application architects and DevOps teams have a goal in common: to create an application that delivers value to the customer. But they also have different perspectives. Traditional agile teams tend to focus mainly on the application itself, whereas application architects are also concerned with the application’s place in the broad enterprise context, and with ensuring a consistent customer experience across a broad portfolio of applications.
Insofar as those concerns are in conflict, tension is inevitable. But it can be a healthy tension, resolved through constructive dialogue to identify the best compromise. DevOps teams, which include representation from operations, often have a broader perspective, and this can help further the case for guided design. However, application architects must be willing to dispense with preconceptions of the final application and to defer architectural decisions to the last responsible moment.
A combination of ongoing, principle-based guidance, regular feedback and emergent architecture leads to an actual application architecture that meets the needs of both agile teams and application architects (see Figure 3). It also helps application architecture principles to evolve.
Ongoing Guidance, Regular Feedback and Emergent Architecture
Figure 3. Three Ways to Meet the Needs of Application Architects and DevOps Teams
To achieve ongoing guidance, discourage application architects from providing a large volume of application architecture artifacts to DevOps teams upfront. The teams will not be ready to consume them, and may perceive them as an attempt to control the design of the application. Instead, encourage application architects to provide input a little at a time, at the DevOps team’s pace, as problems and opportunities arise in the creation of the application. This will require application architects to engage and collaborate continuously with DevOps teams.
Application architecture principles form the basis for this ongoing guidance. Application architects need to be comfortable providing guidance from that foundation. Ensure application architects thoroughly understand application architecture principles and apply them consistently. Application architects should be prepared to convey the principle-based reasons why they have introduced a given requirement or recommended use of a particular pattern. The actual application architecture then emerges through discussions between the application architect and the DevOps team.
Gather regular feedback from this process. The actual application architectures that emerge as teams create applications in this way will not only be in alignment with application architecture principles, but will often reveal new insights into those principles. Use these insights to evolve your application architecture principles and practices. Application architecture patterns such as mesh app and service architecture (MASA), for example, have appeared largely in response to agile demands.
Also consider using the agile technique of retrospectives with your application architecture teams to enhance the feedback loop.
Monitor and Influence the Backlog of Each Team
Just as the application itself changes continuously, so does the backlog of work that drives it. Shifts in business conditions, re-evaluation of the value of features and revised estimates can all contribute to additions, deletions, modifications and reprioritization. The corresponding changes to the necessary application architecture inputs can be challenging for application architects to manage, especially when those inputs have large impacts or when application architects must perform significant investigation to prepare them for the development teams. Application architects must size inputs to a granularity appropriate to the backlog, while keeping them independent and flexible.
The backlog — a continuously shifting collection of work yet to be completed — is the main driver of a DevOps team’s activity. Understanding and monitoring the backlog is key to being prepared to provide application architecture guidance when the agile team needs it. Figure 4 shows the three ways in which application architects can influence the backlog: item order, cross-team impacts and the definition of done. (Definition of done is a set of criteria that all work must meet before the work is considered complete)
Backlog-Related Areas of Application Architecture Influence
Figure 4. Three Ways Application Architects Can Influence Backlog
Ensure that application architects evaluate the application architecture aspects of each backlog item, with a focus on those closest to the top of the backlog. The aim is to prepare “just enough” and “just in time” to avoid rework if the backlog changes. Certain backlog items will have greater application architecture significance, complexity or uncertainty, and will demand that the application architect have correspondingly greater interaction with the DevOps team. Other items will present opportunities to introduce the team to application architecture concepts, to explore alternative application architectures or to refactor existing software to achieve better application architecture alignment.
Application architects should discuss with the DevOps team’s product owner how this affects the order in which they should prioritize the items in the backlog. Some product owners will create separate items for application architecture requirements, while others will address them as part of fulfilling another item. In either case, application architects must be prepared to shift and negotiate to achieve the most important application architecture requirements while sacrificing others — but must also make the DevOps team aware when this would lead to technical debt.
Application leaders should ensure that their application architecture team, as a group, watches the backlogs of all DevOps teams and has a shared understanding of the application portfolio. Application architects may also participate in or even lead meetings that bring product and platform DevOps teams together to agree upon their mutual priorities. This lets application architects recognize cross-team impacts that they can help smooth and synergies they can help leverage. These can include interfaces, user-experience consistency and reuse.
Reuse can be a particularly problematic subject for DevOps teams, because designing for reuse tends to introduce extra work that does not deliver immediate value. Application leaders can mitigate this by facilitating cross-team agreements on who will make investments in reuse, and when. One pattern is the “pay twice” model, in which a reusable service is created the second time an applicable story arises.
The increased amount of interaction with DevOps teams can place extra demand on the already tight schedules of many application architects. Certain requirements, such as regulatory requirements and security requirements, recur frequently and affect most teams. Rather than devoting time to constantly reintroducing these requirements, consider incorporating them into the definition of done. This approach works best when the DevOps teams fully understand the requirements and how to meet them. Combined with automated testing and static code analysis tools, it can reduce the need for manual reviews and time-consuming, low-value application architecture work.