by Ray Arell, Jens Coldewey, Israel Gat, Jorgen Hesselberg
As Agile Methods have become one of the predominant ways for successful software development, we are increasingly confronted with the question: “How can the mindset and associated benefits of Agile pervade the entire organization?” Through our management and consulting work, we have analyzed a number of successful organizations and units that have adopted an Agile management style aligned with Agile methods. Though many different practices and paradigms are used depending on the size, domain, and culture of the respective companies, we found a small but powerful set of underlying characteristics common in all of them. This article summarizes these key characteristics and briefly explains them1.
Over the last three decades, the clock speed of the business cycle has increased dramatically. Changes are taking place significantly faster than traditional product cycles and markets have become unpredictable to an extent that traditional management paradigms are unable to keep up. The software development community reacted to these changes and managed to shorten the development cycle of software down to one or two weeks through the use of Agile development methods such as Scrum or Kanban. Recent achievements, such as continuous deployment go further still, providing extremely fast and flexible reactions to ever-changing business needs. These methods remove the traditional bottleneck of software development from the business value chain and have become well-understood from a management perspective as well as from a technical viewpoint.
All Agile methods are based on the theory of complex adaptive systems, a field of major research in a number of disciplines in the last twenty years. This theory applies the principles of self-organizing teams to software development and provides an environment in which self-organization can grow. The research on complex adaptive systems has shown that these types of systems are optimal for the fast adoption of turbulent and chaotic environments.
In the last decade of implementing Agile methods in enterprises of all sizes we frequently came across two types of problems:
- Supporting business areas such as governance or HR have significant problems in dealing with Agile software development teams because these areas are often built around other, often conflicting, management paradigms.
- Business units are often unable to react to the increased speed demands and fail to turn the fast development cycles into realized business value and consequent sustainable competitive advantage.
These problems led to significant friction within the enterprises and – at best – to suboptimal utilization of the software development capabilities. As a reaction to this, the Agile community started to explore alternative organizational structures that provide the benefits of fast reaction to all parts of an enterprise.
Through our work as managers and consultants, we came across many different enterprises of all sizes that have found solutions to some or all of these problems. Some enterprises were completely built around the ideas of complex adaptive systems leading to radical new organizational approaches. Others have adopted those few areas of the enterprise that directly interface with the Agile development teams. The variety of solutions reflects the heterogeneous nature of the cultures, domains, sizes, and backgrounds of these organizations.
We found that all successful solutions share a handful of characteristics (see figure):
- They understand the system as a whole
- They adopt a catalyst style of leadership
- Their organization is based on continuous learning from experiments
- They foster an open communication style
- Their governance is based upon long-term business value and adaptation
- Their members seek mastery in their respective skills
It is important to understand that “organization” is not synonymous with “enterprise” here. In fact, only a few companies demonstrate these characteristics at enterprise level, and nearly all of these have been founded on the basis of Agile ideas. “Organizations” in the context of this article may also comprise of marketing, product management, development, operations and support for a single product or a certain business unit. What is important is their ability to support complete, end-to-end business value chains.
Let’s explore these six characteristics in more detail:
Understand the System as a Whole
People in Agile organizations take responsibility for what they do in the context of the organization as a whole. Agile organizations appreciate they are systems made up of self-aware individuals and their members understand each of their interactions may affect this system in potentially unpredictable manners.
Organizations are Complex Adaptive Systems (CAS), whether they were designed that way or not. It is one major differentiator of Agile organizations that its members understand and manage the organization as such, rather than treating it as if it were a steam machine driven by pressure and mechanic reactions.
To manage a CAS means to see and optimize all parts of the value chain from a holistic point of view with all its interrelations rather than optimizing each individual part in isolation.
Thus these organizations tend towards cross-functional teams and units or at least collaborate intensely across business functions to achieve a higher goal. They actively fight “us-and-them” mentality and invest in mutual understanding. Optimization is orchestrated across functions using collaborative techniques, such as retrospectives or kaizen meetings.
There are different approaches to managing an organization as a complex adaptive system, from Peter Senge’s “Learning Organizations” [Sen90], via Dave Snowden’s Cynefin framework [Sno00] to Glenda Eoyang’s “Human Systems Dynamics” [Eoy12]. We view each of these approaches as appropriate as guides to management of Agile organizations in the context of complex adaptive systems.
Leadership
Leadership in all levels of Agile organizations needs to adopt at least a Catalyst Leadership style. These leaders thrive by inspiring others without losing the cohesion within the entire system. They know they can trust the organization and its individual members.
Leaders in Agile organizations recognize that the critical difference between content and context is the key to success. They understand the importance of the content behind Agile methods and do their best to enhance broad knowledge of content. However, they realize that ultimately it is Context over Content. The principles and practices of Agile are viewed in the specific organizational context: needs, resources, predicaments, and constraints. They conduct themselves in a way that affects the context toward more effective assimilation of the content. They do so at any level in which they operate with the full understanding that their actions are part of numerous actions that various other Agile leaders carry out.
The key to adopting this kind of leadership is the recognition that individual standards of behavior in the local context are critical. The overall system within which the Agile leader operates might not initially be amenable to Agile principles. For example, a rigid annual budgetary process might conflict with Agile’s inspect-and-adapt practices and possibly constrict them. In this case, the Agile leader will aspire to apply inspect-and-adapt at the level, possibly micro-level, in which they can effectively apply this principle. Transforming the annual budgetary process toward inspect-and-adapt might not be possible for years. However, practicing inspect-and-adapt or any other Agile principle for that matter, apart from the standards used in the budgetary process is of tremendous importance. Being true to oneself – even in small matters such as refining the backlog – could ultimately affect the overall system by progressively moving it toward adopting Agile.
Until such multi-level alignment takes place, a gap is likely to exist between individual behavior at the micro-level and system behavior at the macro-level. The onus is on the Agile leader to minimize this gap. It is not a matter of “beating the system.” Rather, it is about reducing the gap to enable multi-level cohesion. Such cohesion can only be accomplished when the individual, organizational, and company values are harmonized. Working toward such cohesion is the ultimate objective for the Agile leader.
A fascinating paradox exists for the Agile leader. They apply Agile in order to speed up things. However, the Agile leader must possess the maturity to accept that certain changes could be very slow in coming. It might take years to reach the point in which the very same values will be simultaneously practiced at the individual, organizational, and company levels.
It takes a lot of strength to practice Agile at the individual level during a period in which it is not practiced, and might not even be recognized, at other levels. This kind of strength is the acid test for the Agile leader. Having the courage of their conviction is what ultimately leads to successful organizational transformation. Such success is not guaranteed, more often than not it takes a lot of time, and it might wear down an Agile leader who is forced to struggle for a prolonged time without witnessing immediate results. It is, however, this kind of strength that differentiates the Agile leader from the follower.
According to Bill Joiner, this behavior complies with what he calls “post-heroic leadership” with Catalyst leaders being the most frequent leadership style in this category. Post-heroic leaders focus on building the context and organization needed for a system to emerge instead of working in sometimes heroic attempts to fix the problems themselves or coordinate the actions of subordinates by detailed command and control. In most Agile organizations leadership is decoupled from roles, such as “management” or “architect”.
Following Joiner’s definition, that leadership is “taking proactive action to change something for the better”, every member of an Agile organization is entitled and expected to take leadership, at least from time to time”.
Learn
As the marketplace is always changing, Agile organizations deliver value through the process of learning. Any change in the organization is based upon continuous learning through successful and failing experiments. It happens as a sequence of learning events that combine to create paramount value, rather than executing a master plan toward a static goal.
The long-term success of any organization is based on its ability to gather knowledge and turn it into a competitive advantage. This involves not only repeating proven patterns but also gaining from the knowledge acquired when organizations fail at accomplishing a goal. Agile organizations understand that being successful all the time is impossible in uncertain environments and that it is desirable to fail fast to gain the needed experience to achieve a profitable solution. Furthermore, the knowledge obtained through both success and failure is socialized so that the value is amplified to help move the team rapidly to the solution.
To do this, Agile organizations have removed the stigma around failure and turned it into a positive experience. To limit risks, Agile organizations develop mastery in breaking big changes into a series of small, fast experiments that allow them timely learning lessons and a healthy attitude towards failure. You find a good example of this attitude in the “Valve Handbook”, a brochure a highly successful games company hands out to its new colleagues: “Whenever we move into unknown territory, our findings defy our predictions far more often than we would like to admit. We’ve found it vitally important to, whenever possible, not operate by using assumptions, unproven theories, or folk wisdom. This kind of testing takes place across our business…” [Val12, p.13]
Agile organizations have a deeper understanding of the fact that they work in a complex system that rarely can be enhanced or fixed by simple best practices. The emergent solution can only be approached by a series of act-analyze-adapt experiments that help to guide their work, and rapid feedback cycles allow the team to quickly move off a dying theoretical solution.
By reframing “failure” as “learning,” Agile organizations make an important psychological shift that creates a higher functioning and innovative mind state. “Failure” as a concept sets up the mental state of “losing,” and this creates a negative environment that closes down creative thinking. This in turn causes the organization to move slower. In contrast, “learning” sets the mental state as “growth” and helps teams move faster in a positive path toward success.
Lastly, Agile organizations understand that individual discoveries need to be socialized to amplify it to organizational knowledge. They have created a social network and other knowledge distribution systems to share good practices, solutions, and things that did not work across the company. The information is maintained by everybody, and this enables the knowledge base to stay current with the learning pace of the organization. It allows rapid search for people shopping for possible solutions and outdated information is quickly culled. The resulting amplification translates into direct savings by reducing the cost of re-inventing proven solutions and by giving people insight into solutions that did not come to fruition.
Open Environment
To have the capability to handle unexpected events, Agile organizations create vibrant physical and virtual environments that instill openness and transparency. Agile organizations are self-aware; they create and invest in communication cultures in which people collaborate and share.
Agile organizations recognize that how we communicate is more important than what we communicate. As such, they appreciate their patterns of communication and actively work to promote the type of behavior that drives collaboration and active sharing within the teams and across the organization.
Agile organizations excel in adapting to the unexpected. Narrow and efficiency-optimized communication paths, predominant in many traditional organizations, are not able to communicate unexpected information. This makes traditional organizations highly vulnerable to changes and crises. Agile organizations, therefore, have set up unstructured, multidirectional, and open communication paths, such as open space events, (micro-) blogging, or just kitchen corners that allow an open exchange of learning and ideas.
A recent Harvard Business Review article [Pen12] outlines compelling evidence for how observable patterns of communication have a measurable effect on team productivity. In a highly innovative experiment, researchers outfitted team members with electronic sensors for six weeks. As the researchers analyzed the data and identified how the team communicated with each other, it became evident that teams exhibiting patterns characterized by Energy, Engagement, and Exploration outperformed teams that did not exhibit these types of communication patterns.
In this context, the “Energy” pattern describes how team members contribute to the team as a whole. For instance, command-and-control patterns are typically very “top-down” in their style and create dominant cliques within the team. In high-performing teams, we typically see a high frequency of interaction between members – and rarely see clusters within teams. This is consistent with what we’ve identified in Agile organizations: the communication strategy targets a broad audience and encourages active feedback from associates through regular Town Hall meetings and electronic tools such as discussion boards and blogs.
The Engagement pattern illustrates how team members communicate with one another. The key insight identified through this pattern is that it’s important everyone has a ‘voice’ and that teams can share information freely with each other without being ignored or without the threat of retaliation. In many ways, the Engagement pattern addresses the relative quality of the interactions between team members and people in the organization.
Exploration refers to the way teams communicate with one another. Although communicating within the team is important, communicating across teams is just as critical to achieving meaningful alignment at the organizational level.
An interesting consequence of recognizing these patterns is that Engagement, Energy, and Exploration cannot all be high at the same time. For instance, a highly engaged team by its very nature may be less explorative while it’s working on a team-specific issue. Nevertheless, the underlying environment supporting and promoting these patterns of communication is what characterizes Agile organizations.
If there is one pattern that quickly identifies high-performing organizations, it’s Exploration. In high-functioning organizations, we typically see high levels of exploration among all teams.
Implicit in this observation is that silos are effectively torn down and organizational collaboration takes place as a natural way of doing business.
Govern & Adapt in the long term
Agile organizations think about the long term while proving results in the short term. Governance is based upon the long-term business value, not on short-term process conformance. It is based on meaningful, long-term closed feedback loops leading to effective action. Governance evolves and adapts based on the environment in which it exists.
Most government approaches are based on pre-planning and conformance. The primary tools are budgets and stage-gate processes in which intermediate results are reviewed before additional budget is granted. While these tools are essential to run traditional, Tayloristic organizations, they are in conflict with the ideas of the Agile Manifesto that states “We value … responding to change over following a plan”.
Agile organizations we have observed changed their controlling and governance approach. Instead of controlling conformance to processes and plans they focus on long-term business results on one hand and short-term learning on the other. Since Agile teams are relatively fixed in size and management has abandoned trying to fix problems by throwing more manpower on the projects, the expenses are not that much an issue anymore. So the standard controlling questions are not “How do you perform with respect to our plans?” or “How much of your budget is left?” but rather “How much value did you generate in the last month?” and “What did you learn and how do you intend to adapt to these learnings?” This aligns better to a product approach than to project thinking. In the most advanced cases, development efforts are supported as long as the products are profitable. The early launches help keep initial development costs low and thus reduce the necessity to monitor the break-even of new products.
But even if this does not work, the focus is on what actually has been produced, not what was planned to be produced. Burn-up charts are typical tools that reflect this approach. It is interesting to note that these considerations match quite well with the results of the Beyond Budgeting movement (see http://www.bbrt.org/).
Because Agile software development puts a strong emphasis on avoiding technical debt, the focus of controlling has to be on the long-term in most cases – with the most prominent exception probably being Lean Startups. It’s ok in most Agile organizations to come up with a poor short-term performance from time to time, as long as there is a good chance of a long-term payoff. Consider Google’s Slack Day for example: It sacrifices 20% of the short-term performance but plays a key role in Google’s long-term product strategy.
Craft
Members of Agile organizations take humble pride in their work and seek mastery in their skills, producing great value. Just as individuals challenge themselves, they challenge their colleagues to continuously grow and develop. The quest never ends; there is always a search for becoming better.
We’ve found that successful Agile organizations approach their work as humble craftsmen. At first glance, this term may seem contradictory so let’s take a closer look at what this means. There is enormous pride in the work Agile organizations do and the impact it has on society overall – they understand that they have a higher purpose beyond creating a commercial product. Zappos, for instance, takes enormous pride in what they call their “WOW Philosophy”, which they define simply as providing “the best customer service possible” [Ros10]. But this pride is rooted in humility: Zappos is not a company that will display braggadocio or put down competitors; they believe they are put on this earth to create a better world based on a superior approach to customer service. Simply put: Zappos do not sell shoes, it sells happiness.
Conversely, we find that successful Agile organizations demonstrate a craftsman mindset. The concept of the craftsman was first introduced during the guild traditions of medieval Europe. The idea is that in order to obtain mastery in a given craft, it is necessary to go through a series of phases in which the relative skills, knowledge and abilities of the individual increase in a progressive manner.
The craftsman mindset implies a pride in not just the “what”, but also in “how” work is conducted. Apple typically emphasizes the craftsmanship in its products – not only as a marketing story but also to reinforce the culture and pride of its people. On the other hand, companies that lose this sense of pride in how they work tend to lose their edge. Kodak, the once-dominant image company, filed for bankruptcy earlier this year not only because it was unable to recognize and respond to the sea change taking place in its industry, but because its products were widely panned as lacking in craftsmanship and innovation by analysts.
Successful Agile organizations tend to effortlessly combine these two concepts – humble pride and craftsmanship – which together form a powerful combination that create a sustainable competitive advantage. Apple and Zappos display these characteristics; as more organizations move towards an Agile mindset we’re bound to see more in the future.
Conclusion
Whether an organization is Agile is not an issue of practices or method compliance, but a matter of organizational culture. Through observing Agile organizations of many sizes and domains we have found certain characteristics their different cultures show: They are founded on a systemic approach and the belief that every organization is a complex adaptive system and should be treated that way. On this foundation, there are five areas in which Agile organizations show specific characteristics:
- They adopt a catalyst style of leadership
- Their organization is based on continuous learning from experiments
- They foster an open communication style
- Their governance is based upon long-term business value and adaptation
- Their members seek mastery in their respective skills
For an organization to be Agile, not the complete enterprise has to show these characteristics, but all parts of the enterprise that work together to provide the full value chain need to show an Agile culture to generate a competitive advantage.
This model provides both simple and powerful guidance for the quest towards a more Agile organization.
Footnotes
1 This document is a result of the Agile Enterprise Adoption Program of the Agile Alliance Non-‐Profit Organization and is based upon additional input by Diana Larsen, Esther Derby, Jutta Eckstein, Bill Joiner, Kati Villki, Jim Shore, Michael Hamman, and George Schlitz
References
[Eoy12] Glenda Eoyang: “Adaptive Action – See, Understand and Influence Organizations”, to be published
[Joi07] William Joiner, Stephen Josephs: “Leadership Agility: Five Levels of Mastery for Anticipating and Initiating Change”, Jossey-Bass, 2007
[Pen12] Alex Pentland: “The New Science of Building Great Teams”, Harvard Business Review, April 2012
[Ros10] Steven Rosenblaum: “The Happiness Mission: Zappos isn’t a Company, it’s A Mission” Fast Company, June 2010. (http://www.fastcompany.com/1657030/the- happiness-culture-zappos-isn-t-a-company-it-s-a-mission)
[Sen90] Peter M. Senge: “The Fifth Discipline: The Art & Practice of The Learning Organization”, Doubleday/Currency, 1990, ISBN 0-385-26094-6
[Sno00] David Snowden: “Cynefin, A Sense of Time and Place: an Ecological Approach to Sense Making and Learning in Formal and Informal Communities” conference proceedings of KMAC at the University of Aston, July 2000
[Val12] “Valve Handbook for New Employees – A fearless adventure in knowing what to do when no one’s there telling you what to do”, Valve Press 2012, http://cdn.flamehaus.com/Valve_Handbook_LowRes.pdf
