"What kind of projects are suitable for Agile?"
"We've got to be careful in choosing projects that we want to be Agile."
At the onset I must make it clear that a big component of Agile is continuous improvement. That said, generally speaking improvement can usually be made most of the times; especially if Agile is not yet an established approach within an organization. The reason is simple, while most people focus on the structure of Agile and appreciate its aesthetics and lightweight nature, good Agilists focus more on its soffits; rigor, discipline, Socratic approach to problem solving, etc.
Relatively small and/or web-based projects have been experimenting with Agile for quite a while. The knowledge base in the Agile community on what works and what doesn't with such projects is rich. The landscape turns dismal on some other fronts, primarily due to lack of experimentation. In some instances a whole section of software community has been insulated from Agile. A prominent case study is Siebel (more on that in a later post). The idea in this post is to discuss some of the environments that are still a frontier for Agile. Without further adieu here's where Agile is struggling:
Big Opaque Software Frameworks: Ownership, access, and control over the code being written is vital for software teams. If developers can't modify a code base to suit their needs and hook other 3rd party tools into it, they can't really derive a fundamentally sound and quality product out of it. Basic practices that contribute to stability and quality of software like Unit Testing are hampered due to inability to touch this code base and, again, lead to a poor quality product. Lack of unit testing in turn, usually, lowers the confidence level of developers who want to refactor the code. Thus they avoid it , which leads to inefficient designs.
Shared and Aggregated Environments: Rolling software changes into environments and databases that affect organization wide operations are not too well suited for Agile development on their guts. Some changes are impossible to make and other very expensive. An example would be a common environment shared between inventory management, inventory ordering, and inventory finance and accounting systems. A spaghetti like this with myriad of inbuilt interdependencies usually warrants heavy frontloaded analysis. Regulatory oversight only makes the matter worse. e.g. a change made to an Oracle module once made might be difficult to roll back due to financial and regulatory restrictions. Often the Big Opaque Software Frameworks (above) almost mandate a heavy analysis and design phase. Big ERP and CRM applications fall into this category.
Software Hardware Boundary: This is the most exciting and promising challenge that Agile will have for a few years to come. Traditional heavy equipment and machinery manufacturers that utilize large scale control systems are getting comfortable with Agile in their software shops. Some of them are trying to take Agile out to their manufacturing plants and experimenting with how to make the mechanical design teams work with software design teams. Agile has not yet addressed the complementarity in software and hardware design. How can hardware design keep up with the software as it is evolving and vice versa? They inherently seem to have not just different characteristics of work cycles but also skill sets and personality traits in their ranks. Agile approach to software is largely unknown in the manufacturing world. It's a challenge of lag, lack of experience; hence the frontier.
Untamed Team Distribution: Lack of interaction between software development teams and business stakeholders adversely impacts the feedback cycle and affects quality of the product. Development velocity takes a hit and ultimately feedback cycles lengthen to a point where rework increases and impacts velocity. Agile leaders in the industry have figured out approaches to make Agile work in distributed environments but only when the teams distributions are purposefully designed, i.e. they have the luxury to choose the people on the teams and their physical location or have a say on the rotation schedule of people between locations and teams. The persuasion to form teams with individuals that are dispersed between multiple locations across different timezones and are "staffed" on projects because they happen to be available might backfire and lead to woefully suboptimal value. Designing for Agility is a resource management challenge, and therefore a frontier.
Systematizing Agile: A natural propensity once Agile has delivered a few success stories is to systematize it in an organization. This urge for "systematization" is detrimental to Agile's success. Agile is not Coca Cola; each implementation is different and should be different to suite the needs of an individual project's needs. Systematization and templatization curbs the "inspect and adapt" nature of Agile. A challenge, therefore, emerges. How can Agile be institutionalized such that the necessary functions within corporate like budgeting and accounting can still run efficiently, smoothly, and cost effectively? How can financial planners ensure optimal use of available resources and timing of investments? This is not to say that in the traditional world financial planners are blessed to have projects executed as they had planned. Projects are extended, teams underdeliver, and expenses are all over the place. Not to mention that the OpEx CapEx ratios are much different than initially planned. The challenge therefore is to build accounting communication and data-churning tools that can react to the realities of projects on the ground. Agile, I think, catapults this necessity to forefront. However, Agilists haven't yet built the tools or reporting mechanism, to the best of my knowledge, that addresses these specific needs of financial planners.
Agile Scalability: 60-70% capitalization rates in software development industry are not unheard of. They are rare, though. On an average this rate is around 20%. Since software capitalization rate is far less (or even none due to short spans between technical feasibility and actual development) in Agile it essentially becomes unattractive for corporation to even think of agile transformation across the portfolio without getting creative and aggressive about it. That's a risk, I don't think anybody is willing to take. The outdated (in context of Agile software development) FASB edicts pose a huge barrier to scaling Agility in large software development environments.
Talent: With the growing adoption of Agile comes a sad realization. It's still the people that make the difference, as they always have. A common misunderstanding in the community is that it's the lightweight process and use of innovative tools and techniques (e.g. TDD) that make Agile successful. If Agile is as much about the people as it is about tools, then we have a significant problem at hand. Scaling Agile within an organization suddenly seems to be a herculean task. Who are these people that "get" Agile and where can we find them?
Now some would argue that regulation intensive environments are not conducive for Agile. True. But that's all there is to it, it's not conducive but Agile is works in many such cases.
This list is a work in progress. As I encounter other challenges you'll definitely see them reflected here. Change is an eternal process and I hope that the next time someone is thinking of making a change in any of these frontier environments (s)he considers Agile as a creative process that teaches us to confront novelty and improvise as opposed to a new "process" or a "methodology" that comes with its own peculiarities of Do's and Don'ts.
Image take from: http://antarcticsun.usap.gov/aroundTheContinent/images/pole_plane.jpg