Why Agile Software Development isn't More Prevalent

I believe that agile software development leads to better software faster and therefore cheaper, and I think there's a lot of evidence to support that. So why, some seven years after the Agile Manifesto and many years after people started to do agile development, aren't the practices more common?

One factor that I think can't be underestimated is that agile pushes software developers and their managers out of their comfort zones way too much. Most development managers are more comfortable providing estimates up front and then reporting for most of the project that, "everything seems to be on track." At the end of the release cycle you have a period of discomfort, but that discomfort is mitigated by all the effort being put into getting the team to "pull it off."

Compare that to having to say early in the development cycle that, "we can already see that we're not going to get all the features in time, so do you want to drop features, and which ones, or do we move out the date?" That's way more uncomfortable because it's likely to be met by, "No to both." Then what do you do?

For developers, it's much more comfortable to just code what the Business Analyst told you to code, without having to address whether the user will deal with it. Moreover, why would a programmer want to produce code faster and cheaper? They get paid no matter how much work they do. Why would they want to even measure speed, as that could lead to being compared with their colleagues? It's much more comfortable to not even look at how fast the programmers are going.

Of course, successful agile organizations address these points. If you're looking at trying to make a non-agile organization more agile, you need to consider how really big a task you're setting yourself. It's hard to get people to change if they're not motivated to change, and the people who have to change to become more agile are actually motivated to do quite the opposite.