Why Companies Struggle to Innovate

I’ve seen first hand how profitable innovation can be in corporate America, but I’ve also seen companies struggle to innovate. I worked for a company that burned through millions of dollars developing an investment technology platform that everyone knew was beset by countless, insurmountable issues and destined to fail.

New frameworks--like design thinking or scaled agile--have helped companies ideate, prototype and generally innovate better; however, most companies--especially non-technology companies--still struggle. Why?

Part of it is a lack of systematic strategies, success metrics, cost controls and road maps. In my experience, however, companies don't innovate because they can't honestly address the obstacles no one talks about.

Very often, innovation requires a certain type of pain. It requires being open, honest and admitting shortcomings. It requires overcoming egos and office-politics. A recent Harvard Business Review Article points out these tensions:

• Companies encourage embracing or "tolerating failure", but few discuss that it requires zero tolerance for incompetence.

• Companies encourage "experimentation", but few discuss the rigor, discipline and strategy embedded in productive experimentation.

• Employees talk about creating psychologically safe places in which they speak truthfully, but few admit this also requires brutal candor and sometimes unvarnished criticism. This is what Ray Dalio refers to as "radical transparency."

• Everyone talks about the importance of collaboration, but not the need for individual accountability.

• Companies try to create flat organizations in which employees have wider latitude to make decisions, but don't discuss the need for strong leadership to guide strategy. This tension is a balance.

At the unfortunate extremes, true innovation may require sacrificing core beliefs. There's a part in the documentary "Print the Legend" in which a former-MakerBot Designer says:

"You often hear, especially successful entrepreneurs or CEOs say, ‘We sacrificed so much to get to this point.’ And if you’re not them or if you’ve never seen it done, you tend to believe that they are talking about, ‘Oh, we worked long nights or we didn’t see our families or it was really hard.’

When you hear someone say that after having been through a successful startup, you start to realize that those aren’t the sacrifices they are talking about. They are talking about having sacrificed who they are. They are talking about having to have made compromises with their most deep-seated beliefs, about having to cross lines that they promised themselves they would never have to cross."