Who can afford to innovate in an open source world?

To put a finer point on it, consider the case of a web company infrastructure exec with whom I was chatting recently. He swore up and down that he doesn’t always want to build his own big data software, but that there’s no place to get what needs anywhere else. He wants what Cloudera Impala, Hortonworks/Apache Stinger and MapR/Apache Drill are promising. He just wants it better and, well, he wanted it yesterday.

But he also appreciates the challenge these vendors face. Their businesses require significant investments in sales, services/support and general community education, and trying to build something like a new database is really hard. Of whatever portion of the budget goes toward product development, a good portion probably goes toward improving the core products to address what their customers need.

Even if they have the budget, companies still must find a way to recoup the development costs. Hadoop is an open source technology — an Apache project — at its core, and companies pushing proprietary or even open core software aren’t always greeted with open arms. Open source software maintains the status quo, but development can be slow if you rely on a community and it can be hard to monetize.