CEO Blog: Does What Worked Still Work with Big Data?

One of the things I like most about my job is being surrounded by a lot of very smart people. I am forever amazed at how innovative our developers are. Even more amazing is watching what our customers build using our software. In fairness, I like looking at and learning from the competitive offerings out there as well. The Big Data industry is filled (more so each day) with very clever people and companies.

That said, what was absolutely right yesterday may not be absolutely right today or tomorrow. As technology changes, there are generally macroscopic implications that often call into question the "best practices" of yesterday. A great example of this is the approach to data models. Bill Inmon is a giant in the database and data warehousing world. No question about that. But there are those who have a religious devotion to star and snowflake schemas based on how appropriate they were for conditions yesterday that may not match the conditions of today, especially in the world of Big Data. There is a fantastic presentation by David Walker, a data warehousing consultant in the UK, that can be viewed in Slideshare by linking through this link. Dave points out, for instance, that the basic assumption is that the database being used is a row store database and not a column store, and that users will be running reporting tools and OLAP cubes to access the data. While these can sometimes be true, neither will necessarily be true. The presentation is definitely worth a look.

And the rather obvious point is that in the fast changing world of Big Data, there are implications everywhere. The business problem we are trying to solve has changed. The tools to solve the problems have changed. And the way the tools are used to solve the problems, more and more, need to change as well.

And by the way, as far as the smart people I interact with, Dave is among the best.

Regards,

Don