Don DeLoach's CEO Blog
To be honest, I don't think I really know what I am talking about. So perhaps I should have titled this "Why I am guessing Database Schemas really matter". These thoughts are borne out of listening to people whom I think are seasoned database people, and certainly not attributable to any deep insights I have produced. My book "Lessons from a Database Savant" will have to wait. Sadly.
But I do listen. And I go to a lot of meetings. I meet a lot of people, many of them smart database people. What I have observed is that the database landscape has changed. Not many people doubt that. A long time ago, databases were hierarchical like IMS on IBM Mainframes. Then came System R and relational databases. These were row-oriented databases, and the database schemas were often developed using the concepts pioneered by the likes of Bill Inmon and Ralph Kimball. These were enterprise data models with highly normalized data (Inmon) or star schemas and snowflake schemas (Kimball). They worked well. But today there is a variety of databases - from row based, to column based, to various permutations of NoSQL databases - where the schema approach should absolutely contemplate the use case and the underlying database characteristics. An enterprise data model that works great for an Enterprise Data Warehouse for a retail store is unlikely work as well for the data mart used to store sensor readings for smart grid deployments.
The issue is sometimes people use the notion that "what worked in the past is good enough for me" mentality. That may be true when it comes to cooking ingredients or personal hygiene, but in a world that changes as fast as the technology world does, there is a penalty for not keeping abreast of changes.
We have seen people be wildly successful using our technology with schemas that were just counterintuitive to what they were use to doing. De-normalized data in "big wide" or "fat" tables with bazillions of records may just seem wrong. And yet, we see this working every day. We have also seen instances where there was an insistence on doing things "like I always did" that did more harm than good. Really. It's like buying a Ferrari for off-loading. Good car, wrong use case.
One of the biggest reasons technology fails is that it is too complicated and people don't know how to use it. Another is that it is NOT too complicated, but people refuse to use it as it as intended. The Israeli physicist and thought leader in manufacturing systems, Elijah Goldratt, once complained to me when I was attending his workshop that his biggest issue was that companies would bring in his system, OPT, and run it, then disregard the results because they were counterintuitive to what they thought should be done. He finally co-authored the book "The Goal" to explain the concept better so people would stop disregarding the results.
At Infobright, we are HUGE believers in simplicity. And the database schema definition, while usually very simple, is really important in achieving superior results. And superior results, after all, are pretty important to most everyone.
Read Comments (1)
I think there is a lot to be learned from paying close attention to what is successful. I think there is as much or more to learn from failures. When it comes to IT projects, I have been exposed to a large number of "fantastic learning opportunities." Many come to mind, but I thought I would comment (without mentioning the specific companies) about the three most egregious cases I can recall.
The first was back when I was a co-op student at GA Tech and worked for the local power company. I was basically a kid, and knew less than nothing. I thought everyone around me was a genius. For a while. There was a project called "RMS" for "Records Management System" (clever, huh?) that was going on across the hall from my office. This was to support the nuclear power plant that at the time was under construction. At first, there were about ten people doing nothing but working on this. Cool. Then there were twenty. Wow, it was like a club. Then forty. Then fifty. Lots of meetings. Several managers, who were the clear power brokers. I had no idea what was going on. Then I started to hear from some people not associated with the project that it was not going well. The people on the project laughed a bit less. The managers yelled a bit more. Then one day, after two years, they were gone. Done. Finished. No more project. As a kid, this seems just really stupid. How would a very big company filled with super-smart professional people work on a project for two years with fifty people then just shut it down? How little I knew. Welcome to the real world.
Many years later, I was deeply involved with a bank in Charlotte that was implementing a massive CRM project. Key management talked about it as if was going to boost earnings by 40%. It was pretty clear that they knew just how clever they really were. They were leading-edge, insightful leaders, ready to pounce on opportunity…. POUNCE!!! The pounce turned out to be more like the cat mistakingly jumping off the fourth-floor terrace of an apartment building. The big bang was more of an eight-digit thud. The plans had been very aggressive, very elaborate. Demos were great. Except demos are often the best representation of a system you ever see.
The last was in the mid to late 2000s, with another bank, this time in Europe. The bank had a very aggressive plan to change its core banking system. The "old one" was, well, old. It worked, but the screens were tired. The architecture was the old Tandem NonStop operating system. So old. Need new. The new system was sexy. Sexy screens. Sexy innovation. But the thing about core banking systems in large banks is that they are tied into everything. It's not like you are unscrewing a doorknob and putting a new one on. It's more like rewiring and re-plumbing the house without turning off the lights or the water. I was later told by an ex-employee of the bank that by the time they shut the project down, they had spend a whopping 138M Euro. That's million with an M. Makes the CRM miss look like a rounding error.
What do these three projects all have in common? Probably a great deal, but the characteristic that jumps off the page is that they all had highly elaborate, complex plans that involved huge numbers of people, significant budgets, and lengthy schedules. In each case, the failures were not spotted early, nor were they halted once the issues began to surface. The leadership doubled down. Money and people were thrown into these in each case to salvage what looked to be sinking ships. Yet they all sunk anyway.
Don't get me wrong. There are successful projects that are elaborate, complex, expensive efforts involving a great deal of time and people. But the more complex, the more time, and the more people you put into the mix, the more likely you are to see the project go off track. This does not even begin to explore issues around management, issues associated with the rapid changes on the business environment and the relationship between those changes and the complex plans, or issues associated with changing technology and the relationship of those changes to the projects. It also does not consider the impact of certain corporate cultural phenomenon on these projects either.
My take away? Simpler is better. Less is more. Shorter works. Do bite size projects that align with long-term objectives. Make sure everyone clearly understands the objectives. Succeed in small ways, then iterate. This is less glamourous, but it works.
The power plant got built, but way over budget. The bank put two different CRM systems in. The simpler one was a much greater success even though it was a fraction of the cost. And the core banking system project was scrapped and the old Tandem based system was extended. To my knowledge, it is still running fine to this day.
Home-run sluggers are notorious for also being strikeout leaders. That may be an acceptable tradeoff in baseball, but it is an expensive, disruptive, and more often than not, ill-advised approach to technology projects. It's like the Brad Pitt line in Moneyball, "What do we want Pete?" To which Johah Hill replies, "To get on base."
May your projects be simple and successful.
Tags: database+analytics, database+schema, infobright, machine-generated+data