Why appliance power matters in data warehousing

Analysts “buzz” about speed-of-thought analytics. One observer noted that if the bandwidth of the human mind could be measured in technical terms, the sheer volume of information processed at any given second—including visual, tactile, olfactory, audio and taste, all integrated in real-time context to constitute an “experience” that can be recalled and replayed—would require the fastest, most powerful machines imagined by mankind, all running faster than the speed of light.

How does a common computing experience purport to keep up with this kind of bandwidth? Well, the mind has a response-latency that works in favor of such efforts, to the effect that an action is always faster than a human reaction. The Kung Fu television series was famous for the master saying “snatch the pebbles from my hand.” This is more of a testimony to the master’s speed of reflex than the apprentice’s speed of the snatch, because the snatch always has the upper hand, so to speak. In Kung Fu, the student’s reflexes are trained to respond faster than his mind can think about what to do.

In the case of speedy analytics, the machine has the upper hand in its speed of delivery. The human mind needs time to assimilate, respond, refactor and retry. While this cycle might take minutes, this usually occurs in mere seconds. The analyst is ready to strike again within moments of receiving a response. In fact, if these iterations continue, the analyst may achieve an “Einsteinian” immersion experience, dilating time itself. Hours pass like minutes. They will forgo lunch, snacks and perhaps forget to pick up their children from school.

How disappointing must it be to have the analyst’s readiness thwarted by slow technology? A high-powered data warehousing (DW) appliance can support the analyst’s needs. Why then do so many DW professionals settle for slow response times? Moreover, why do they embrace workarounds rather than learning to use the machine well?

Workarounds appear in many forms. If the database truly is slow (for example, a transactional platform) then the environment may see a proliferation of large-scale extracts. Where is all this data going? To Excel spreadsheets, cubes, caches and so on. The vendors of these technologies have streamlined the user experience with a wide range of jazzy capabilities that allow the user to navigate and drill on the information. It’s understood that the data is a limited subset and that it grows stale by the minute. The users will then chant a constant refrain of “refresh” as often as possible. This is when we see the large-scale extracts proliferate further, with greater frequency.

For reporting environments that use traditional databases, this scenario is so common that the developers of the various consuming technologies will, by default, install one of them with the full intention of leveraging the cubes and caches without even considering that they may be completely superfluous when standing in the shadow of a high-powered DW appliance. Still others recognize the machine’s power and fully intend to getting rid of the cubes at their first opportunity, but find that the users cannot be weaned from the functionality the cube supports.

In the discussion above, something isn’t so obvious: the lack of power in the core DW platform shapes the culture of those who depend upon it. This is why power matters, and why lack of power can build a toxic culture. After all, who really wants to wait an hour or two for a data refresh, and who wants to support those who tolerate it? The user screams for the IT staff to make the extracts faster, ignoring the more obvious truth: they shouldn’t be extracting in the first place.

This is reminiscent of one site I visited long ago where the administrative teams were on virtual roller skates moving from system to system to fight this fire or that. Rather than stabilize the systems, which would require considerably more power than their installed hardware, they had chosen, in an abstract sense, to standardize on a roller-skating culture. Newbies were introduced to the ways of navigating around the shallow power base and the many nuances of the workarounds.

The lost opportunity in all this is that when a powerful DW appliance is introduced, it doesn’t fit into the culture at all. In fact, toxic cultures like this could turn the platform into nothing more than a storage location; it’s the equivalent of chaining a racehorse to a plow.

However, a sophisticated DW appliance can be a powerful cultural change agent. Rather than being completely overwhelmed by a toxic culture, its mere presence starts to change the culture in a very positive way.

Explore the power of DW appliances, such as IBM PureData System for Analytics incorporating IBM Netezza technology, in a hybrid, fluid data architecture.

Follow @IBMBigData