Big Data analytics techniques play a role in CloudPhysics' instrumentation data-sensing results analysis. Here is the second part of Willem ter Harmsel's interview (click here for part one) with Cloud Physics CEO John Blumenthal, who explains how Cloud Physics takes its cues from Google and LinkedIn.

WtH: Is CloudPhysics still limited to VMware, or are OpenStack and Microsoft Hyper-V in the picture already?

John Blumenthal: We are exclusively focused on vSphere customers today. We are in the throes of making a decision what hypervisor to support next, whether it will be either KVM and Openstack or Hyper-V. We should be making that decision in the next month.

WtH: The VMware-specific experience in your team must be crucial to the non-disruptive collection ‘secret-sauce’. This experience cannot be leveraged in Hyper-V or KVM environments, right?

There is very specific ‘secret sauce’ that we filed patents around that go beyond just the specific nature of our expertise in collections out of VMware environments. The collection techniques have direct application to other hypervisors, independent of the nature of how you would collect.

WtH: So there is an hypervisor-independent part of the "secret sauce", but still you will need to hire experienced Hyper-V or KVM engineers?

John Blumenthal: Absolutely, expertise on those interfaces is a specialised skillset and we absolutely invest in those.

WtH: More generally now, the whole vague term of ‘Big Data’ frustrates me a bit. What is your view on what people call ‘Big Data’ today?

John Blumenthal: I share your frustration and I think it is an overused, hyped term. What is big today may be small tomorrow.

Our vision around it and the labels we’ve used is really derived from Google and other large-scale infrastructures. If you look at how Linkedin or Google run their infrastructures, what you quickly realise [is] they have data collections, heavy instrumentations that are collected and put onto an operational platform for running analytics to actually then operate the infrastructure.

The platform that collects all that data internal to these large backend system is in fact the same thing we are doing for VMware environments.

What we are doing is mimicking the type of analytical approach that is being used to run the world’s largest infrastructures in the world and bringing that approach to VMware users.

In that sense, large amounts of data are absolutely required for analysing configurations, performance. Using real data science techniques for understanding optimisations as well as risks.

Unfortunately that falls under the marketing term of Big Data, which is overused. We believe it is essential to taking IT infrastructure to the new level. We are giving people that are running VMware a chance to run at the same type of utilisation as the people at Facebook, Linkedin and Google.

WtH: Are the data-collection and analysis techniques that you are applying to IT Infrastructure today, applicable to completely different types of datasets?

John Blumenthal: Actually this is one area where I am going to have to disagree, Willem. The lesson we are learning from operating in the Big Data space, is that the platform that you construct to manage and process the data and be able to model it and turn back causation, you actually have to tie that very specifically to the very data that you are collecting.

We believe that the first rule of business in IT Operations management and analytics is "Know thy data".

If you look at the history of many of the system management platforms, the underlying algorithms that were developed that drive a lot of these things were never really constructed with the intention of running IT Operations.

Examples of this are Netuitive and Integrien [which were] acquired by VMware and some of the things IBM has been running as a part of Tivoli. The algorithms driving the processing of the data were never constructed from the ground up with domain expertise around what the data contains and how the data is actually modelled.

One of the fundamental premises at CloudPhysics is that our backend is very specific to the processing at scale of systems management data, where the core four resources of Network, CPU, Memory and Storage are related in a way that virtualisation domain expertise can only do.

It’s that that drives the data structures, how we manage it on the backend and how we process it algorithmically. We believe this approach is just fundamentally different [from] the way other systems management approaches have been assembled using fundamental algorithms that [were] never really designed for systems management.

Therein lies the difference in the quality of the type of analytics that we can generate.

WtH: That approach makes a lot of sense. So the opposite hypothesis I proposed is something that some of the competitors are doing?

John Blumenthal: Yes, you’d see elements of exactly what you’re saying with Splunk for example. Their platform is restricted to unstructured log data that can be indexed and then searched. Splunk’s ambition is to do this not just across IT but also the internet of things where any log is subject to their type of indexing and searching.

I believe that is valid, I just don’t believe it adds to the ability to find causation in systems management data.

WtH: All right. See, you helped to lift the veil on Big Data, the smoke is slowly clearing... Thanks for that!

John Blumenthal: I’m glad!

WtH: Back to the team you have now, how big is CloudPhysics now?

John Blumenthal: We’re 25 now and growing, most of the team is engineering. We have recently attracted a first class VP of Marketing, Melinda Wilken, who also spent time at VMware and was most recently at Couchbase. Melinda is designing much of marketing and messaging ... right now.

So we’re filling out the business side of the company as we speak. We’re doing this in concert with the onboarding of our first users.

WtH: Diane Greene and Mendel Rosenblum are investors in your company: how involved are they?

John Blumenthal: Well Mendel and Diane are some of the best possible investors in virtualisation and enterprise software. They are very actively involved across a whole suite of companies. I am very lucky to get access to Diane in the course of looking at the operations and strategy of the company. She has been very gracious with her time; we get access to her as much as she can afford it. It is a great relationship. We’ve all spend time at VMware under her tenure and we’re not only benefiting from that experience but also from her ongoing interest in the company.

WtH: How involved are Kleiner Perkins and the Mayfield Fund?

John Blumenthal: They are very involved. Our latest round was led by Kleiner Perkins’ Mike Abott, who was the VP of Engineering at Twitter. At the strategic level, Mike Abott and Robin Vasan of the Mayfield Fund provide great guidance and strategy, and they help to draw talent to the team. The latter is especially essential in a competitive labor market like Silicon Valley’s. They really help us there.

WtH: VMware has various departments like NSX that are largely separate from the rest of the organization. Could you have imagined yourself running CloudPhysics from within VMware?

John Blumenthal: At the time that I was at VMware, no. I was there until the end of 2011 and until that time the entire focus has been on on-premise, proprietary licensable software.

That is very different from what we’re doing which is much more SaaS oriented and analytically-driven as opposed to feature-driven which is how the Hypervisor on which I was working was developed.

At the time VMware did not have the genetic make-up to build a SaaS oriented approach to the world. I think that is changing at VMware now. What we’re doing now was not possible inside VMware at the time. It is really critical to find investors that understand the domain and the stakes deeply, otherwise you end up in a bad situation of misalignment.

WtH: Looking back at your career at Symantec and VMware and now CloudPhysics, what has been central to your drive?

John Blumenthal: You have to fall in love with a problem. You have to find a problem that keeps you up at night and that drives you almost obsessively.

You combine that with a certain level of grit and the desire to see something through. For me, it’s really that primitive.

I think in start-ups, especially taking on venture money, you really need to find something that you’re passionate and obsessed about solving. I think that if you do find that, people in your domain will be attracted to it and will jump on board trying to solve something that is a harder problem and that is compelling.

Compared to my time at VMware, the experience at CloudPhysics is highly intensified now. We’re a small company, you have employees that need to share in the drive to be successful. It is not for the faint of heart and it is extremely exciting.

It is a tremendous experience. Seeing your idea in reality is great, I wouldn’t trade it for anything. ®