The team behind Siri have a new idea: a voice-controlled personal assistant – linked to all your devices – that will take care of your every need

So I’ve arrived late at the office of Viv, an artificial intelligence company based in San Jose, California. I missed my train from San Francisco after dawdling leaving my apartment and then finding the taxi service app on my phone wouldn’t work. Dag Kittlaus, who I’ve kept waiting, looks on the bright side. “Your trials of getting here are a perfect illustration of how Viv will be helpful,” he says. “Wouldn’t it be nice to say ‘I need to get to San Jose, give me my options’ and Viv would know how close you are to the train station, when the next train is coming, where the nearest cars, how much it was going to cost…”

Kittlaus is the co-founder and CEO of Viv, a three-year-old AI startup backed by $30m, including funds from Iconiq Capital, which helps manage the fortunes of Mark Zuckerberg and other wealthy tech executives. In a blocky office building in San Jose’s downtown, the company is working on what Kittlaus describes as a “global brain” – a new form of voice-controlled virtual personal assistant. With the odd flashes of personality, Viv will be able to perform thousands of tasks, and it won’t just be stuck in a phone but integrated into everything from fridges to cars. “Tell Viv what you want and it will orchestrate this massive network of services that will take care of it,” he says.

We collectively decided Siri was only chapter one of a much bigger, longer story Dag Kittlaus

It is an ambitious project but Kittlaus isn’t without a track record. The last company he co-founded invented Siri, the original virtual assistant now standard in Apple products. Siri Inc was acquired by the tech giant for a reported $200m in 2010. The inclusion of the Siri software in the iPhone in 2011 introduced the world to a new way to interact with a mobile device. Google and Microsoft soon followed with their versions. More recently they have been joined by Amazon, with the Echo you can talk to, and Facebook, with its experimental virtual assistant, M.

But, Kittlaus says, all these virtual assistants he helped birth are limited in their capabilities. Enter Viv. “What happens when you have a system that is 10,000 times more capable?” he asks. “It will shift the economics of the internet.”

Matthew Wong, a research analyst at CB Insights, which tracks private investment in the tech sector, says the virtual assistant is an area where the big companies are currently doing battle. The tech giants are hungry to acquire new technology to make their products more sophisticated because there is money to be made in automating everyday processes. If they can pull it off we will spend more time with our phones – which means more ad revenue or device sales. It makes it a potentially lucrative area for startups. “Companies are hearing from Facebooks and Googles as soon as they announce what they are doing,” says Wong. Wit.ai, which turns speech and text into actionable data, had only existed a little over a year when it was acquired by Facebook in January 2015 for an undisclosed sum. Its technology has helped build M.

“We are going to be talking to our computers,” says Oren Etzioni, who heads the Seattle-based Allen Institute for Artificial Intelligence, set up by Microsoft co-founder Paul Allen, adding that this includes phones. “But the truth is, all the devices that exist today are short on actual ability compared with the kind of capabilities we want.” The future, as Etzioni sees it, belongs to the company that can make a personal assistant something like a good hotel concierge: someone you can have a sophisticated dialogue with, get high quality recommendations from and who will then take care of every aspect of booking an evening out for you.

Facebook Twitter Pinterest Dag Kittlaus, co-founder and CEO of Viv. Photograph: Erik Unger

Viv’s ancestor Siri was created after Siri Inc spun out of Silicon Valley non-profit research lab SRI International in 2008. Kittlaus had been working on the early mobile internet when, in 2007, he was hired by SRI International as an entrepreneur-in-residence. It put him in close contact with software engineer Adam Cheyer, who was leading the Calo project (Cognitive Assistant that Learns and Organises), a $200m, five-year AI effort to help soldiers, funded by the Pentagon’s research agency, Darpa. Kittlaus convinced Cheyer that Siri was perfect for mobile phones and they honed the idea, building a prototype before adding another technical co-founder, Tom Gruber. The iPhone app Siri launched in early 2010.

Out of tinted windows through a line of slender palm trees touched by the late afternoon sun, Kittlaus points out the building that was home to Siri Inc. He recalls the day he nearly missed a call from Steve Jobs, Apple’s then CEO, because he couldn’t get the slide bar on his iPhone to work. “He said, ‘We love what you are doing. Can you come over to my house tomorrow and talk to me about it?’” says Kittlaus. The co-founders joined Jobs for more than three hours the next day. “We sold it to Apple because Steve sat us down and said, ‘Let’s change the way the world works together’, and that was pretty irresistible,” he says.

For the next year and a half, Kittlaus ran the Apple team preparing Siri for the iPhone, with Cheyer and Gruber the engineering leads. The day after it launched – and the day Steve Jobs died – Kittlaus left for family reasons. Cheyer stayed for another nine months before following Kittlaus (Gruber is still there).

Goodbye privacy, hello Alexa: here's to Amazon echo, the home robot who hears it all Read more

The idea for Viv, the Latin root for “life”, came around mid-2012 as Kittlaus, Cheyer and another Siri Inc software engineer, Chris Brigham, who had left Apple around the same time as Cheyer, were brainstorming what to do next. It was Brigham, frustrated with the way existing personal assistants were constrained both technically and for business reasons, who had the initial insight for Viv. “We collectively decided Siri was only chapter one of a much bigger, longer story,” says Kittlaus.

Kittlaus pulls out his phone to demonstrate a prototype (he won’t say when Viv will launch but intimates that 2016 will be a big year). “I need a ride to the nearest pediatrician in San Jose,” he says to the phone. It produces a list of pediatricians sorted by distance and with their ratings courtesy of online doctor-booking service ZocDoc. Kittlaus taps one and the phone shows how far away the Uber is that could come and collect him. “If I click, there is going to be a car on the way,” he says. “See how those services just work together.”

He moves on to another example. “Send my mom a dozen yellow roses.” Viv can combine information in his contact list – where he has tagged his mother – with the services of an online florist that delivers across the US. Others requests Kittlaus says Viv will be able to accomplish include “On the way to my brother’s house, I need to pick up some good wine that goes well with lasagne” and “Find me a place to take my kids in the last week of March in the Caribbean”. Later I test out how well both Siri and Google’s virtual assistant perform on these examples. Neither gets far.

Viv can be different because it is being designed to be totally open, says Kittlaus. Any service, product or knowledge that any company or individual wants to imbue with a speaking component can be plugged into the network to work together with the others already in there. (Dozens of companies, from Uber to Florist One, are in the prototype). Other virtual assistants are essentially closed. Apple and only Apple, for example, decides what capabilities get integrated into Siri.

Viv’s biggest secret is the technology to bring the different services together on the fly to respond to requests for which it hasn’t been specifically programmed. “It is a program that writes its own program, which is the only way you can scale thousands of services working together that know nothing about one another,” says Kittlaus. Other personal assistants generally have their responses programmed by a developer. They are, essentially, scripted. There was no choice but to do things differently, says Kittlaus. To think of every combination of things that could be asked would be impossible. Viv will also include elements of learning; it will adapt as it comes to know your preferences.

Hey, Siri! Meet the real people behind Apple's voice activated assistant Read more

Expect a phone app initially, says Kittlaus, but the loftier ambition is to incorporate Viv into all manner of devices, including cars. He imagines Viv’s icon becoming ubiquitous. “Anywhere you see it will mean you can talk to that thing,” he says. Of course this will require time: for companies to volunteer their services, and for users to come on board. But Kittlaus says some of the world’s largest consumer electronics companies are “very interested in plugging in”.

Viv has the potential to upend internet economics, says Kittlaus. Companies currently spend billions to advertise online with Google, and much traffic arrives based on web users’ keyword searches. But if instead requests are directed at Viv, it would cut out the middleman. The team are still exploring different business models, but one involves charging a processing fee on top of every transaction.

Etzioni at the Allen Institute was impressed by an early demo of Viv and applauds the company’s vision to bring about a paradigm shift in the capabilities of virtual assistants. “They are hunting big game and if there was any team that I would bet on to do that it is them,” he says. But he also adds the jury is still out as to whether Viv will achieve its ambitions. “It is a high-risk enterprise,” he says.

Etzioni singles out Viv’s ability to go beyond simple utterances and engage in a sophisticated two-way dialogue as a particularly hard AI problem. Viv might, drawing on a review site, suggest a “good” bottle of wine that can be obtained on a particular route, but users are bound to want complex clarifications. Is the bottle within their price range? “There are so many nuances, you want something that handles those,” he says.

Viv also faces competition not only from the internet giants upgrading their personal assistants but from other startups with their own takes on the theme. Will Viv, as was the case with Siri, sell out to one of the tech titans? Kittlaus isn’t ruling anything in or out. “Our goal is to be everywhere, we want ubiquity… We are not going to say there is a certain path to that.” Either way, I for one will be happy just to get to San Jose on time. Bring it on, Viv.

Facebook Twitter Pinterest Apple’s Siri, the original personal assistant. Photograph: Getty Images

The competition: today’s virtual PAs

Name: Siri

Company: Apple

Communication: Voice

The original personal assistant, launched on the iPhone in 2011 and incorporated into many Apple products. Siri can answer questions, send messages, place calls, make dinner reservations through OpenTable and more.

Name: Google Now

Company: Google

Communication: Voice and typing

Available through the Google app or Chrome browser, capabilities include answering questions, getting directions and creating reminders. It also proactively delivers information to users that it predicts they might want, such as traffic conditions during commutes.

Name: Cortana

Company: Microsoft

Communication: Voice

Built into Microsoft phones and Windows 10, Cortana will help you find things on your PC, manage your calendar and track packages. It also tells jokes.

Name: Alexa

Company: Amazon

Communication: Voice

Embedded inside Amazon’s Echo, the cylindrical speaker device that went on general sale in June 2015 in the US. Call on Alexa to stream music, give cooking assistance and reorder Amazon items.

Name: M

Company: Facebook

Communication: Typing

Released in August 2015 as a pilot and integrated into Facebook Messenger, M supports sophisticated interactions but behind the scenes relies on both artificial intelligence and humans to fulfil requests, though the idea is that eventually it will know enough to operate on its own.