Apple’s Greg Joswiak: ‘It’s absolutely incredible what people are already doing in so little time’.

Imagine your child reading a bedtime story on an iPad while watching fairies play out the story dancing around their real-life bedroom. Or a young boy viewing a NASA rocket taking off into outer space from his real backyard.

They’re among experiences that augmented reality offers through mixing real and imaginary worlds.

Last year Pokemon Go triggered global mayhem with 45 million daily users chasing critters at its peak. Now augmented reality (AR) will offer new experiences regularly on iPhone and iPad following Apple’s release of AR coding tools to its developer community last month.

In four weeks they’ve created many imaginative applications. Documented on @madewithARKit they include a virtual tape measure that you can stretch across your iPhone camera image to measure anything.

Others show a robot dancing around a real living room, virtual racing cars on real roadways, tiny players dribbling a basketball on a computer table, a Space X rocket landing in a real backyard, and a game where you defend your office against descending space invaders.

Commercially there’s iKea’s app that virtually places showroom furniture in your lounge room at correct scale. You can see if it fits and matches. Interior design could be one major use.

It’s absolutely incredible what people are already doing in so little time, but that’s the point of it,” says Greg “Joz” Joswiak, Apple vice president of product marketing.

In an interview with The Australian, Mr Joswiak said Apple saw the commercial value of AR ahead of virtual reality, and would stick to offering AR on iPhone and iPad, rather than through a headset, at least for now.

“The fact we have a billion of these devices out there is quite an opportunity for developers. Overnight, when iOS11 ships, it will become the largest AR platform in the world, with hundreds of millions of devices that are enabled for developers to take advantage of. We’ve got 14 million registered app developers.”

Apple’s bullishness about AR is fuelling speculation that the 10th anniversary iPhone 8 will have new sensors including 3D sensing to enhance its AR capability.

HomePod

Apple is late to bring a personal assistant to market. In the US, Amazon Echo and Google Home are well established but HomePod is only rolling out in December. That includes in Australia.

HomePod looks like a late-in-market decision, but Mr Joswiak said Apple wanted first to develop a secure platform for HomeKit and to get vendors supplying HomeKit devices signed up before going to market.

“When we finally went customer facing, which was a year ago, and unveiled it and started shipping it last September, there were devices for people to actually use,” Mr Joswiak said.

Maybe so in the US, but are there enough HomeKit certified devices in Australia for HomePod to link to?

Mr Joswiak said Apple’s worldwide developer group is tasked with making sure that happens. “They go out and evangelise in a particular market.”

He said HomePod was primarily a music player. “We’re pretty clear in our positioning of it. It is first and foremost an unbelievable music product, an unbelievable speaker system. It is much better than a Sonos Play 3, there’s really no comparison to the (Amazon) Echo which is more like a radio.”

But will you be able to use HomePod to connect to other bot services, for example to order an Uber or a Pizza simply by asking as you can with Home and Echo? .

“First off, home pod supports your HomeKit devices. One of its advantages is its connected to a network, connected to the internet, means that you can use it remotely.

We haven’t got into specifics yet. There’s a lot more of our story that will emerge as we get closer to December.”

Apple HomePod on show at the 2017 Apple Worldwide Developer Conference in San Jose.

Machine Learning

Artificial intelligence and machine learning are dominant technologies in 2017, so dominant that they’ve become buzzwords. But Mr Joswiak said Apple was using machine learning “long before it was fashionable to call it machine learning”.

“There was so much machine learning in the original iPhone, even in the keyboard. The hitzone on the keys as you were typing were actually changing as you typed. Even back in 2007.”

“One of the things we’re always proud of is we just didn’t have apps running blindly in the background, consuming your battery.

“We had technology that was based on machine learning. If every morning you launch The Australian and read your news on your phone, and didn’t do it the rest of the day, we wouldn’t leave it running in the background.

“But every morning we would have fired it up knowing, again, if you’re using machine learning, what time you are likely to do that.” He said this featured had operated for years.

When iOS11 rolls out later this year, Apple will include autocorrections in messages and mail based on what you’ve just been reading as a machine learning enhancement.

In Notes, Apple will use machine learning to interpret even “atrocious handwriting” made with Apple Pencil, he says.

“You can actually search for what you wrote in handwriting. We index that and allow you to search it.”

The right side of the new extended dock also features machine learning. It shows apps you frequently use. “But those aren’t just blindly suggested,” he said. Mr Joswiak gives the example of himself, an IMDb junkie. “When I watch television at night, it’s always where is that actor from, what were they in before? Every evening, when I pull down my suggested apps, IMDb is there waiting for me. It knows that I’m very likely to want IMDb in the evening.”

Apple Watch

Apple Watch

Apple rejects any notion that Apple Watch is not enjoying traction, or that the smartwatch form factor has limited appeal. “We were never disappointed at how the Apple Watch is doing because we know these are businesses that grow over time,” he said.

“This last year, Apple Watch has done phenomenally well, growing at an unbelievably high rate, and we continue to make it better.

“Now we’ve introduced features in watchOS 4 that bring more intelligence to your watch, a Siri watch face that serves up information to you about what you need next, easier and more seamless workouts, and more coaching.”

A new gym kit feature will let people exchange watch information with gym equipment, such as heart rate information and steps.

Mr Joswiak said this will work well in Australia. “The GymKit suppliers that we announced will start rolling equipment out as soon the (US) Fall and represent about 80 per cent market share of Australian gym equipment.”