Apple is designing a television that you can shout and gesticulate at – and it will understand you.

Having revolutionised the music and mobile phone industries with its iPod and iPhone, the company is planning an "iTV" to turn couch surfing into a hi-tech experience.

The Japanese firm Sharp has been asked to begin commercial production of Apple TV screens in February, with the sets available in the second half of 2012, according to analyst Peter Misek at the American bank Jefferies.

"Other TV manufacturers have begun a scrambling search to identify what iTV will be and do," Misek claimed. "They hope to avoid the fate of other industries and manufacturers who were caught flat-footed by Apple."

An internet-connected TV offering seamless links to cloud services to download films – as well as a new level of interactivity – could threaten other manufacturers as well as taking viewers away from pay-TV giants such as BSkyB. Senior engineer Jeff Robbin, who built the iTunes service and helped to create the iPod, is reported to be overseeing the project. Speculation reached new heights when the TV project was mentioned in a biography published soon after the death of Apple's founder Steve Jobs last month.

Jobs told his biographer: "I'd like to create an integrated television set that is completely easy to use. It would be seamlessly synced with all of your devices and with iCloud. It will have the simplest user interface you could imagine. I finally cracked it."

In October the US patent office published a filing by Apple for "real time video process control using gestures", which describes using infrared, motion and other sensors to read the user's movements. Gestures could edit video, or "throw" it from one device to another, say to transfer a film from a mobile phone to a bigger screen.

The patents mention facial recognition, to tag individuals within videos so that home videos can be grouped according to who features in them. This autumn the iPhone was updated with a highly accurate voice control application called Siri, and some observers believe it will feature on the TV sets.

What sounds like science fiction is already in use by niche products. Chinese manufacturer Hisense unveiled an internet TV last month which runs on Google's Android software and allows the viewer to issue commands with a wave of the hand.

Microsoft's Kinect, which links to the Xbox games console to TV sets, acts as a digital video recorder and reads voice and gesture commands. Users can rewind, fast forward, call up menus, or select games and channels without having to find the remote control.

Thanks to four microphones, it separates the user's voice from other noises in the room and users must say "Xbox" before speaking a command.

The venture is a risky one for Apple, whose previous foray into the world of television resulted in a rare failure. Apple TV, introduced in 2007, is a box that can store video and connect the TV to the internet or to the laptop to view photos. However, its latest model is thought to have sold no more than 2m units, compared to 40m iPads since March 2010.

Apple TV already incorporates some gesture commands, using the iPhone as a remote: users can flick it left or right or drag two fingers across its screen to fast forward or rewind, and tap to play.

Misek says the iTV could have an initial production run of 5m to 10m units, and will use liquid crystal displays. The company declined to comment.

Apple has taken over an entire Sharp factory to make its latest generations of phones and tablet computers, which will appear next year with brighter, less battery-draining screens. It is estimated to have spent $500m to $1bn buying manufacturing equipment for the plant, and retooling of a production line is believed to be under way to produce TV screens.

Technology researcher Benedict Evans at Enders Analysis was sceptical about revolutionary an iTV could be. He said that accidental gestures could disrupt viewing, adding: "It would be like sitting in Sothebys and desperately not moving so you don't accidentally make a bid for £10m."

The project will open a new front in Apple's battle with the South Korean consumer electronics giant Samsung, which already makes internet-connected TV sets and has been competing against the US group to produce high-end tablet computers and smartphones. The two are locked in an international patent disputes, with some 20 court cases around the world.