Revision History

17 June 2016

Expanded on the use of OK Google

Refined discussion of Focus Speech Audio

Introduction



If you read this blog on a regular basis, you’ll already know that I enjoy playing with new technology. Since one of my roles here at Mosen Consulting is to train people in the use of technology, I can even justify playing with new gadgets and calling it work.



Three years ago, I bought a Nexus 7 tablet. I became broadly familiar with the Android operating system in conjunction with TalkBack, Android’s official screen reader for the platform. I then left it neglected, because I didn’t consider it an environment that was either pleasurable to use or that could meet my productivity requirements.



A lot of the apps that shipped on the device were inaccessible, and I had a lot of trouble getting gestures to register. I got tired pretty quickly of performing a double-thump, just to perform an action.



Several factors led me to pick up a new device and take a fresh look at the Android experience.



First, three years is a long time in technology. The operating system has matured and TalkBack developers have responded to feedback.



Second, using some of Google’s apps on my iDevices has given me huge respect for the user experience they are creating. Google Now, Voice Search, YouTube and Maps are great tools. It made me curious to learn more about what the native Android experience might be like, unconstrained by Apple’s sandbox approach.



Third, while it appears that there may be a way with the iPhone 7 series to charge the phone while using wired headphones, the headphone jack controversy got me thinking about the compromises I have made in exchange for the very good accessibility experience iOS offers. The many books, podcasts, email messages and blog posts I’ve put together since I bought my first iDevice are a testimony to the fact that iOS and VoiceOver have changed my life beyond measure. Yet there are still things I can’t do with my iDevices that I was doing with my old Nokia phones running Symbian all the way back in 2009.



Apple’s sandbox approach is designed to keep our phones safe and protect our privacy. All in all, it succeeds very well with this objective. But experienced users pay the price in terms of flexibility. Even though I consider myself a proficient iTunes user, and except for when Apple breaks accessibility I can do what I need to do with it, I still resent that I can’t cable my own phone to my own computer, see it as a drive, copy music to a music folder, and play that music in any number of apps. If needed, I want to be able to delete music from my phone in the same way.



I want to do simple things like assign a piece of music as a ring tone, without having to go through a bunch of hoops to have it contain a certain file extension and get it imported in just the right kind of way.



I miss the ability to run certain kinds of apps like call recorders, in those situations where being able to record a quick podcast interview when I’m mobile would be useful.



In short, there are a bunch of ways in which Apple limits my ability to do what I want with the device I paid for. I’m sure they’d say that it’s for my own good, but as an experienced computer user, I think I’m the best judge of what’s good for me.



Admittedly, through the use of various application programming interfaces, they’ve backed off a little bit over the last few years as Android has become the dominant player. But the operating system itself is fundamentally locked down unless you jailbreak. Jailbreaking is harder to do as Apple closes the exploits that make it possible. It also eliminates the possibility of participating in Apple’s beta programme.



So philosophically, as someone who likes to tinker and customise, my heart is with the ethos of Android. But as the old saying goes, philosophy bakes no bread. As blind people, we’re constantly battling ignorance and discrimination. We owe it to ourselves to be as productive and efficient as we can be with our technology. So I’ll go with the option that allows me to get as much work done when I’m away from my computer as possible.



With that background in mind, this is my account of what it was like to spend some quality time with a current Android device. This blog post is no more than one blind guy’s experiences, and it’s written from the perspective of an end user, designed to be digestible and helpful to end users. Any review like this is going to be subjective. It’s influenced by the things I do most with my phone. At least some of them will be different from the things you do with yours. The post has a lot to say about TalkBack as a screen reader, but accessibility is a means to an end. The end is being able to use the device and applications you choose. So I’ll also spend some time talking about the user experience I’ve had getting some tasks done.



This post has been reviewed by expert Android users, because while it contains my personal opinions, I want as much as possible to avoid errors of fact. There may be some though, so I encourage you to read any comments left in response.



Even though I’ve tried not to let this colour my review too much, of course it’s based on the fact that I’ve been using iDevices for years.



While some blind people, annoyed with those who point out Android’s shortcomings, would claim that it isn’t fair or appropriate to compare TalkBack and VoiceOver, I couldn’t disagree more. When a sighted person looks at purchasing a smartphone, they’ll compare the two platforms and the way things are done on each. I agree that TalkBack doesn’t have to do things in the way that VoiceOver does, but the two screen readers should be compared based on how efficiently a blind person can get the job done, and the number of apps that work well with it. So I’m not going to shy away from drawing comparisons in this post.



Choosing a Device



Choosing an Android device can be both liberating and confusing. Not only do you have to consider how much storage you might need and the size of the screen as you do when purchasing an iPhone, but Android gives you a vast array of options from which to choose, from many manufacturers.



In the end, I decided to purchase a Huawei Nexus 6P. It’s a phablet, about the same size as my iPhone 6s Plus. I bought a Nexus device because they are produced by original equipment manufacturers to Google’s specifications. They run Android as Google intended it to be, without added apps or modifications.



The price that many Android users have to pay for a wide range of devices and user experiences is fragmentation. Operating system updates can either take months to appear on some devices, or they may never appear at all. Five months after the release of Android 6.0 Marshmallow, it was only running on 2.3% of devices.



When you buy a Nexus device, you can be assured that you’ll be one of the first to receive updates to the operating system. As someone who likes to be on the cutting edge and try new things, that appealed to me, particularly given that accessibility seems to be improving steadily with every release.



Indeed, I briefly upgraded my Nexus 6P to the Android N preview, until I realised that beta testing an operating system and screen reader with which I was not intimately familiar was a bit too much to take on at once. Upgrading and downgrading was a snap. Simply enrol in the beta, and the update is pushed to your device. Unenroll, and you’re given a software downgrade and have to start over. But let’s go back to the beginning.



Set-up



Once I got the Nexus 6P home from the store, I was unable to complete the initial set-up process without sighted assistance. I’ve set up a number of Android-based devices without issue in the past, the most recent of which was the Kindle Fire I bought a couple of months ago, so I’m very familiar with the process. You should be able to power up the unit, and when at the initial set-up screen, hold two fingers down on the screen. Despite having sighted assistance on-hand to confirm that the initial set-up screen was indeed being displayed, the gesture to start an accessible set-up did absolutely nothing. I have since learned from some other Nexus 6P users that they have had a similar issue, and that it may have been corrected in newer versions of the software. That seems indeed to be the case. I erased my phone after applying software updates and started over. This time the accessibility gesture worked. After holding two-fingers down on the screen for a couple of seconds, a highly intelligible text-to-speech engine prompted me to keep my two fingers held down until I heard a beep. TalkBack then allowed me to complete the rest of the phone’s set-up independently. However, when I unboxed it, the only thing I could do was get sighted assistance to complete the set-up as well as start TalkBack from the device’s Accessibility Settings. Not the best of starts.



If you are able to complete the setup of your Android device independently using TalkBack, be prepared to have a set of headphones on-hand. TalkBack with the default Google keyboard will not speak passwords if you’re using the phone’s built-in speaker. While I appreciate the intentions behind this feature in that a blind person may not know who is around them when they’re typing in sensitive password data, this restriction seems a little arbitrary in a way that doesn’t sit well with Android’s philosophy of greater user flexibility and responsibility. Forcing the user to have a pair of wired headphones on-hand in an era where some may not own any could cause real issues getting the device configured. It seems to me a rather pointless measure anyway, since the device is not smart enough to know what’s at the end of that 3.5 mm cable connected to the headphone jack. Sure, most people will connect headphones, but they could just as equally plug in a big wired speaker and blast the password to the neighbours. In the end, we as blind people know when we’re in an appropriate environment to set up a device. Another work-around if you don’t have headphones handy is to skip the Wi-Fi setup, enable the speaking of passwords in accessibility settings, then connect to a wireless network. I recommend enabling this setting anyway, because when a password field is populated and the speaking of passwords has not been enabled, TalkBack doesn’t tell you the number of characters that the field contains.



Yet another option is to install one of the many accessible third-party keyboards that don’t feature this password restriction, but of course you have to set up the device before you can do that.



TalkBack guides a new user through a familiarisation process by way of a tutorial, in which you’re introduced to the gestures and given a chance to practice them. A form of this tutorial has been available for some years, but it’s structure and language have become increasingly comprehensive and friendly. It’s a nice touch. For me, it’s benefit at start-up was somewhat lessened by the fact that an older version of TalkBack shipped with my device, and the gesture set of the update has changed in some key respects. This is difficult to avoid if a product is evolving, and it’s better to have this issue than be stuck with a product that is stagnant. I was impressed that after eventually applying the update from the Google Play Store, I received a notification telling me that gestures had changed. And you can go back and run the tutorial at any time should you need to practice.



Many other core Android apps, such as Contacts, Gmail and even the default keyboard are individual apps in themselves and can receive updates through the Play Store. This is a different approach from the one Apple takes, where many of its core apps are built into the OS. It’s a great strategy, particularly in an environment like Android where there’s so much OS fragmentation. Even if your device manufacturer takes an age to update you to a newer version of Android, at least you can grab the latest screen reading technology and new core apps that are available. It’s worth keeping in mind though that some accessibility improvements may be dependent on changes at the operating system level.



Part of the setup process involved registering my fingerprint with the sensor located on the back of the phone. This was totally accessible, giving me feedback all the way through and clearly showing me the various security options available.



Initial Impressions



As I immersed myself in Android, I took some notes about things that stood out for me. Some of these issues have accessibility ramifications, while others are the observations of an experienced iOS user having a play, and are not blindness-specific.



Enormous Improvement with On-board apps



There was a time, not so long ago, where you couldn’t enjoy a truly accessible experience on Android until you replaced or enhanced many of the default applications and features with more accessible alternatives. That’s fun for the geeks among us, but for those who just want to get on with using their new phone and may not be too tech-savvy, it was a steep hill to climb. Compared with the last time I took a serious look at Android, it’s like night and day.



I haven’t played with every feature of every app, but I’ve opened most of the stock apps and explored a little.



Google Maps works well, although I miss the ability to explore the screen with my finger and get a feel for the layout of streets and intersections as I can in iOS. Nevertheless, it’s a snap to get directions and other information. Transit information isn’t available through Apple Maps in New Zealand, so the accurate information I get from an app so intrinsic to the operating system is a welcome change for me.



I found it easy to navigate the calendar and add appointments.



The Gmail app is useable, but in my view not terribly efficient. I appreciate though that efficiency can be subjective, and some people may consider it adequate. I was given the tip to install and use a third-party app called Aqua Mail. Since I have so many email accounts to manage, I had to pay for the premium version, but it’s one of the best mobile email clients I’ve used. It even supports Imap push, which means I’m able to be even more responsive than with my iPhone. It’s a beautiful thing.



Google’s default keyboard is now accessible. Some people will appreciate the haptic feedback, which gives you the impression that you’re getting some traction from the virtual keyboard as you type. Typing is similar to touch typing on the iPhone, in that you slide your finger around the screen, lifting your finger when you find the character you wish to enter. If you prefer what iOS calls “standard typing”, where you must double-tap or split-tap a key, then you’ll need another keyboard. There are plenty of these in the Play Store. It seems to me that there are more accessible keyboards on Android than there are in iOS, possibly because Android has had third-party keyboards for much longer.



My Nexus 6P comes with Google’s Messenger app. It feels to me a lot like the iOS Messages app. It’s accessible and a pleasure to use.



I got up and running with Play Music from Google without any trouble, taking advantage of the free trial offered to all nexus purchasers.



Surfing the web with the latest build of Chrome is now truly useable, and the granularity features in TalkBack make it easy to navigate by elements you need when on a web page. I found myself missing reader mode in Safari though, which helps a blind person to get past the clutter and onto the important content.



In short, long gone are the days when you’ll get an Android device out of the box and throw up your hands in horror. I understand that Google has accessibility champions for all of their major product lines now, demonstrating a real recognition of their need to step up to the plate and ensure that accessibility is a part of their DNA.



Consistency



The Nexus 6P has no physical home button. I’ve not found this a problem at all, since it remains an icon that is visible on the screen at the same place at all times. But if you have issues with locating the Home icon, there is a TalkBack gesture, which I actually find more cumbersome than locating the button. More on that when I get to discussing TalkBack.



To the right of the Home icon is the Overview Button. This is a little like the App Switcher in iOS, and shows you apps and other items you’ve used recently. From here, you can check how much RAM and energy an app is using and how much storage it consumes. You can close the app, and uninstall it if you like. If you know your way around a computer, you’ll appreciate all of this information being so close to hand.



To the left of the Home icon is the Back button, a feature I like very much in Android. Some iOS apps have a back button, and some do not. Some implement a back button inconsistently, so that it’s available in only some parts of the app. The Back button is always available, without exception in Android. It’s a function of the operating system that is kind of like the Back button in a web browser. Pressing it repeatedly retraces your steps until you’re at the Home screen, which is the top level.



As someone trying to come up to speed, I found this consistency very helpful.



A Smart Home Screen



If you’re a user of newer versions of Windows, you’ll be familiar with the concept of active tiles, which may display news, weather, and other information that changes. In iOS, widgets are available, but they’re tucked away in the Today view, leaving your Home Screen a static grid of app icons where the only thing that changes is the badge count. Widgets are tiny applets that display useful information.



Android allows you to jazz up your home screen with a combination of apps and widgets. If you like the weather visible right on page one of your Home Screen, it’s doable. If, like me, you work a lot with currencies other than your local one, you can put exchange rates right there. There are thousands of these things, so one needs to be a bit selective or you’ll get information overload. Widgets certainly make you feel like you have plenty of information literally right at your fingertips.



Your home screen can be even more useful thanks to the ability to add a contact right to it. Third-party apps make this possible on iOS with more effort, but in Android, it’s a feature that’s part of the OS. This could be particularly useful for less tech-savvy users who may need a small set of contacts they can reach in emergency situations.



Feels like Home



Product pricing is always a tricky business for companies serving many markets. Some of us do the numbers on the cost of these devices, and feel that when the current exchange rate is being taken into account, we’re not getting a fair deal. This has been a common complaint with the pricing of iPhone in New Zealand, but what makes it all the more irksome is that a number of flagship features aren’t available here. So we’re paying more for less.



Apple has not seen fit to make its News app available in New Zealand, but Google’s is and it seems to work very well. It takes advantage of the well-established web-based Google News service, which mines articles from the web and can be customised to your preferences. Being Google, it gets better over time at understanding what you like to read.



Similarly, Google’s weather information in New Zealand is more accurate, because they’ve taken the time to integrate with New Zealand’s Met Service, the official provider of weather information here.



As already mentioned, transit directions work here whereas they don’t in Apple Maps.



Google’s Voice Assistant knows much more about local things than does the present implementation of Siri, including rugby and cricket information, something I’m sure my readers in Australia, the UK and a number of other countries will appreciate.



Apple Pay isn’t available in New Zealand, and nor is Android Pay at the moment. However, the fact that the Near Field Communications chip on Android phones isn’t locked down to the hardware or OS manufacturer means that alternative payment solutions can be used.



In short, my Android device seems more aware and more capable of serving me in my location.



While I’m on the subject of NFC, the openness of the technology on Android lends itself to some awesome applications. We own a couple of UE Meggaboom Bluetooth speakers, which are NFC-aware. All I had to do to get the speakers paired with this Android device was to touch the two devices together in the right place. Very impressive.



Our bus system here is also NFC-enabled with a supported SIM, so you’ve paid for your bus trip just by getting on the bus with your Android phone.



The Play Store



I’ve enjoyed using the Play Store, Google’s App Store equivalent, very much. There are two areas where it has really stood out for me.



First, while the Store experience is fully accessible on the device itself, it’s also fully accessible via any browser. It’s a pleasure to use Firefox with JAWS to explore the Store, search for specific apps, and then nominate a specific device to which I want to send the app. If the device is switched off, all of the requests will be queued for when the device is next on and connected to the Internet.



Yes, a similar function is available through iTunes, but I find a browser-based experience more speedy and pleasant.



The second thing I like about the Play Store is so beneficial to anyone with accessibility needs that it gets a “hey wow” award. If you buy an app and find it to be inaccessible, you can press a button within two hours of making the purchase that fully refunds you, no questions asked. This has been extended from an initial limit of 15 minutes. Two hours is ample to determine whether the app is fully accessible, completely unusable or somewhere in between, and if there are issues, whether you’d like to chance your luck on trying to get the developer to make some changes.



Text-To-Speech



I’ve been a very happy camper while conducting this evaluation, because I’ve had Eloquence on my phone. You can install a range of voices onto your device, and any app can hook into those voices. It’s elegant, and it works. You can also purchase the full range of Nuance iVocalizer voices.



The voices I purchased from Code Factory all offer a feature many of us have been asking for in iOS for a long time – a pronunciation dictionary. The pronunciation of unusual words can vary widely between text-to-speech engines, so there are advantages in having the ability to change pronunciation on an engine-by-engine basis. It would still be useful at times to have a pronunciation dictionary in the screen reader itself, when you want to make global changes.



Presently, it’s not possible to change pronunciation when using Android’s default text-to-speech.



It’s a little thing, but I do appreciate that when I’ve set my language to a non-American version of English, the text-to-speech will use words like “full stop” instead of “period”.



Voice Commands in Google Now



When Google was established, it was all about search. So it’s not surprising that Google does a mind-blowing job of responding, often with remarkable precision, to specific questions. Ask Siri a question, and while it will sometimes give you a specific answer, it will more frequently tell you, “I’ve found something on the web, take a look”.



Google Now also integrates with third-party applications, vastly extending the feature’s capabilities. If I tell Google Now to take a note, I get prompted for the name of the app I want to use.



I can tell Google Now to play a specific clip on YouTube, or an artist in Google Play Music.



If I say “Send a WhatsApp Message to Bonnie”, that’s all it takes.



Just as with Siri, you can post on Twitter or Facebook by voice.



I found Google Now to be snappy in issuing its responses, and highly accurate with dictation. The latter is hard to quantify and it may just be wishful thinking on my part, but it’s backed up by some studies which suggest that Google has a higher accuracy rate.



I have, however, come away from this process with a new appreciation of Siri. First, I can issue the “Hey Siri” command, or hold down the Home button, from anywhere in my iPhone. The process of configuring “Hey Siri” to respond to my voice is completely accessible.

For some but not all users, Google Now can be launched system-wide, or even when the phone is locked if you choose to configure it that way. There are two problems.

First, the process is not easy for a blind person to set up in currently shipping versions of Android, although it is vastly improved in Android N, currently in beta. To get “OK Google” to work system-wide, you currently have to quit or suspend TalkBack before invoking the screen where the configuration choices are located, then start or resume it again. Compared to the simple, fully accessible “Hey Siri” process, it’s not a good experience at all and will put many people off configuring “OK Google” for global use.

Second, while I can use “Hey Siri” with my language set to New Zealand English, the OK Google feature is disabled for me altogether in Android because of my language choice. This is unusual, since I have found overall that Google generally supports New Zealand well. Global OK Google is a rare exception.

Google Now’s ability to control system functions is lacklustre compared with Siri’s. With Google Now, I can toggle off Wi-Fi and Bluetooth, but not cellular data. I can add an appointment or reminder, but I can’t change or delete one. I can adjust the brightness, but can’t turn on or off do not disturb.



Most important of all, I can’t turn TalkBack on or off, despite Google clearly knowing what I want. If I say “OK Google, turn TalkBack on”, I get taken to accessibility settings. That’s not very helpful, since I’m blind and have no idea where to perform the two taps that would enable TalkBack for me.



Chromecast



I took another foray into the world of Google with the purchase of a Chromecast Audio device. It’s a low-cost dongle, configurable via an Android or iOS app, that connects to your Wi-Fi. It has a range of audio output jacks so you can connect it to wired speakers, a rack system, a Sonos CONNECT etc. Once set up, you can beam audio from your device to the Chromecast Audio.



There’s also a full audio and video version, that plugs into an HDMI port.



In iOS, functionality is limited because specific apps have to support the technology. But in Android, Chromecast support is baked into the operating system. It’s a great experience, and it is superior to Apple’s AirPlay technology in one important respect. In iOS, unless you’re using an iTunes in the Cloud service, your iDevice is responsible for sending the entire TV show, movie or podcast from the iDevice to the device receiving the AirPlay signal. Your enjoyment of the content can be affected by Wi-Fi glitches, crashes, or incoming calls. It can also be a battery drain.



Once you start “casting” something, your mobile device is taken out of the mix. Chromecast takes over the streaming. You can make calls, play other content on your phone, even shut it down altogether if you want to, and the content just keeps on playing on the device you’re casting to.



Chromecast is also fully multi-room aware.



I came to have a real appreciation for how well the technology functions on Android. It just works.



Third-party apps



I’ve briefly tested over 200 apps in the time I’ve been working with my Nexus 6P. Some of them were recommended by experienced Android users, and some were Android versions of apps I have and like on my iPhone.



Almost all of the blindness apps I tried operated similarly to their iOS counterparts. I had excellent results with KNFB Reader, finding it to be accurate and fast. I appreciated being able to take a few pics without having to pay for the app again, having plonked down the cash for the iOS version already.



TapTapSee also didn’t disappoint.



I worked with a couple of currency identification apps specific to Android that performed their function well.



Sadly, BlindSquare isn’t available for Android and I missed having it around.



Voice Dream Reader exists on Android, although it is doing such clever things with gestures for VoiceOver users in iOS that I noticed how less efficient it was to use on Android. That’s not at all a criticism of the developer, but I think more of a reflection on the constraints when developing. I suppose if the app were completely self-voicing and took TalkBack out of the loop altogether, a similar experience might be possible.



Dice World, whose developers have shown outstanding commitment to accessibility, works very well on Android, although there isn’t as much customisation of what the app speaks as there is in iOS. I have heard from some Samsung Galaxy users that some Samsung Games features need to be disabled to get Dice World to work correctly.



In terms of general apps, a very small number worked better for me on Android than on iOS. The one that stood out was the Healthmate app for my Withings Smart Body Analyser which I’ve blogged about previously. There were far fewer unlabelled buttons than on iOS, and the app was much more pleasurable to use.



Some apps were similar in accessibility between the operating systems. Apps like The Guardian, NPR News, and BBC News were very good.



Android seems well-served by the major social media and messenger apps. It seemed to me that I did a little more swiping around in Facebook for Android than I do in iOS, but it was accessible and I was able to use it without issue.



The official Twitter app works well, although just like its Mac equivalent, I found no way of stopping it from verbalising both the full name and the Twitter username in every tweet. I would prefer to hear just the full name.



I installed the Android version of my old friend Tweetings. By default, the user experience for a TalkBack user isn’t optimal, requiring multiple flicks for every tweet. But I got some great advice on how to reconfigure the app, after which it was a more than satisfactory experience.



I had no difficulty getting up and running with Skype, making and receiving calls easily with a layout that is similar to the iOS app.



I found the majority of apps were less pleasant for me to use on Android than iOS. Some of these relate to the user interface of, and what I perceive to be deficiencies in, TalkBack. No actions rotor and less intuitive gestures make for more long presses and scrolling around. But others were far less subjective. Unlabelled buttons were more common. In my banking app, which is 100% accessible on iOS and a joy to use, I couldn’t even log in once I’d set it up, because the keypad to enter the security code was inaccessible. Eventually, I worked out that I could get around that problem if I had a Bluetooth keyboard handy. Once I’d logged in, I could pay bills to external sources, but not transfer money between accounts.



The DSAudio app for my Synology NAS was useable, but not terribly efficient. I have a very large music collection, and in iOS, the table index means that I could scroll to artists beginning with a specific letter of the alphabet with precision. There was no such table index in the Android app, perhaps this is a kind of control that just isn’t present in Android. TalkBack has some excellent scrolling capabilities which I was able to use, but it’s not as precise as choosing a specific letter of the alphabet to get to, and took me much longer.



An app I use a lot in iOS, the Sonos controller, was useable I think only because I knew what I was looking for thanks to the fully accessible iOS app. The Android version is verbose and erroneously identifies the kinds of controls in use.



What was impressive though is that when using Google Play Music, I was able to send music directly to any of my Sonos devices, rather like the way AirPlay works in iOS.



In fairness to Android, there’s probably a chicken and egg element about the high number of inaccessible apps in at least some of these cases. In many English-speaking countries at least, I don’t think there’s any doubt that the majority of blind smartphone users are using iOS. App developers are therefore likely to receive more requests from iOS users to address accessibility deficiencies in their apps. So I do intend to take up some of these accessibility issues, especially with my bank, to see how easily they might be addressed.



It won’t solve all the problems, but if the inaccessibility of an app relates to little or no text labels on buttons, you can label them either through experimentation to see what the buttons do, or with sighted assistance.



Some apps I use regularly aren’t available for Android at all, but usually, I was able to find accessible substitutes. For example, in the absence of Downcast or Overcast, I used Pocket Casts as my podcast client. It’s available on both iOS and Android, which meant that I was able to keep my podcasts in sync.



Using TalkBack



A screen reader is at the very heart of a blind person’s user experience of a computer or smartphone. So how capable is TalkBack and how does it measure up?



Adjusting the Volume



After getting TalkBack started, one thing that flummoxed me right away was that the volume controls on the phone were not controlling talkback volume. The tutorial, which was a bit of a strain to hear because of the low volume, made no mention of the fact that if I want to adjust it, I need to place a finger anywhere on the screen and then use the volume controls on the side of my device. It would have been very helpful had this been mentioned in the tutorial, and is a serious omission. This approach of having to place a finger on the screen to adjust TalkBack’s volume took a lot of getting used to. I seldom want to adjust the ringer volume, which is the default behaviour for these controls, but for various reasons, I want to adjust screen reader volume regularly. I often carry my phone around in my pocket, cabled directly to my hearing aids. Being able to adjust the volume one-handed is important to me.



A couple of days in, I stumbled upon a little free utility, Rocker Locker, that loads at start-up and locks the volume controls into adjusting media volume. This suited me fine until I got more ambitious with the device. You can use the volume controls on your device to move the cursor around an edit field, so you can review, modify and delete text. But if you use a utility that locks the volume controls into performing media adjustment, it prevents this function from working. You can perform cursor movement via the menu system, but it’s nowhere near as simple. So ultimately, I surrendered to the force and just accepted that placing a finger on the screen before adjusting the volume is just how it is.



TalkBack Tutorial



As mentioned in the section on set-up, TalkBack comes with a tutorial which teaches users about how the screen reader works. It runs automatically the first time you run TalkBack, and it is available at any time from within TalkBack’s Settings. If you’re getting to know a new device and user interface, there’s a lot to take in. So users may get the basics down first, then revisit the tutorial several times as they add to their knowledge. The tutorial is well-structured for this, being divided into five lessons. Lesson one introduces the user to navigation basics, starting with exploring by touch. If you’re coming from another device that uses touch, these concepts will be rudimentary, but it’s a great introduction for those who haven’t used touch before. It covers locating icons on the screen, then double-tapping to activate the last icon that was spoken. It then moves onto swiping left or right to navigate between icons.



Lesson two introduces us to scrolling with talkback. This is the point at which anyone switching from iPhone will start to notice some key differences. More on scrolling shortly.



Lesson three covers the global and local TalkBack menus. Some of the functions on these menus are now available through the use of more convenient gestures. Others are still handy, such as spelling the last thing that TalkBack said, repeating it, getting to settings for TalkBack and TTS among other things.



Lesson four covers text navigation, and how one can move by common units such as character, line, word, sentence, paragraph and page.



Finally, lesson five covers text editing.



As previously mentioned, I’d like to have seen the tutorial cover the important first step of setting the volume to a level that’s comfortable for the end user. I would also like to have had a lesson of the tutorial devoted to the earcons. These are sounds made by TalkBack that are designed to provide contextual information to the user without the need to have the information spoken. But nowhere within TalkBack itself is there an explanation of what all these bings and bongs, which makes TalkBack sound a bit like an old video game, mean.



While the tutorial is a wonderful feature and could be improved with an extra lesson or two, I think there is a separate use case for a good old fashioned practice mode, something available in most screen readers. Shortly, I’m going to have a lot to say about the angular gestures used a lot within TalkBack. They can be hard to master, and I’d like a mode where you can perform a gesture and hear what it does. This should apply to gestures specific to TalkBack, as well as those belonging to the operating system.



Hints



Just as in VoiceOver, TalkBack by default speaks hints, providing users with information about how to interact with a control. One feature I came to appreciate was that when customised actions were available for the current item from the local context menu, TalkBack would tell you what they were without the need to invoke the menu.



Working with lists



The need to scroll through lists within TalkBack has in the past made navigating larger lists convoluted, because swiping left and right didn’t auto-scroll the view of what was on-screen as is common in other touch environments. TalkBack therefore includes a range of commands to scroll through both vertical and horizontal lists.



TalkBack now offers an auto-scroll feature which is on in its settings by default. This has made a big difference, but it’s still not as robust or seamless as iOS. One difference between iOS and Android is that when you swipe using TalkBack, the screen wraps. For example, if you are in TalkBack’s settings and reach the bottom, swiping once again will cause you to go to the first item within the app. At least, that’s how it should work. I find that usually, if I swipe from the start of a list of items to the end, auto-scrolling works pretty well, and mostly offers a seamless experience similar to VoiceOver in iOS. But once you wrap back around to the top again, auto-scrolling may not work as predictably. When auto-scrolling stops working reliably, swiping left, right and then left again may not produce predictable results. And indeed if you do that too quickly, you’ll activate one of TalkBack’s multi-layered gestures.



TalkBack Gestures



If you’re coming from iOS, one thing you’ll need to get used to is the implementation of what I’m calling angular gestures. Google may have an official term for them, but I’ve not seen it documented anywhere. The closest thing an iOS user has to this approach is VoiceOver’s two-finger scrub to activate the Back button. The two-finger scrub gesture is fairly tolerant in that you can perform it in all directions, whereas TalkBack makes use of a number of such gestures, so they must be performed precisely. For example, you may need to swipe left then right, up then right, down then left, down then up, or right then down, all in a fluid motion.



Even though I’ve been dabbling in Android since 2013,and have been immersing myself in it as I get to know my Nexus 6P, I still find these gestures difficult. I think the issue is that you have to be holding your device fairly straight before you perform them, since if you don’t, the vertical part of a gesture may be interpreted by the software as a diagonal one.



I also believe that these gestures are better left to lesser used functions, with simpler gestures being used for functions users are likely to perform regularly. For example, TalkBack makes no use of triple taps. It makes no use of two, three or four-finger taps at all. I don’t know whether there is a limitation in the operating system that prevents TalkBack from using such gestures, but having a wider range of taps available would improve the user experience a great deal.



When I’ve raised this with Android users, one suggestion that came back was that some older devices aren’t capable of detecting multi-finger gestures. It’s hard for me to imagine that there might be many such devices in 2016, but if it is in fact the case, then it’s another example of how fragmentation can constrain the accessibility experience.



While I appreciate that Android is not iOS, there is a set of gestures that has become common to both iOS and Windows. Some of these, such as swiping right and left, are supported by TalkBack. Others, such as the rotor and a two-finger swipe down to perform a read all function, are not. At present, this gesture is part of the set used for scrolling.



While I understand that we can’t expect all user interfaces to be identical, if a convention has been established as I believe it has been in terms of a gesture set for accessible touch screen navigation, using that convention is in the company’s best interests since it reduces the learning curve, and that, most importantly, is in the end user’s best interests.



One of the core functions of any screen reader is the ability to read continuously from where you are to the bottom of the screen or end of a document. When you want to read a newspaper article on the web or in a news app, or when you have a long email you want to hear, you’ll perform a say all function. TalkBack offers the ability to perform a say all in a number of ways, none of which I believe are particularly appropriate given the importance of this feature and the frequency with which it is used. You can shake the phone, and you’re offered some control over how hard you need to shake it to invoke a say all. Set the threshold too low, and you’ll be reading the screen continuously just by walking around with the phone in your pocket. Set it too high, and you’ll get fatigued pretty quickly from shaking your phone every time you want to read something continuously. Try shaking it on the bus too often, and you’ll be approached by bemused passengers who want to know if there’s something wrong with the blind guy’s phone that they might be able to help with. Or maybe you’ll elbow the passenger next to you due to your enthusiastic shaking.



You can assign it to an angular gesture. Again, the issue I have here is that I find them difficult to perform at the best of times, and I’ve given up trying to perform them when the phone is in my pocket.



If your device supports it, you can also single-tap or double-tap the side of the device. This can also lend itself to accidental activation, and can equally be difficult to perform when you want to use the feature.



What I would give for a simple two-finger swipe down to get this essential function, just like in iOS and Windows.



Even when you manage to perform the read all function, in many apps I have found that it doesn’t work at all. In some apps, there are significant pauses or there is a little repetition, while in some apps it works very well.



Similarly, you can get to the top or bottom of a screen with layered vertical gestures, while the much simpler four-finger tap at the top or bottom of the screen goes unused.



A gesture has not been implemented to stop TalkBack’s speech. The only way to do so at present is to wave a hand in front of your device’s proximity sensor if it has one. I’ve found this problematic at times because when I’m using the phone, one of my hands seems to rest in around that location naturally, causing continuous reading when I’ve been lucky enough to get it working to stop.



The TalkBack team has made some welcome changes to gestures, dispensing by default with the circular menus that many of us found frustrating. Now the default is to show these menus in list form. Perhaps we’ll see a gesture set that is more familiar to users of other touch screen products in future.



Your Commands Your Way



TalkBack gives you the ability to reassign gestures and keyboard commands to suit your preferences. With my trainer’s hat on, I can se that this may cause some issues for trainers if they don’t realise that a user has changed the gestures and keyboard commands around. With my end user’s hat on, I appreciate this flexibility.



The configurability is somewhat limited, in that you first choose from the available list of gestures, then you choose the function you’d like to assign. Obvious candidates such as multiple taps of two or more fingers are not available.



The same kind of functionality is available for Bluetooth keyboards. By default, a set of TalkBack keys are assigned using Shift and Alt as the modifier keys. I find them intuitive and quite effective. But if you’re a creature of habit and you want to use a key combination that makes the keyboard support feel more like VoiceOver, you can do that.



I have not found a way to navigate through an edit field or select text by character and word from the Bluetooth keyboard, but you can type into edit fields, hear what you’re typing, and backspace over text to delete it. Pressing Control+A to select all text, then Delete to erase it, also works. . Shortcut keys, an area where Apple and some third-party developers have been making significant progress of late in iOS, are scarce in Android apps. Occasionally, I find that I can press Control+N to start something new in an official Google app such as Gmail or Calendar. A few other keys work, but not many.



One only needs to look at the efficient experience offered by Twitterific in iOS with its suite of keyboard commands to know what a productivity boost well-implemented keyboard shortcuts can be.



Granularity



Granularity refers to the unit by which you navigate, such as character, word, line or paragraph. It is great to see TalkBack supporting paragraph granularity, VoiceOver on iOS does not. When you’re in a web environment, you’ll also want to navigate by elements such as links, headings and form controls.



TalkBack now has easy granularity control. One simply swipes up or down to choose the granularity, then swipes left or right to navigate by the unit you’ve chosen.



It’s elegant in its simplicity, although some may find the simplicity to be limiting. In Windows and iOS, a rotor feature allows you to choose what will happen when you swipe up and down. The rotor need not be limited to text navigation. You might, for example, want to adjust your speech rate, or toggle a certain function on and off. In iOS, apps that are programmed appropriately can include an actions rotor, making it easy to select commonly used functions based on context. I find that the actions rotor gives me a significant productivity boost.



By hard-wiring the swipe up and down to only apply to text navigation, TalkBack has limited the functionality of one of the simplest gestures one can perform.



For example, in certain controls, swiping up and down with one finger might have been able to change values. Instead, one interacts with certain controls a little like one does in OS X. You double-tap a control, and double-tap to select a value which causes you to be exited from the control. You then evaluate the impact of the change you’ve made, then repeat the process as many times as needed in order to find the value you want.



The lack of a rotor means that at present, there is no simple way to adjust speech rate on the fly. You’ll need to get all the way into the text-to-speech settings to do that. And there isn’t a way to adjust punctuation, an important feature when you’re proofing



On the flip side, of all the iOS and Windows gestures, I’ve found that people seem to find the rotor gesture the most difficult to master.



Playing Nice with other media



It’s important that any smartphone screen reader co-exists effectively with the many audio and video apps available. TalkBack attempts to do so, but in my experience there are often significant issues.



TalkBack’s volume is controlled using the multimedia volume, the same volume control that governs audio and video applications. So it’s not possible to set TalkBack’s volume independent of the playback volume, other than determining that it should be a certain arbitrary percentage of the playback volume. VoiceOver, on the other hand, has a rotor item that provides for separate control of its volume, independent of media apps.



A number of screen readers now offer a feature that VoiceOver calls audio ducking. This is where the volume of any media you’re listening to is slightly lowered when the text-to-speech engine says something, and raised again when speaking has stopped. In TalkBack, this feature is called focus speech audio. At least on the hardware I have, it is far too buggy to be left enabled. It works as intended in some apps, such as Google Play Music, but it makes a number of other media apps unusable. When I’m listening to a radio station via TuneIn Radio or CSpan Radio, focus speech audio causes audio to stop playing altogether whenever the screen reader speaks, resuming exactly from where it stopped when speaking has finished. This makes it impossible to have a radio station on in the background while doing other things, something I do regularly. I’m advised that going into TuneIn’s settings and enabling a toggle telling it not to pause audio in the background will improve the experience with that app. But there are still many other apps where keeping focus speech audio on breaks things.



I’ve also not found a way to play and pause audio from anywhere, as one can with the handy magic tap in iOS. There’s an argument to be made that the magic tap is trying to perform too many functions and sometimes gets confused, such as failing to pause audio that’s playing when you try to use the gesture to answer the phone. I have some sympathy for that argument, but there still in my view needs to be a way to toggle playback from anywhere with TalkBack.



Dimming the Screen



iOS has a feature known as screen curtain, which protects the privacy of blind users by darkening the screen when VoiceOver is on. It gives you peace of mind to know no one’s looking over your shoulder while you’re working with sensitive data.



TalkBack’s equivalent is called Dim Screen. You can enable a shortcut to toggle it on and off, and/or enable it in TalkBack settings. Unfortunately, I’ve had to stop using the feature, because I’m too often prompted by messages within other apps that an overlay is obscuring the screen. This feature is not fit for purpose, and it would be better to remove it than frustrate users with a very poor user experience.



BrailleBack



Regular readers to this blog will know how passionate I am about ensuring that mainstream manufacturers pay appropriate attention to Braille. While I have expressed reservations on several occasions about the quality of Apple’s Braille support, it is considerably more advanced than BrailleBack on Android. I truly wish I had more positive things to say.



BrailleBack is a separate, free application, available from the Play Store. It’s not part of TalkBack, but it is dependent on it. If you shut down TalkBack because you want to use Braille only, you’re out of luck. You’ll get a little Braille feedback, and can still control the Android device from your Braille display, but you won’t have sufficient information to use the device successfully.



The most critical flaw in the integration of TalkBack and BrailleBack is that you can’t turn speech off. Even if everything else about the Android experience had exceeded my wildest expectations, this one thing would be the deal breaker. I often work on all of my devices with speech off. Whether it be in Windows, iOS or OS X, I can always toggle speech off with a quick command. You just can’t do it with Talkback. What this means is that when I’m in a meeting, I’d have to have something connected to the headphone jack, or turn the volume all the way down, just to use my phone as a Braille-only device. If I wanted to read something while I enjoy some music, I can’t do that, because every time BrailleBack scrolls to a new chunk of text, TalkBack speaks it.



Once you’ve installed BrailleBack, you’ll find that the command set is unorthodox. I’ve been using Braille devices since the VersaBraille 33 years ago. All Braille software I’ve used has adhered to a command set that has become a standard. As a product manager in this field for a couple of companies, I was always careful to adhere to it, and it’s not as if BrailleBack offers anything that is logical or better, it’s just deficient and confusing.



Normally, you would expect dots 1-2-3-cord to take you to the top of a document or screen, and dots 4-5-6—cord to take you to the bottom. In BrailleBack, dots 1-2-3-cord opens keyboard help, while dots 4-5-6-cord does nothing. This standard set of commands include navigation and speaking of lines, words and characters. Few of them are observed.



There seems to be a lack of structured mode in BrailleBack, meaning that several icons are placed one after the other on the screen. On the Home Screen, icons are separated by a colon. You can press a cursor routing key to activate an icon. I found no reference to how you might activate an icon if you don’t have cursor routing keys, but this may be because the software is sensitive to the display that is connected.



Being able to hold down a cursor routing key to simulate a long press is a nice feature.



Unified English Braille is now supported for contracted and uncontracted output, but not for input. Contracted input is not supported at all . Contracted input admittedly is a difficult thing to get right, Apple still hasn’t managed it.



Before you can even enter any text from your Braille display, you need to go into the device’s settings and enable the Braille hardware keyboard, setting it to be the default.



While I had excellent results as a speech user with Kindle, I could not find a way to read material beyond what is on the current screen in Braille. Contrast this with the seamless way VoiceOver scrolls through pages in iBooks and Kindle with a Braille display.



The combination of all these factors makes a mainstream Android device a non-starter for the serious Braille user. It makes it not viable in education where Braille is important. And most significantly, it is a non-starter for DeafBlind users, who are information-deprived, and deserve far better than this. BrailleBack desperately needs some love, and I believe Google needs to take it seriously. It should be part of the core accessibility suite, not an optional download, so it’s easy to activate from early on in the device set-up process.



Conclusion



I’m predisposed to liking the way Android does things, and as I’ve outlined in this lengthy post, there is a lot to like. In terms of ensuring that Google’s own apps are accessible, the progress made has been enormous. You can buy an Android device from the store, hopefully get TalkBack up and running yourself although that’s not guaranteed, and be assured of a pretty decent out-of-box experience. That’s great news for people who are on a budget, or who for whatever reason just want to have a choice. That freedom is available to sighted people, and I think it’s fair to say that it’s now available to us.



Can you get things done and truly use an Android phone effectively? In my view, the answer is that depending on your requirements, an Android device may work for you. If you’re a heavy Braille user, the inability to mute TalkBack and the limited Braille command set is probably going to be a deal breaker for you, as it is for me.



If you don’t use Braille, I think Android is very much viable for users in two categories. If you’re highly geeky, you have a profound philosophical objection to the way Apple prevents you from using your device as you see fit, and you like customising the heck out of your device, there’s really no contest. I enjoyed being able to work with file managers, cable up my phone to my PC to copy data both ways, and just generally basking in the lack of constraints.



At the other end of the spectrum, if you’re on a budget and an iPhone’s out of your price range, and you tend to stay on the beaten path in terms of the tasks you want to perform, I think Android now offers a reasonable experience.



I wouldn’t call it an optimal experience at this point. There are probably some ongoing underlying issues to be addressed at the operating system level, and work will assuredly continue with Android N, but the major reservation I continue to have with Android is the user interface of TalkBack. It’s come a long way, with complex circular menus no longer the default, and granularity far simpler now, but it still feels convoluted and geeky to me. In saying that, I readily concede that if you use it day in, day out for a long time, it may become second nature.



As I mentioned about 8000 words ago, I bought the nexus 6P so I could be assured of receiving Android updates promptly. I think that was a sound decision for someone like me to make, nevertheless I do have a bit of buyer’s remorse and wonder if I would have been happier with a Samsung device, running their screen reader which has a gesture set that is more orthodox from the perspective of a Windows or iOS user. You can perform a say all with the familiar two-finger swipe down. A swipe up or down will adjust granularity just as it does in TalkBack, but there is also a gesture, far easier for many to perform than the iOS rotor, which allows you to swipe up and down to adjust speech rate, screen reader volume, toggle screen dimming and more. The magic tap works to play and pause media anywhere. While I haven’t minded not having a physical Home button on my Nexus 6P, Samsung’s devices tend to offer one, allowing you to perform a familiar triple-click home to toggle their screen reader on and off. So if it weren’t for how long it sometimes takes Samsung to push the latest version of Android, I would be replacing my Nexus 6P with a Galaxy S7, specifically because I believe TalkBack is letting down what could now be quite a good user experience for speech users. Since Android fragmentation is an issue that affects everyone and is of concern to Google, I’m hopeful we’ll see some improvements here, which might remove my primary reason for not going with the more intuitive Samsung solution.



I also remain hopeful that Samsung will do for Braille on Android what they’ve done for speech.



That said, I have only examine an S7 briefly, so there may be issues with its product that I’ve not had the chance to experience. For example, some people have expressed concern about the amount of software that is included on Samsung devices that can’t be removed. It also has a speaker that sounds mediocre given its price.



If Samsung offered their screen reader for sale to owners of other Android devices, I’d buy it. After all, I’d have two obligation-free hours to give it a try and could get my money refunded if I found it wasn’t much better.



I’m excited about keeping my Android device around, because unlike iOS, it’s undoubtedly possible for someone else to come along and produce an alternative screen reader, as Samsung have demonstrated. Would sufficient people pay for one if it offered a better experience? I’m not sure they would. But perhaps a group of open-source developers who see the efficiency flaws in TalkBack and believe in an open platform may do something special. Then again, TalkBack may continue to evolve. There are some very capable people involved with its development for whom I have immense respect. It seems to me that the fundamental problem TalkBack now has is a legacy user interface that is familiar to those who have used it for a while, but may put off potential adopters.



While I’m on the subject of the capable people, it has been gratifying to see members of the TalkBack development team engaging with end users on email lists. We have a long-standing culture of this kind of exchange of ideas in the blind community, and it’s good to see Google open to this kind of communication.



While I’m encouraged by all the progress that has been made, and optimistic about the progress that will continue to be made, for now this exercise has given me a renewed appreciation of the VoiceOver and iOS experience. I’m no fan boy and have offered what I hope is constructive criticism over the years when I think its warranted, but Apple got the basics right back in 2009 with the core gesture set. From there, year on year, they’ve added things that have enhanced efficiency. Bluetooth keyboard support, Braille display support with all its idiosyncrasies, handwriting, Braille screen input, the actions rotor, the item chooser, in-app keyboard shortcuts, all help a blind person to get and enter information efficiently and reliably.



While surprising liberation continues to occur, I do wish iOS would be less of a control freak. But in the end, what matters to me most is how efficiently I can manage my busy life. I love to tinker, but I also have clients and commitments.



And of course, most of the practical benefits in terms of the services Google offers are available on iOS, with most of them now being highly accessible.



So unless you’re a major geeky hacker, or an iPhone just isn’t in your budget, I do believe that for now, as I write in June 2016, iOS is a more polished, reliable, efficient experience from an accessibility perspective. That said, those who say Android is unusable by a blind person are I think selling the platform short. If you have the opportunity to use an Android device for yourself, I highly encourage you to take that opportunity. If you’re coming from iOS, give yourself some time. Some things are different, but that doesn’t automatically make them inferior. Some stores and carriers offer a 14 or even 30 day right of return policy, and if you like to try new things, having a look at an Android device will I think be worth your while.



I can’t wait to see what happens next.



Have you switched from one platform to another? What prompted you to make the switch, and how well has it worked out for you? Are you an Android user with something to share on this post? I’d welcome your constructive comments.



Share and enjoy: Twitter

Facebook

Pocket

