About this talk

Gordon will show us how we can use the Web Bluetooth APIs and JavaScript to control things in the real world, straight from a webpage.

Transcript

- And so I gave a talk here just over a year ago. So out of interest, how many people saw the talk then? Okay, cool. Not that many. So I'm gonna try not to repeat that much. But it's probably just as well that there's a bit of overlap. So, yeah, I make these all JavaScript devices. And specifically the new one that I've been doing is this thing called Puck.js, which is actually a device that's got Bluetooth Low Energy and executes JavaScript. And at about the same time as I was working on this, there was something called Web Bluetooth that got put into Chrome and it's now in Opera and it's kind of pushing out to some other browsers, hopefully soon. And it's this really cool way of actually accessing Bluetooth devices straight from a web page. So, yeah, I'm gonna talk a little bit about how you actually go about controlling something physical from a web page. So, yes, first off, all of these devices actually use microcontrollers. Things that are a bit like this. This chip is the one that's actually on board, this circuit board. It has no radio on it or anything, but it's a completely self-contained computer. So microcontrollers I guess they're a bit like the SOC you might have on a phone, but they're designed to have like instead of going out for performance, or something else, they're designed mainly for very small sized, low cost, lower power consumption. And everything you need is on that chip, so it's got all the RAM, it's the got the ROM, it's got the processor, this one's got an ARM in it. It's got various peripherals for reading analogue values, outputting analogue values, talking to those other bits of hardware. And it's got oscillators and voltage regulators. You can literally just apply a battery to that and start executing code. And this is a sort of mid-range microcontroller. In terms of performance, I looked it up today and it'll give the first Pentium a run for its money. It will probably beat it in benchmarking scores. And so it's ridiculous, this thing that costs two or three dollars that fits on the tip of your finger can outperform a, what was at one point, a desktop computer that I was really happy to have. So these things are absolutely everywhere. ARM shipped 15 billion cores, well, they licenced them, in 2015. And they're only one of many manufacturers. So all of these things probably have a microcontroller in them, stuff that you really wouldn't expect, like an electric drill probably has one in, especially the more expensive drills that do like slightly fancy things. Bike lights that have different modes will probably have one in, because, again, it's much easier to buy a little bit of hardware, like a microcontroller that might be produced in the billions, than it is to buy a special purpose piece of hardware for flashing bike lights. They may only be produced in the million or two. And the economies of scale just mean that it's much easier to use something general purpose. In the same way, if you were trying to generate even quite basic behaviour using discreet components, you'd end up with 10 or 20 different components, all of which you'd have to source, which you'd actually have to change if you wanted to change behaviour. And when the solder all of those together, each connection is like a failure point, something that could go wrong. So this would pay for more with one chip, it's a massive bonus. This is, I guess, most people's introduction to microcontrollers. Though, who's actually used an Arduino Bot? Great, so quite a lot of you. So this is one of the early ones. Here is the microcontroller. It's a much more basic version than the one I showed you before. It's not an ARM core, and it's got pins and they're much easier to solder to. But it's same kinda thing. You've got a controller here. This doesn't have any USB on it. So you've got a little USB interface up here. And you've got paths by that. And you programme it a bit like this. This is the Arduino IDE, a really simple bit of code, they have example up here, but I think for flashing an LED on and off. And it's just C code, the X gives procedure here. So if a lot of you are writing JavaScript codes, which I guess you are, this is obviously like really crazy bad news, because this function's gonna take two seconds to execute. And you can't do anything else while it's happening. You can do things with interrupts. But it all starts to get quite serious. You start to have to really know about the microcontroller and you can do some really bad things that will make it crash and behave in ways you don't expect. So from that point of view it's not great, but it's also lets you write very simple code. And that's quite handy, because there's no real way to debug this. This is literally a black box. You can turn like pins on and off, which turn lights on and off. You can output text which will be sent back down USB. But if the print statement that you've written isn't actually getting executed, because something bad's happened before, then you have no idea of really knowing what's going on. And you can't, for instance, break or step through or inspect variables and things like that. Some of the more modern boards do have that, but it's still very far from commonplace that people actually have any debugging tools on these things. So 35 years ago, this was released, this is BBC Micro. At the time it was reasonably expensive. But it has a keyboard, it has video output, it has a basic interpretor, where you could write basic code. You could actually even do certain debugging by checking what values were in variables after an error had occurred. And it might even give you error messages if something went wrong. And, so for instance, that one right at the start is around the same memory and significantly faster, probably 10 times faster execution speed. So there's no real reason why you can't put that kind of interaction onto the microcontroller, and actually make it so that you can now, you can actually be talking directly do it, and you can do things like debugging. So there are a few options for this. And I started about five years ago now, trying to put JavaScript on it, 'cause it felt like a really good mix. And it comes pre-installed on so many devices now, even something like a phone that will probably come locked down by default. You can still execute JavaScript code. You can still go to a JS bin or something and run code. So now you can run JavaScript on kind of all these devices. And the nice thing is if you're doing some kind of IoT device, you could write an algorithm in JavaScript. You could choose whether you wanted to run that on your server, or you could send it off and run it on the client's device, maybe they want to be able to tweak some settings and get the results back in real time. Or maybe you want to do all the calculation in the IoT node itself, and reduce the amount of data that you're sending. And you can do that without re-writing your code, you're just keeping it in the same language. And the thing that makes it really nice, or at least in my point of view, is that you've got closures. Instead of that example before, where you had a function that flashed an LED that took two whole seconds to execute, this is the example of maybe something you might quite often do in a microcontroller. You want to read the value on pin, maybe you want to turn something like a heater on, wait a second and then read the value again, and then return both values so that you can compare the two. But doing this let's you... This bit of code executes and then the whole thing's idle for a second. And then it executes this function. And then when it's finished, all the memory's automatically freed, you just don't have to worry about any of these things. I guess the question is if you're gonna use JavaScript on a device, why wouldn't you just use RAW speed file, because these things are amazingly good value, amazingly cheap. You can get one with wi-fi and Bluetooth on it now for $10, I think it is. But the main thing is power consumption, these have sort of 256 or 512 megabytes of RAM on them. And that RAM, every bit of that RAM has got a transistor in it. Those all have to be refreshed. And that actually takes quite a lot of power, regardless, of whether the computer is actually doing anything or not. So the power consumption isn't great. I think the minimum you can get the kind of least stacked up RAW speed probably down to is about 50 milliamps, it's around that. Which is fine for running off a battery for a day or two, maybe a week, but you really struggle for anything longer than that, and especially when the batteries get smaller. If you're trying to run it off something like a watch battery, it's pretty much impossible. But if you could, it would probably flatten it in about two or three hours. So this is one of the reasons why microcontrollers work so much better is their sleep modes are great. They generally use static RAM, which doesn't have to be refreshed, so they can really go to sleep. This is a logarithmic power usage scale. And it's running a really simple bit of code that every second it's waking up and it turns one LED on, the other LED, both LEDs, or turning them off. So here you've got it going down to like three microamps, which compared to probably 50 milliamps of the var speed pi, it's 10,000 times less. And then it kind of peaks up here, when it's executing some code. But this is turning the first LED on, which is half a milliamp, and second LED< half a milliamp, and then one milliamp here. But when it's actually doing something else, the power usage by what it's powering is basically dwarfing everything else. And this means that you can write code using these callbacks and event-based code that really is very power efficient. No one's had to worry about power consumption here at all. But when it's actually not doing anything, it's perfectly fine. And that would run off a battery for years. So I started off with Espruino, ages ago, with a really, really dodgy website, just kind of hacked up, trying to just give away the software, pre-compiled for a few little boards. And it quite quickly grew. So this is actually not all the boards I have in my desk that run Espruino. I forgot a few. Now it's over 40 different boards in the official Espruino GitHub repository that it will compile for. And there are a whole load of forks as well for other boards. So it's a huge amount. And all of that's kind of completely open-source, completely free. But what I try and do is, installing the software is often a bit of a pain. So I try and pre-install the JavaScript interpreter and the boot loader so you can easily update it on devices. And then I sell those devices. The Kickstarter campaigns, and actually now just the kind of day-to-day sales of people buying those pays me to work on it full time. So this brings me to question of if you have a button, something like this, or a big light, how would you control this from your computer? And it used to be easy. 20 years ago, and I can't really believe it's 20 years ago that the USB spec was first sorted. I think the anniversary's now passed. The same USB connector, that was a first around then is still around now and it will run a USB device that bought 20 years ago, which is kind of unheard of in the computing industry. But before that happened, you had this, which was the printer port. And you'd even get things like webcams that would plug into it. And this is basically a parallel data port that sat on the back of the computer. And you really could just stick a light bulb in the back of it. This is not recommended at all. Not desperately good for it. But you could do that. So pins like, I think that was pin two, all the way to pin nine were just eight data pins. And you could write one line of code with no drivers, you just access a peripheral directly and write a byte to it. And each bit of that byte would turn that light on and off. And that was great. It was lovely and easy. But, obviously, really bad news from the point of view of any kind of stability, especially when you start talking about multitasking. You can have multiple applications trying to do this. So when sensible operating systems, like XP, started coming along, they started completely stopping people from doing this. And you started having to worry a lot more about giving the APIs and stuff like that. And around that time as well the original parallel port just disappeared, because USB is so much easier. So now you've got the situation where you've got USB on pretty much every device. And, in fact, in a lot of devices, it may be the only port you have. And the USB is a serial bus that you've gotta have something to decode the serial data. And there's a bit of protocol on top of that as well. So realistically you're gonna need a microcontroller, another computer at the other end to decode, or that USB and do something based on it. So like the classical way of doing this is it's moved now from having this really simple way of just pushing the data out, the individual lines you could just by writing to the address in memory, to now you have to write a lot of code for a microcontroller to handle the USB. You have to write a driver for your computer and one pro OS to handle the interface to that. And then you've gotta write an application per operating system to talk to that driver in a completely different way in each operating system. To then do what you want. So what a lot of people do is say use a completely standard device. And this is a USB to serial converter. It's for, especially the one up there, it's kind of a much more consumer friendly thing. If you happen to have a USB device, sorry a serial device, and I'm struggling to think what you might actually have now. You can just buy one of those and connect it up. And it's basically a pipe that you can shove characters in and they go out the other end, and you can receive characters. So it's a nice, easy thing to write code to interaface for. So what you can do is you can actually make a device that just pretends it's one of these. And the driver's built in to every operating system. And then you can just send characters. And the only thing you have to worry about is making sure that you are connecting to the right device. So that's what's being done with this. This is the Espruino Pico. And it's got that exact same little microcontroller on it, running a JavaScript interpreter. And when you plug it in, it pretends it's a USB serial converter. And when you do it, if I go in here, and run a serial terminal programme, this is Linux, but it's very similar to most other platforms. And so you now have a record. And you could be almost forgetting to thinking that it's very, very similar to something like Node.js. And the way it's programming it is not that dissimilar from if you were SSHing into another computer which was running a full version of Node.js. We can do things like asking it what it's temperature is. And we can, for instance, tell it to light a light up. So if I do digital light, and it's got command completion and history, and stuff like that, just built into it. So, hopefully, not all of you will be able to see that, but you can light the light up. And, actually, controlling something like this is just this easy, because you just plug it in. And then you, instead of saying LED, you give it the name of the pin that it's attached to. So if I change that to B3, which I think is the light, it'll light up. And you can, for instance, if I turn this into a function that will then toggle the light... I've got to find a variable. Every time I call this, it will toggle it, hopefully. Yeah, so I can then do something like setting default. And while this is not actually multitasking, it's able to flash the light and it's still able to surface the console here. So you can still interact with it. And, for instance, you could change this function on the fly to do something a bit different if you wanted. Now all this is happening without any of the computer's involvement, the interval that's running is purely on the microcontroller. So if I touch Save, okay, so if I have USB power pack and I just take that out and I plug it in here, hopefully it'll start working. So as it happens, this is actually drawing so little power now that the USB power pack will timeout after a few seconds, because it won't think anything's connected to it. So all of this is done. Come on. I'll get rid of that. All of this is done by just sending characters. And we could actually just echo characters directly to that serial port. So you don't even need very much in the way of software on the computer to control it. And it's getting even better, because Chrome have added this thing called Chrome.serial. So if your app is in the Chrome Web Store, you can access Chrome.serial and you can access the serial port directly from a web page. So I actually have web app. This isn't actually the one, this is running as a website. But this lets you programme the device in a much more simple, like easy, fun way, with code completion and all that stuff. And it's got like graphical entrance and stuff in it as well. And this will run on any computer. It will run on Chromebook, on Mac, on Linux, and let you programme the device. And it's great. Or it will be great until the end of this year when Google are ditching the Chrome Web Store. So now you've got this kind of problem of how actually do you try and write a cross-platform app that's gonna interface to a bit of hardware. And it's kinda difficult. And also you may well end up with, you may have forgotten your adaptor. It's like a whole bunch of people do. Like modern computers don't always have a USB Type-A connector. And it maybe that Apple next year decides to have another batch of courage and just remove every kind of standard connector from their devices anyway. So there's a solution here, which is to go wireless. And Bluetooth has always been quite a kind of heavyweight thing. But there's a simple Bluetooth low energy that was introduced in Bluetooth 4.0, which is very, very different. Despite being called Bluetooth and working in the same frequency range of sort of 2.4 gigahertz. It's actually very little alike at all. And it's designed, it was originally designed by Nokia, for very low energy devices. And because of kind of its simplicity and I think better licencing, it's become a lot more prevalent. So pretty much every personal fitness tracker will have one. A lot of wireless toys are starting to have them now because it's actually cheaper to use your phone than to ship a little controller with them. And, yeah, it's looking really promising. It's working its way into a lot of stuff, basically every phone, every laptop produced now will have a Bluetooth LE radio inside it. So this is an example of one of the robots. So this is BB8 robot that runs off Bluetooth's Low Energy. I think one of the things with, especially these little toys is if you have to have an app for it, that app will sit in your phone, it'll probably update itself like every few months. It may well have annoying notifications that pop up telling you when a new version of the toy is available. It will just be generally really annoying. And it will be much better if you could just go to a website, control it, and then close a website, and everything would be gone from your phone when you were done. And that's exactly what's happening here. So this is something using web Bluetooth, which is running Chrome. And it will run on Chromebook, it will run on all Macs, if your Mac's got Bluetooth Low Energy. It will run on Android. And there are hacks at the moment to make it run on Linux as well. But there are kind of half-hearted solutions at the moment to run it on Windows and on iPad as well. So it's really kind of starting to get really quite multiplatform. So this is, again, this little device that I've done, which is basically taking what was on this here, the little Pico device and sticking it on a microcontroller with a radio. So you can control in the same kinda way. So just give you a bit of background on Bluetooth's low energy. The idea behind it is quite simple. What they're trying to do is, really, they don't care very much about getting a lot of bandwidth. What they care about is getting a lot of life out of a battery. So the default mode for a device is this one at the top, which is an advertising mode. And the device itself, like this, will sit there and a few times a second, in one of three different radio channels, it will just transmit data. And that data will probably have the name of the device. It will have some information about what kind of device it is, like what kind of things it's capable of doing. And it may have, or it can have, a URL in it. So, for instance, things like iBeacon and Eddystone use these so that, for instance, if your phone goes within range of them, it can give you a notification telling you you're near so-and-so, would you like to find out more information about it? So the other option you've got is that right after it's transmitted, it receives through a very small amount of time, and actually let's say, these gaps here are completely out of proportion, so this is probably a thousand times longer at least than the actual amount of time it's advertising for. So you can get some very simple two-way communication going on, but you can't actually, or you're not supposed to be able to do anything but change its state on this device without a connection. So what you want is to make a connection. And once you connected, there's a lot of negotiation goes on about encryption and stuff like that, and the rate at which things work. So some devices can react very, very slowly, but have a long battery life. Some react very quickly, but use a battery more quickly. Once you're connected it all switches around. And these roles are really confusing. So the peripheral is obviously the peripheral, but central happens to be your laptop or your phone, or something like that. Here, declined is still actually your phone or your laptop. And it's now sending out data every so often at a fixed time period. The device will know when to listen for it. So it's only listening for a very short period of time after every second so it can make the battery work a lot better. So all of this would result in just being able to send packets either way. But a lot of that's really hidden from you, and behind something called GATT. And this is, in a way it simplifies it, but it also hides a lot of the underlying stuff from you. So you've got these things called Services. And a service like something a device can do. So you might have a Bluetooth Low Energy lightbulb. And the service would have characteristics. Which you might have a characteristic for the colour of the light. You might have another one for the brightness. You might have one for whether it's on or off. And the light may have another service. Like it may have a service for former updates, or something like that. So all these things are nicely encapsulated. And much like we had the USB to, say we'll dongle, that the little chip there pretended to be, you've got something called UART Service, that's actually been made by the manufacturer of the Bluetooth Low Energy chips that are used in there. And once you've got this, you've got these two characteristics, the transmit and the receive. And they work in a very similar way to the serial port. This one, the transmit characteristic, has a notify property. And that means that you can ask to be told when it changes. And when it changes, you're actually given the data that's in it. So when this actually wants to transmit any data, you just get a new event basically saying here's the new data. And for this one, the RX characteristic, you just write data into it. And when you write data, that's treated as data that's being received by this. So now we go onto Web Bluetooth. So the stack isn't, it's a draught spec, but at least the basics I don't think are gonna change at all now. It all seems quite well sorted. So your code actually ends up looking a bit like this. You do requestDevice, and then you give it some filters to help to narrow it down what kind of device you have. And when you do this, it will present a box to the user which will let them choose from a series of devices. And it's found to match those filters. Then once that's happened, it's obviously just return to promise. You can connect to it, you can get that primary service, which the ID here actually corresponds to the ID that was up there on the left. And then you can get the characteristic to the transmit and then you can write the value. And because this is just executing JavaScript code, you just write the JavaScript that you want to execute. And so if you run this, this will turn the LED on. There are few little gotchas here. One of them is that, again, because it's a very limited thing, you can't write very much data at once. So you can't write more than 20 bytes. So, conveniently, this is a lot less than that. But if you wanna do more, you've gotta repeatedly call write value. So your actual code will probably end up looking a bit more complicated. And just to add, so this device can not only be a peripheral it can control other devices. So it can be essential as well. So I've kept the same APIs for our Bluetooth. So you can literally say nrf.requestDevice, instead of navigate.bluetooth.requestDevice. And it will do exactly the same thing. To kind of hide a lot of this complexity, I've just come up with a really simple library that you just say puck.write and it will arrange everything for you and will then send that data off. So as an example, if I do this. Where is it? Okay, I've got an example page here. But if we do this, we get the device here. And hopefully we should see. Okay, so the light lights up. And once it's connected, it's actually quite quick to respond to presses. And it's that easy to make this a kind of a more mobile-friendly little site. So if we take this one, hopefully the same thing will happen. Yep, okay. But it's not one way, either. You can easily receive data. So, for instance, we've got one here that, obviously a bit more complicated, but what it's actually gonna do is when it connects, the code here is actually saying 10 times a second actually print the current value of the amount of light that's falling on this device. Then that's read back. And it changes the brightness of the light bulb on the screen. So if I do this and I connect again, there we go. I should say that the Linux support still isn't amazing on the version of Linux I'm using. So it's entirely likely I will have to reboot the Bluetooth stack at some point. Oh, no, that's great, okay, so it's done. So hopefully if I cover this up, yeah, okay. So it's responding reasonably quickly to 'em. And you can connect things to these pins and can control proper things in the real world. This also has a infrared transmitter in it, so you can use it for doing controlling air conditioning, the TVs, video recorders, stuff like that, if you need to. So, again, the web ID as I've showed you before, the web ID can connect and run in the browser and you can develop your whole software completely wirelessly without ever having to have a physical connection. So this brings me to this, which is something I came up with recently. Which this is actually the coil pack of a BMW car. So the idea is that you have maybe a four cylinder car, you have four spark plugs, and each one of these goes on top of the spark plug. And it's basically a high voltage transformer. You put like a little bit of voltage in there and it amplifies it a lot and creates something that will jump across a gap. In the car it's probably off like 14 volts or so, when the car's running. But I'm just running off like three-and-a-half, four-ish here, so it's nowhere near as bad as it would be in a car. But if I take this and plug the power in. Okay. Are we gonna see this? Just a second. Amazingly not. Not properly, yet. But it always surprises me every time I get through. I think they figure that anyone with something that's so obviously a bomb in there. It would be too stupid to go through. Okay, hang on, let's try this again. Linux. Okay, see if that works. Yay! Much better. Okay. So now we've got this command prompt, which is basically the same as the terminal that we had before, but going via Bluetooth and doing stuff actually on here. So this whole coil is connected via a little transistor here to pin D1 on board. So all I have to do is actually pulse that. So if I do digital pulse, D1, oops. And I'm just gonna give it a small one millisecond bolster now. I don't know whether I'll actually be able to see me doing anything. You could probably see my hand moving a little bit, but not much. So we can actually make this a lot larger so you will see it. So you can probably see a little bit of a jerk there. And you can actually, you don't have to pass in just one value, you can pulse in an array, in which case it will pulse on for a certain period, then off for a period, then on. So if I do it on for five and then off for 50, and I just repeat that a few times. And then I do this. And I won't do this very often, because it's really painful, but you will definitely see a response if I do that. Yeah. So, yeah, you can do all kinds of fun stuff with this. Now the really cool bonus thing you can do here is that it's sort of like, a little bit like a cutdown version of Node. And so it has this event that you can hook onto called uncaught exception. So if I do, let's just pull the example out of here. Oops, I can't spell. Okay. So if I were put this in here, nope, I won't. And then I put this in uncaught exception, it would be a really good way to learn how to code, because every time you got something wrong... So if I do one plus two, everything's gonna be great. But if I do one, plus, plus two by accident, then I get a massive shock. So, yeah, I can definitely advise you. Yeah, that's fine, do that again? - [Man] Ow! Fuck! It makes you look - To get something shocked, you might. - [Man] Volunteers? - I can do the little one. If I give you the little one, just a sec. You try that. No, no, no, try that, it'll be fine, it'll be fine. - [Man] All right. Oh, yeah, that's not bad, that's not bad. - So-- - [Man] You may want your torture back. - So, yeah, I mean, all we've done so far are like really simple IO things. But you can write like proper quite serious JavaScript programmes in here. You can use something called the HTTP Proxy service, which I've got a little Node.js app, which implements, which will allow you to use what's basically a proxy server on a RAW speed pi to get some Bluetooth onto wifi and then to actually contact the real world on the Internet. And the other thing you can do is you can use devices that actually have wifi on them. So this one's got the wifi chip built onto it and it'll even do stuff like HTTPS. Or this, which is a ridiculously cheap Chinese microprocessor thing called the ESP8066. And these you can buy them for about three dollars. In quantity, the drop to about a dollar and a half. And this will actually run Espruino on that chip. And these are tiny, they're like that size. So you can actually run like full JavaScript that connects to the Internet via wifi and does all kinds of fun things, just with that. So, yeah, it can do a whole bunch of other stuff as well. So it's got a graphics IB, other stuff built in. And I thought just to finish up, I think that's what I forgot. Yeah, this whole little hardware stuff is sported with it. This is from a few years ago, but bunches of displays, motors, GPS, GSM modems, all kinds of stuff, it's all like library for that stuff got committed on website. Yeah, so now we've got this shock machine. We can actually use that same library I had before. So the one that lets you say like puck.white to make a game much more interesting. So if I get rid of this, there's a really neat little thing on GitHub called Floppy Bird, which is just a clone of Flappy Bird. So what I've done in here is, let's see, it's not showing up. Okay, here, well, we'll disconnect. But basically there's a really handle little function in there called Player Dead. And all I've done is I've put Puck.white in Player Dead. And I give it the small shock, not like the nasty shock. So hopefully if I do this, then, yeah, you get a little shock every time. And annoyingly, this one is really, really difficult for some reason. So it's actually quite difficult to get. And I'm doing it left-handed as well. Oh, yes, okay. So, yeah, interesting party game. It's that easy to just rig things like this up. I mean, this is like literally two wires connected from here going to here. So, yeah, that's it. And I will have this stuff up if anyone wants to brave it after it's, but yeah, that's me done, and yeah.