Now we'll modify the draw() function to do more with the serial data stream than just print it to the console. We want to reverse the procedure we did on the Sparki. There we read every pixel and sent ASCII characters to represent it. Now we want to read the ASCII characters and use that information to draw pixels in the Processing window.

So we'll first look for the <img> and </img> tags that start and end our image transmission. That will tell us when to start and stop reading lines to look for pixel data. Once we've found the <img> tag, we'll set a variable receivingImage to true . When we find the </img> tag, we'll set receivingImage to false .

While receivingImage is true, we'll process each line of pixel data, (representing one row on the screen) and read each character from left to right to determine if that pixel should be on or off. We also need to keep track of what line we're on (the y-coordinate) so we'll use a variable called lineIndex for that. These are declared at the top of the sketch.

When we read a pixel that should be turned, we'll draw that pixel on the screen. Remember that we enlarged our display in Processing by a multiple of 4 so that it would be easier to see. We also have the pixel aspect ratio to think about. So each Sparki pixel will actually be a rectangle 4 wide by 6 high when we draw them to the screen in Processing. In order to make that simpler to work with, I declared a pixelWidth and pixelHeight variable at the top of the sketch.

Take a look at the new Processing sketch at this point:

/* CaptureSparkiLCD.pde Receive a screenshot of the Sparki LCD over the serial port and save it as a PNG image. */ import processing.serial.*; Serial myPort; short portIndex = 0; //select the com port //Pixel aspect ratio = 1:1.5 = 2:3 = 4:6 static final int pixelWidth = 4; static final int pixelHeight = 6; //Image status boolean receivingImage = false; int lineIndex = 0; void setup() { //Width: 128 Sparki-pixels * 1 aspect-ratio * 4 multiplier = 512 Processing pixels //Height: 64 Sparki-pixels * 1.5 aspect-ratio * 4 multiplier = 384 Processing pixels size(512, 384); background(7, 23, 197); //Red 7, Green 23, Blue 197: To match Sparki's blue LCD screen connectSerial(); } void connectSerial() { String portName = ""; String[] ports = Serial.list(); println("Available ports:"); println("portIndex portName"); for(int i=0; i<ports.length; i++) { println(i + " " + ports[i]); } println(); print("Connecting to port index " + portIndex + ", "); try { portName = Serial.list()[portIndex]; println(portName + " ..."); myPort = new Serial(this, portName, 9600); println("Connected."); } catch(Exception e) { println(""); println("/!\\ Could not connect."); println(""); println("Note:"); println("If the serial port can't connect, make sure that Sparkiduino's Serial Monitor is not open."); println("You can also change the port index by changing portIndex at the top of the Processing sketch:"); println("short portIndex = " + portIndex + "; //select the com port"); exit(); } } void draw() { if(myPort.available() > 0) { String line = myPort.readStringUntil('

'); //read one line of data from the Serial connection into a String variable print(line); if(line.trim().equals("<img>")) //If this line consists of the <img> tag that starts an image transmission... { background(7, 23, 197); //Reset the draw window to a blank blue background. receivingImage = true; //Set the flag that means the next lines have image data (until we find </img>) lineIndex = 0; //Set the lineIndex to 0. This represents the y-coordinate of the pixel. } else if(line.trim().equals("</img>")) //If this line consists of the </img> tag that ends an image transmission... { receivingImage = false; //Set the receivingImage flat to false - stop looking for image data lines. } else if(receivingImage) //Otherwise, if the receivingImage flag is true... { int length = line.length(); //Find out how many characters are in this line. for(int x=0; x<length; x++) //Loop over every character { if(line.charAt(x) == '*') //If the character is a *, representing a white pixel... { noStroke(); //Turn off lines around boxes - just fill the exact rectangle size we specify below. fill(240, 240, 240); //Set the fill color to off-white. Full white would be 255, 255, 255. //Draw the rectangle that represents this pixel. pixelWidth and pixelHeight help represent the pixel's aspect ratio. rect(x * pixelWidth, lineIndex * pixelHeight, pixelWidth, pixelHeight); } } lineIndex++; //Increment the lineIndex, equivalent to moving the y-coordinate to the next row of the image. } } }

Do you notice how the image that is created doesn't quite look like the image on Sparki's LCD? If you look closely at Sparki's LCD, you can see a tiny little line running between all the pixels. In order to make the Processing display look more like Sparki, I changed the drawing code to:

First draw a ghosted out white rectangle the full size of a pixel. Then draw a second rectangle at full opacity, and one pixel less in width and height than the first box.

So the code that draws the rectangle changes to this:

if(line.charAt(x) == '*') { noStroke(); fill(240, 240, 240, 128); //Set the color to off-white, half-transparency rect(x * pixelWidth, lineIndex * pixelHeight, pixelWidth, pixelHeight); //Full pixel box is drawn fill(240, 240, 240); //Set the color to off-white, no transparency rect(x * pixelWidth, lineIndex * pixelHeight, pixelWidth-1, pixelHeight-1); //Draw a pixe box one pixel smaller in width and height }

That results are now much closer to what the Sparki LCD looks like. (See the second image above.)