Saturday, June 29, 2013

Video: The Rover In Action


A brief demo of the rover running around under WIFI control.

Link to higher quality Vimeo version:

Next step is some code cleanup on the client and server code so I can put it here. I hope it might be useful for someone.

Possibilities I am tossing around for next steps:

- Adding some sensors and allowing autonomous driving
- If I did decide to pursue the video preview frame some more, I could use the camera for some simple machine vision, and that would be cool.
- Adding GPS support and teaching it to drive to waypoints.
- An Android client so it can be driven from a phone instead of a laptop.

Saturday, June 22, 2013

Video streaming and services

Well, I found that I could get the camera preview callback function to work if I used it in an application instead of a service, but so far the approach to getting it to run from a service eludes me. I could make a separate video streaming app that runs as an application, but since IP Webcam already does a good job of that, I think I'll switch gears back to other functionality for a while.

I really like running the robot control code as a service. An application has to worry about screen orientation changes, screen blanking, and other events imposed upon it by the operating system that a service cheerfully ignores. I am not sure I want to give that up to implement my own streaming since I already have a working solution. Perhaps I'll come back to it later.

I am thinking of adding GPS support next.

Friday, June 21, 2013

Camera preview frames from a software service in Android

OK, so getting preview frames from a service in Android isn't as easy as I thought. It appears that it really doesn't want you getting preview frames unless they are being displayed visibly on a SurfaceView on the screen. I can't get my custom onPreviewFrame to fire. I'll drop back and see if I can get it to work with a simple non-service application - there is a very simple demo app that comes with the framework that I'll try next.

It must be possible - IP Webcam can run in the background - but it's not immediately obvious how.

Sunday, June 9, 2013

Integrated video streaming: Initial steps

Currently, the rover uses the excellent IPWebcam program for Android to serve video frames. The client just makes HTTP requests for an image in a tight loop. At low resolution (320x240) I'm getting 5-10 fps over my ancient WiFi. This works, and was easy to implement, but now that I've gotten the underlying architecture sorted out, and the rover driving well, I really want to integrate the video server. I expect this to be a challenge, but part of the point of the project is to learn about Android development. So here goes.

Yesterday, using examples on the web, I got a function working that opens the camera, takes a photo, and saves it to the SD card. The main challenge was doing this from a service, with no GUI - most examples are centered around giving the user a preview to aim with.

It turns out that the use of a preview surface is not optional - I tried using the camera API without one, and though it took pictures and returned byte arrays of varying sizes, the images were all black. Once I found a post describing how to set up a preview surface, it started working as expected.

My phone is set up to emit an audible shutter sound when you take a picture, and you can't turn it off. This is presumably for privacy reasons. I figured that since IP Webcam isn't emitting shutter sounds multiple times a second, it must be capturing the preview stream and converting it into images (which doesn't result in a shutter sound).

It appears that the previews are coming off the camera in YUV format, and each time a frame is available, it fires a callback function that you can define. It should just be a matter of converting the YUV image to a JPEG and then shoving it over the network.

For higher resolution, I'd need to investigate H.264 streaming, which I suspect is not trivial, so for now I am going to focus on the simpler approach.

Camera code: very alpha. This might come in handy for later to snap a high res picture of whatever the rover is looking at. I think I can use the camera object to turn on the flash LED to use as a headlight, too! :-) This code works from a service.

This code was heavily based on examples found on these sites and some others on StackOverflow:

private void takePicture()
  Camera cam =;
  Camera.Parameters parameters = cam.getParameters();
  parameters.set("jpeg-quality", 70);
  parameters.setPictureSize(320, 200);
  //So you can't take a picture without mapping it's preview to a surface. If you do, you get all black images.
  SurfaceView view = new SurfaceView(this);
  try {
  } catch (IOException e) {
  //give the startPreview time to complete, or you get a black image.
  try {
  } catch (InterruptedException e) {}
  //this is what gets fired on the jpeg data when a picture gets taken below
  PictureCallback mPicture = new PictureCallback() {
         public void onPictureTaken(byte[] data, Camera cam) {
          Long d = new Date().getTime();
             File pictureFile = new File("/mnt/sdcard-ext/DCIM/Camera/" + d.toString() + ".jpg");
             if (pictureFile == null) {
              Log.d(DEBUG_TAG, "Something bad happened while writing image file.");
             try {
              Log.d(DEBUG_TAG, "Byte array: " + data.length + " bytes");
                 FileOutputStream fos = new FileOutputStream(pictureFile);
             } catch (FileNotFoundException e) {

             } catch (IOException e) {
     cam.takePicture(null, null, mPicture);
     try {
   } catch (InterruptedException e) {}

Saturday, June 8, 2013

PWM Throttle Goodness

I realized that changing the motor controller input to a pin capable of PWM was as simple as flipping the connector that plugs into the motor board and then remapping it in software. It didn't even require lighting up the soldering iron. :-)

The throttle control works great at 1000 hz PWM. It makes the rover capable of much finer control. Right now I'm driving around with the mouse, and it's  a bit clunky. At some point I might order a USB joystick now that the rover side is working well.

Now that basic control is working, I have the following goals:

  • Make a decent video showing the thing driving around.
  • Integrate the video streaming into the Android app that handles the IOIO, rather than using IP Webcam.
  • Improve power management.
  • Possibly write a client program for Android as well, so you can drive it from a phone.

Friday, June 7, 2013

PWM motor troubleshooting

Well, flush with my easy success of making the LED controlled with the PWM, I set about modifying the code to drive the motors that way. I changed the relevant digital IO pins to PWM outputs, set the duty cycle in each function that controls the motors, and.... nothing. Nothing at all. Yeah...

Some troubleshooting led me to the chart at
that shows which pins can be used as PWM outputs. I made a basic error - I connected one of my 4 motor driver pins to pin 8, which is not PWM capable. The interesting part was that it didn't just disable the motor controlled by pin 8 - by trying to set that pin to be a PWM output in my program, it disabled ALL PWM functionality.

As soon as I commented out the references to pin 8, the other motors spun up just fine, and are nicely speed controlled. It's easily fixed - I'm going to swap the motor controller input on 8 to pin 11, and update the code. But it's late, and that can wait until tommorow. :-)

IOIO PWM outputs... or a network dimable LED!

The more I work with the IOIO, the more impressed I am with it. It's just a joy to code for - so well thought out and documented.

It turns out PWM is very easy. PWM (Pulse Width Modulation) refers to hardware on the IOIO that generates regular pulses of varying "on" durations. This can be used to simulate analog voltages, drive servos, or rapidly pulse the digital inputs on a motor controller to control motor speed. It's this last use that is my immediate goal.

In reading the IOIO wiki at I learned that the yellow status LED on my board is hooked to a PWM output. If you apply PWM to an LED, it dims as you change the duty cycle. Perfect. My little client program already has a JSlider component that has a value of 0-100 that's sent over the network to the rover as part of the command string. It was really easy to set up and use that value in my IOIO loop.

//initialize the pin as a PWM output
private PwmOutput led_;

//in the IOIO loop, set the duty cycle on that pin according to the value of the JSlider. Scale it to 0-1.
float dc = (float) (servoPanValue/100.0);
led_.setDutyCycle(dc); //0-1

This results in the LED being dimmed to varying brightness (off to 100%) as you move the slider. Too cool. Now I need to update my motor driver code to set up the pins as PWM outputs rather than pure digital IO to pulse the input pins on the motor controller board.

Sensor values and Metrics

Since my goal is to learn about controlling vehicles remotely over the network, I decided it was time to add some simple metrics to the client, and to return my first sensor value from the rover (aside from video).

I added some timing code to compute the video frame rate, the rate at which commands are being sent to the rover, and added a text display of the rover's current signal strength.

The control protocol is very simple. The client sends a string of commands, and the rover replies with a string of sensor values. Currently this only contains an integer representing signal strength, but will eventually include voltages from the IOIO, GPS data from the phone, orientation information from the accelerometers, etc.

Here's a quick snapshot of my little client program in action. Next step is to figure out how to use PWM on the IOIO board to control the rover's motor speed.