Tuesday, February 21, 2006

How To: Use Bluetooth to Take Pictures & Drive a Robot

In the previous article, we introduced how the robot could interact with you via your Bluetooth phone. There are a number of other commands that may be useful when the robot controls the mobile phone. The commands presented here are based on the Sony Ericsson T630 mobile phone.

Using the Camera

If the phone has a camera, then you can take a picture by sending the command:


The first time you send this command, the phone will move into camera mode. The second time you send it, the picture will be taken. The third time you send the command, the picture will be stored in the phone's "My Pictures".

To get back from the camera mode to the main screen, use this command:


So, to take a picture, save it, and return to the front screen, your script would be to send the following as ASCII to the serial port:


Each command must be on a new line. Each command will return an "OK".

Showing a Message

If you just want to show a message on the screen of the phone, for instance as a debug message, then you can use the following command:

AT*EAID=1,0,"the text of your message goes here"

The command immediately returns "OK". When the user presses OK button, you'll receive the code "+EAII: 1,1" over the serial port. If the user presses the cancel key, then you'll receive the code "+EAII: 0".

Getting Direct Keyboard Feedback

To control a robot, it is really useful to be able to read the keyboard directly. Using the robot in the image, suppose you used the following keyboard mapping:


To start reading the keyboard directly, use the following command:


The command will immediately return "OK". For the rest, it will return the following code, where the k is the key you pressed, and p is 1 when you start pressing, and 0 when you release it. So when you start pressing 1, the serial port returns “+CKEV: 1,1”. As soon as you release the key, it returns "+CKEV: 1,0". Here’s the complete map of the keyboard:


+CKEV: R,p

+CKEV: <,p+CKEV: J,p+CKEV: >,p
+CKEV: v,p

+CKEV: c,p

+CKEV: 1,p

+CKEV: 2,p

+CKEV: 3,p

+CKEV: 4,p

+CKEV: 5,p

+CKEV: 6,p

+CKEV: 7,p

+CKEV: 8,p

+CKEV: 9,p

+CKEV: *,p

+CKEV: 0,p

+CKEV: #,p

You can press two keys at the same time, and both will be sent to the serial port.

To stop listening to the keyboard, use the command:


Putting It All Together

Finally, to demonstrate the above concepts, here is a flowchart that turns your mobile phone into a live remote control for your robot. This software runs entirely on the NXT robot, there is no programming required in your mobile phone.

This program results in the following screen on the phone:

And below you see the commands sent to the phone, and received back. As you can see, I first pressed '2' to go forward. Then, I stopped. Next, I went forward again, but this time, while keeping '2' pressed down, I also turned left by pressing '4'. The robot would now turn left while still going forward. I released the left key, which made to robot go forward again. Then, I release the forward key. Finally, I pressed the OK button, which terminated the program.


At February 21, 2006 7:36 PM , Blogger Koshdukai Yordarth said...

hmmm... now there's some new AT commands for me :)

Can't wait to try them on my T610.

Thxs for these tips Filip !

At February 21, 2006 9:23 PM , Anonymous Filip said...

Thanks for the feedback!

At March 02, 2006 3:56 PM , Anonymous Julien said...

Thank you Filip for the digging into the bluethooth protocol! This feature is very exciting!

Are the AT commands specific to your phone model, or is it part of the bluetooth protocol when communicating with a phone?

At March 02, 2006 4:38 PM , Anonymous Filip said...

Hi Julien.

I think these commands are general, but I have no way of testing it on different types of phones/PDAs. Especially taking a picture (and whether or not it stores it using those commands) may be phone-dependent.

If anyone has a Bluetooth-enabled PC and Bluetooth-enabled phone, it would be nice if they could follow the intructions in the article and let me know whether it works for they phone too!


At May 19, 2006 12:38 AM , Anonymous Drew said...

I tried my 3 BT devices - 2 motorola phones and a palm - Here is the first round set of results but I plan to be more thurough (Motorola has a Developers site but I got lost lots so I'll start with teh alphabet for now).

Through experimenting here are the things I learned:
Different com #’s connect to different Bonded Bluetooth dvices.
Motorola Phones have similar syntax but different details to Eriksons.
That some commands have little or no effect on each screen (ie back would not do anything when your on the desktop)
I plan to conduct a much more detailed analysis but I went through the alphabet and 1,2,3 and 0 and got these results on a motorola rokr. ( I customized my menus so I am not sure if these match a function or a key stroke. I will change my buttons around later to test if its a key press or a hardwired functionality.

COM 24 – ROKR- Trying Phone top screen and Letters AT+CKPD=":X”
x= Message center (up?)
c = phone book or down
d= Ring down or vol button down
j= Send Button
L=dialed calls or left
R = right direction or Java
U = Vol up or vol up button
1= cam (twice to click shot) 3 to get store screen but no save yet need down scroll

I tried a motorola flip phone as well with these results

Flip Phone is COM 25
c is down
TO shoot and store a picture
zer0 is the select or center button or select

Assume Palm is Com 23 but no response and freezed program

I will try more later and try to present a map for everyone.
Others please map your brands and devices too then we can make a rosetta stone (using the brand return comand detailed above we should be able to make Mindstorms scripts that run on DIFFERENT brands by requesting that info and then using the response to decide what programing branch (and thus comands) to use to do the same function.

At June 30, 2007 11:18 PM , Anonymous Jack said...

This is cool... but most of the commands didn't work for my phone. I have a Sony Ericcson W300i, and it just keeps returning "Error"

At September 22, 2007 11:40 PM , Anonymous iostream said...

I'm using a Sony Ericsson T630, but in the HyperTerminal program I first have to type:
which returns an error, before functions like AT*EASM and AT*EAID will actually show something on the phone.
AT+CKPD=":C", AT+CMER=3,2,0,0,0 etc works without typing that.

When sending text from the NXT to the phone, the only function that works is AT+CKPD=":C" which launches the camera.

But I have to run the program twice for it to open, twice to take a photo and twice to save it.
No other functions works, it doesn't even return something.

Anyone know what the problem is?


Post a Comment

Subscribe to Post Comments [Atom]

Links to this post:

Create a Link

<< Home