Ever tried waving your mobile phone around in the air for a better signal? Of course you have. But if you see assistant professor Romit Roy Choudhury of Duke University in North Carolina apparently doing so, he's not worrying about poor reception. He's using his phone as an electronic pen to write reminder notes.

In 2005, Choudhury was a PhD student at the University of Illinois working on computer science. Often forgetting things, he wanted to have a quick and easy way of jotting information down.

"I envisioned having a pen with a wireless interface and an accelerometer. My idea was to be able to write in the air with the pen, and click a button to email the handwriting to my email address. Such pens were unavailable, and I shelved the idea," says Choudhury.

From the pen to the phone

Since moving to Duke University, he's revisited the concept thanks to accelerometers in the latest smartphones. An accelerometer senses positional changes: it's the device that prompts the screen image to flip from portrait to landscape when the phone is turned sideways. It also enables games with repetitive movements.

"At that point, I had the idea of using the phone as a pen, since the phone has both the accelerometer and the wireless capability," says Choudhury. "My students jumped on to the idea, and did a wonderful job of turning it into a good prototype. This first round of prototyping took around six months."

Sensing the phone's orientation or playing a game is trivial compared to recognising air writing. His students started by using the phone to gesture a simple square, soon finding it was tough to achieve the right shape. "Then, one day, after innumerable revisions to the algorithm, we suddenly saw a near-perfect square."

Working through the alphabet (and numbers) has proved challenging for Sandip Agrawal, electrical and computer engineering senior at Duke's Pratt School of Engineering and graduate student Ionut Constandache. The standard Nokia N95 being used for the experiments has a three-axis accelerometer to detect the X, Y and Z directions.

The students wrote a Python script to obtain 30 to 35 instantaneous acceleration readings a second, and taking an average to overcome "noise". Using a wireless link, the readings from the phone were processed on a separate computer with MATLAB, a high-level computing language for mathematical computation.

The prototype software now recognises capital letters, numbers, and simple shapes. It works by interpreting the geometric shapes and the order in which the user gestures them. An A, for example, consists of three distinct strokes. Every stroke, angle, and lift of your pen (the phone) is significant if you're trying to interpret air writing on an imaginary plane (the paper). As the phone doesn't have a gyroscope, there are difficulties in distinguishing rotational motion from linear acceleration if the phone is turned in your grip.

At the moment, the phone wirelessly sends the recorded data to an adjacent computer for processing. It quickly returns the results to the phone as text, image, and an email. You also have to write large, distinct letters – not joined-up – for the recognition to work.

"The computations are simple enough to be done on the phone. It's just a matter of time," says Choudhury. "We simply didn't have time to port the code since we are working on so many interesting projects."

So what are the remaining challenges for the device, now dubbed the PhonePoint Pen by the researchers? Correcting mistakes when writing is an obvious one: the idea is to allow users to choose a movement that means "delete", such as several horizontal shakes. When writing on the move, the phone's accelerometer picks up your movement as well as its own, distorting the output. Better algorithms, more sophisticated built-in accelerometers, and a gyroscope would all help.

Air writing is useful for noting where you left the car at the airport or quickly jotting down an appointment time. Following recent interest from the medical profession, Choudhury also thinks it might help people with poor finger control or speech problems.

So when will the researchers release the prototype software for people to try out? "We believe that will be possible in few months," he says.

"We have the current prototype working on Nokia N95 – but the technology can be used for any programmable smartphones with small tweaks."

All done in two shakes

Robert Hardy, a research associate in the computing department at Lancaster University, sees the advantage of one-handed mobile phone use which doesn't (unlike text messaging) need your close visual attention. Leaving aside the remaining technical work, he also believes social factors might come into play. "Concerning the latter issue, it may be possible that the user feels embarrassed using this prototype in public places," he says.

Another example of mobile phone gesturing was tried by two Lancaster University researchers – it involved shaking two phones together to initiate easy Bluetooth pairing. The emerging technology of near field communication relies on mobile phone touch gestures for payments, ticketing, and door access. However, air writing uses a much larger free space area.

Hardy adds that, "by taking advantage of the user's familiarity of physical interactions in the real world, mobile phone gestures can be very intuitive. Relating back to the prototype, the barriers to entry could be potentially low as users are very familiar with handwriting in daily life."

Many people are already happily using lifelike gesture control for playing golf or tennis on the Nintendo Wii console. But there's a difference between gaming at home and using your mobile phone in public as an electronic aide-memoire. Even if you're not embarrassed doing so, writing words in thin air is definitely going to earn you some strange looks.