Loredana Paglialonga leans across her father's prone body and whispers in his ear: "Spinta, Papi, spinta" ("Push Daddy, push"). It is impossible to tell whether Anselmo Paglialonga, a former major in the Italian carabinieri, has heard. Paralysed from head to toe with amyotrophic lateral sclerosis (ALS), Anselmo is completely "locked in". Unable to speak or open his eyes, his only hope of communicating is via a state-of-the-art neuroheadset attached to his scalp.

Designed by Emotiv Systems, a Californian neuroengineering company, the Epoc headset purports to give users the power to control objects with their thoughts and was a succès fou on Kickstarter, where users stumped up $1.6m to fund its development – 16 times Emotiv's original target.

Indeed right now, I am told, the 14 marble-sized electrodes in the Epoc's plastic clip-on frame are monitoring the EEG signals from Anselmo's brain and sending them wirelessly to a control unit. By analysing those signals using a machine-learning algorithm, BrainControl, a Sienna-based developer, claims to be able to distinguish Anselmo's thoughts and intentions from other brain noise and use those signals to operate a cursor on a tablet computer. The cursor has three positions: "si", "no", and "non so" ("yes", "no", and "don't know"). In theory, all Anselmo has to do to move the cursor from one position to another is to think "push".

"I usually ask people to imagine pushing an object with their mind," says Pasquale Fedele, the engineer who designed BrainControl's machine-learning algorithm. "Once we have captured the thought "push", the next stage is to train the subject to get control of the cursor."

According to Loredana, Anselmo, who is 77 and has had ALS for 11 years, has previously used BrainControl to correctly name his five-year-old grandson, Theodore, who was born after he became locked-in, and to inform the family that he is suffering dental pain. It is because Loredana is convinced her father is cognitively aware and that BrainControl is his only hope of communicating that she has invited Fedele to her home in Bari to install the latest version of BrainControl's operating system.

"If it wasn't for BrainControl, we wouldn't have known that Anselmo was in pain and that he can still see, even though he can no longer open his eyelids unaided," she tells me. "Before he started using the headset, doctors at the hospital in Bari had diagnosed him with dementia. Now we know that is not true and that he is fully aware of what is going on around him."

I first met Fedele and his business partner Jarrod Fath at a "masterclass" for digital health startups in London in November 2013. Wearing a mismatching jacket and tie and with his low-key delivery, Fath cut a far less assured figure than his corporately attired competitors but, though BrainControl failed to place in the top three, his presentation was, to my mind, the most intriguing.

How could a device funded by Kickstarter and incorporating nothing more than a clever machine-reading algorithm read the very stuff of consciousness? Was this just the way medical breakthroughs happened in the era of Big Data or was the Epoc headset the modern equivalent of one of those quack phrenology devices from the early part of the 19th century that claimed to be able to read personality from the shape of a person's cranium?

Fath's pitch at the masterclass was certainly persuasive: worldwide, some 3.7m people suffer from severe communication and learning disabilities due to conditions such as ALS, stroke, multiple sclerosis and traumatic brain injury. Of these, approximately 660,000 live in the developed world, he explained. The majority communicated using eye-tracking systems but for those who could no longer open or move their eyes and who were effectively imprisoned inside their bodies, neurosensing technology was their only hope. "BrainControl gives disabled people the power to control objects with their thoughts," said Fath. "These thoughts are acquired by an EEG headset, interpreted by a computer and can be used to control just about anything from a communicator to home automation or robotics. Think of it as like a mental joystick."

Afterwards, I approached Fath and Fedele for more information. I was keen to understand precisely how the Epoc headset and their brain-control interface (BCI) worked and whether the system had been tested in a laboratory setting. Fedele and Fath responded that no, until now trials had been on an ad hoc basis using patients recruited by word of mouth, but that they hope to conduct a controlled study with a specialist neurology hospital shortly. To date, 21 people with tetraplegia (paralysis of all four limbs and torso), 18 of whom are in a locked-in state, had experimented with BrainControl, mostly with positive results, they said. However, when I pressed them about how precisely the headset and control panel could distinguish between different cortical outputs, including those generated by facial muscles, their response left me uneasy. The technical specifications of the headset and details of the background research were available on Emotiv's website, they said. They had simply licensed a software development kit from the company and tweaked Epoc's "Cognitiv Suite" using "an adaptive Bayesian algorithm" that could be customised for "different needs in different patients". Unfortunately, as this algorithm was a "new proprietary method", Fedele and Fath could not give me further details.

ALS is a neurodegenerative disease that usually attacks both upper and lower motor neurons and causes degeneration in the brain and spinal cord. First described in 1869 by the French neurologist Jean-Martin Charcot, ALS attacks about one person in 100,000. Between 1-2% of these cases are associated with the mutation of a specific gene, SOD1, that is believed to make a defective protein that is toxic to motor nerve cells. A further 10% of cases run in families. The vast majority are a mystery, however.

ALS tends to affect people between the ages of 40 and 70 and men more than women. A common early symptom is a painless weakness in a limb or difficulty with speech or walking. However, this weakness quickly spreads to other parts of the body, causing progressive paralysis of the legs, trunk and diaphragm, at which point patients may elect to have a tracheotomy so their lungs can be ventilated mechanically. Typically, this degeneration takes three to five years, with the head and face being the last part of the body to be affected. When a patient cannot speak or move any part of their body, including their eyes, they are deemed to have "completely locked-in syndrome" (CLIS). In the US, where up to 30,000 Americans have the disease at any one time, past victims include the baseball player, Lou Gehrig – hence the alternative name "Lou Gehrig's disease" – while in Britain, past victims include the actor David Niven. (Italy, for reasons that are unclear, has one of the highest incidence of ALS in Europe, with unusual clusters having been reported in professional football players and agricultural workers.)

Caring for a person with ALS is a full-time commitment. Patients have to be washed and turned regularly, and in the latter stages, when it becomes impossible to chew food or swallow, a feeding tube has to be inserted in the stomach to ensure a steady supply of nutrients. One of the most difficult moments comes when the lungs begin to fail and the patient, if they are able to communicate, or their family must decide whether to connect them to a ventilator. Once you elect to have a tracheotomy there is no going back. In theory, the ventilator can keep you alive indefinitely, even if you slip into a coma.

For obvious reasons, there have been few dispatches from this nether world and fewer still that have attained literary permanence. The exception is The Diving Bell and the Butterfly, the remarkable memoir by the former French editor of Elle, Jean-Dominique Bauby, which he dictated by laboriously blinking his left eyelid, the only part of his body that was not paralysed, as a colleague held up letters arranged in order of their frequency in the French lexicon. In this way, blinked Bauby: "My cocoon becomes less oppressive, and my mind takes flight like a butterfly."

But Anselmo, unlike Bauby, is completely locked in. Although in the early stages of his illness he could use eye-recognition software to spell out words and phrases, he can no longer open his eyes and for all intents and purposes is a prisoner in his own body.

Not surprisingly, families of ALS patients are desperate to find a way of releasing their loved ones from this purgatory. In a Catholic country such as Italy, where religious belief runs deep, the search for a solution can become a test of faith. For Loredana, the second of Anselmo's four daughters, that search led her to the Italian ALS association, which in turn put her in touch with Fedele. It was in response to her approach that Fedele decided to take a closer look at Emotiv's Cognitiv suite to see if there might be a way of adapting it for patients with locked-in syndrome.

Two years before, in July 2010, Tan Le, the 37-year-old Australian co-founder of Emotiv, had wowed TED Oxford by showcasing the Epoc headset and Cognitiv suite on stage. Accompanied by a collaborator, Dan Evans, Le demonstrated how with just a few seconds' training Evans could make a virtual cube appear and disappear just by thinking about it. The demonstration drew cheers from the Oxford audience and gleeful hand-clapping from Le. However, the technical details of precisely how the Epoc had enabled Evans to "control a virtual object with his mind" were never addressed. Instead, Le finished her talk with a slideshow of the headset's diverse applications and an invitation to developers to license the technology.

One of the advantages of the Epoc headset is that it retails for just $300 – a fraction of the price of a clinical grade EEG – and within months of Le's talk Emotiv's sales were booming. Emotiv's customers included everyone from gamers to car and aerospace companies, to home experimenters attracted by the idea of using the headset as a "mind-training" device to improve their powers of concentration or to perform party ticks, such as operating a remote-control toy helicopter. For the most part, these early adopters were unconcerned by precisely how Emotiv's machine-learning algorithm was able to "read" thoughts. All they knew is they liked it and it appeared to work.

Professional scientific researchers were also intrigued by Epoc's potential. Ever since the German neurologist Hans Berger discovered electroencephalography in the 1920s, psychologists have dreamed of being able to read thoughts and intentions from EEG traces. This research took a huge leap forward in the 1960s with the discovery that biofeedback mechanisms could be used to prompt the self-regulation of slow cortical potentials (SCRs) – changes in brain voltage emanating from nerves in the cortex – opening up the potential for the self-control of epilepsy and other neurological conditions. At the same time, researchers began looking at brain control interfaces to communicate with locked-in patients.

However, even with clinical grade EEG sets it is difficult to avoid recording muscle movements (when concentrating on a mental activity, people tend to furrow their brows and make involuntarily facial movements – see box). It is also notoriously difficult to design experiments in such a way so as to exclude the possibility of subliminal signalling and placebo effects. Nevertheless, in some experiments scientists reported that partially locked-in patients were able to achieve up to 70% accuracy, so we know that it is possible to communicate with someone via EEG. However, these levels have only been achieved by a few patients after extended periods of training in a controlled setting and to date no CLIS patient has ever satisfactorily demonstrated control of a brain signal. Little wonder that scientists who have spent a lifetime studying BCIs are dismissive of the hype surrounding the Epoc headset and BrainControl.

"Emotiv and BrainControl do not have not the slightest scientific evidence that the technology can do what they claim," says Niels Birbaumer, director of the Institute of Medical Psychology and Behavioural Neurobiology at the University of Tübingen. "We tested [the headset] on an ALS patient in our lab and concluded that in most cases it is not measuring EEG but eye movements and muscle movements."

Klaus-Robert Müller, co-head of the Berlin Brain Computer Interface at Berlin Technical University, is similarly dismissive of the hype surrounding the Epoc, telling one journalist: "If you have a small car and you put a Mercedes label on it, that doesn't make it a Mercedes."

Although Emotiv claims that the Cogntiv Suite allows users to "manipulate objects only with the power of their thought", its website makes clear that Epoc is not a medical device. It also warns developers who are thinking of working with disabled people that, while the headset "is capable of being used in a support setting", the risks lie entirely with the developer and end user. Emotiv's disclaimer continues: "All mental detection systems suffer from a finite rate of false positive and false negative detections. In other words, sometimes detection can occur without the user's deliberate intention, and sometimes a deliberate attempt to make an action occur may fail or select a different output. This is as much to do with the user's level of training and state of mind as it is to do with the detection systems – it is unavoidable in any BCI system."

So what is going on here? Is the Epoc recording pure brain wave activity or some combination of EEG and muscle movement? And does it really have the potential to transform the lives of people with locked-in syndrome?

I put these questions to Tan Le at a developer event in San Francisco in February where she was unveiling a lightweight version of the Epoc, called Insight, comprising just five EEG sensors instead of 14. Le is no longer involved in the day-to-day management of Emotiv Systems, having stepped down from the board in 2010 to concentrate on a new company, Emotiv Lifesciences. But to all intents and purposes Le remains the public face of Emotiv.

To my surprise, Le freely admitted there had been no controlled clinical trials. Instead, the initial studies on what would become the Epoc were conducted at a Melbourne lab under the auspices of an "independent review board" using student volunteers. However, while acknowledging that the hype generated by her TED talk may have raised expectations about the Epoc to unrealistic levels, Le made no apologies for encouraging developers to experiment with the Cognitiv suite. "We say, look, this is not a medically approved device, it's not a diagnostic tool, it's a consumer electronics product. But it does have these features that you can train up a machine-learning algorithm. If people are willing to try it and there's no other alternative we are not going to say no."

She also said that it was irrelevant whether the Epoc was also recording muscle artefacts, as even clinical grade EEGs could not eliminate all sources of cortical "noise" and what mattered was that the algorithm was able to train correct correlations. "We don't hide the fact that we are classifying the noise artefacts that are normally superimposed on the EEG," she told me.

Finally, Le directed me to several independent peer-reviewed papers featured on Emotiv's Wiki. These show that in laboratory tests the Epoc performed "significantly better than chance", suggesting that the system could distinguish intentional thoughts from other mental activity and artefacts, such as muscle movement. But another study stated that the Epoc was "not as good" as a professional clinical grade EEG, while in another its performance had been "significantly worse".

The Paglialongas are a close, religiously observant family (the Madonna occupies pride of place above Anselmo's bed). The main burden of Anselmo's care falls on his wife Anna, but he also has a series of nurses who visit each day to check on his ventilator, change his feeding tube and monitor his heart rate and blood pressure.

Anselmo's job in the carabinieri was to oversee anti-mafia operations in Calabria and before his body began to fail Loredana says he led an active life. It was in 2002 that the family first noticed a problem. The loss of motor control began in Anselmo's mouth and throat, then spread to his arms and legs. Doctors at the hospital in Bari were baffled and Loredana had to take her father to a specialist in Marseille for a full neurological workup. At first, Anselmo took the diagnosis of ALS badly.

"The idea that he might not be able to walk or talk again made him very depressed," said Loredana. "He told me he felt it might be better to die." With the support of his wife and children, however, Anselmo gradually came to terms with his condition.

In the beginning, Anselmo wrote or typed messages but by 2004 he had lost the use of both hands and switched to an eye-tracking system that enabled him to pick out letters and phrases on a Plexiglas screen. Then, when in 2005 his lungs began to fail, he elected to have a tracheotomy. As the paralysis spread to his face and he began to lose the ability to open his eyelids, however, Loredana grew desperate. Without the use of the eye-tracking system, the only way of communicating with Anselmo was by holding his little finger – the only part of his body where he still had some movement. Then, in 2012, she heard about BrainControl. After trialling the service with her mother, Loredana became a convert and Anselmo has been using BrainControl ever since.

Unlike a clinical grade headset where electrodes are attached to the scalp by tape, the electrodes on the Epoc have to be primed with saline in order to ensure a good contact. Only once all the electrodes on the screen turn green can the BrainControl training begin. The idea is that when relaxing and not engaging in any motor movement, the Epoc can form a picture of a subject's resting mental state – the brain in neutral, as it were. Once this benchmark has been established, the next stage is to train the "push" action by asking the subject to imagine moving an object forward in his mind for more than three seconds. The resulting decreases and increases in electrical rhythms produce another set of readings known as event-related desynchronisation (ERD) and event-related synchronisation (ERS). Through repeating this process, BrainControl learns to distinguish the "push" thought from neutral and move the cursor from the "si" to "no" position. In practice, it doesn't matter what thought you use. As long as they are considerably different from another, BrainControl will detect the difference.

To verify that BrainControl is detecting intentional thoughts and that the cursor is not simply responding to random outputs, Fedele puts subjects through a series of eight-second trials, during which they are asked to select the options on the tablet in sequence. The object is to hold the cursor on "si" for three seconds, followed by a five-second pause, followed by "no" for three seconds, followed by another five-second pause, and "non so" for three seconds. Only after five repetitions, during which each option has consistently been selected for more than three seconds, is the subject considered to be in control of the tablet and actively communicating with the trainer.

Unfortunately, on the afternoon I visited it was impossible to tell whether Anselmo was moving the cursor intentionally or whether the illuminations were the result of random electrical rhythms (initially, Anselmo seemed to illuminate the "si", followed by "no" and "non so", but never for more than two seconds and on the third repetition the cursor got stuck on "no", suggesting Anselmo was either concentrating furiously or had fallen asleep). Loredana also struggled to find a comfortable position for Anselmo's head and it took much rearranging of his pillows to ensure the headset was secure and the electrodes were transmitting a continuous signal. I also noticed that throughout the experiment she was never far from her father's side, constantly stroking his hand and whispering words of encouragement. At times she even claimed to be able to read her father's thoughts from small movements of his mouth and eyes, calling into question the extent to which he was really "completely" locked in.

Eventually, after consulting with Loredana, Fedele called a halt to the experiment, suggesting I return the following morning when Anselmo had had a chance to rest. But the following morning Loredana informed me that her father was feeling poorly and that it would not be possible to continue. "It is not always easy to get control of the cursor," she said. "You have to be patient and practise."

I wanted to believe Loredana but when Fedele allowed me to experiment with the headset my concerns deepened. I began by trying to train the "push" thought. After several minutes' practice, I was able to get the cursor to illuminate the boxes more or less in sequence. However, holding "push" for more than two seconds was a struggle and I became aware that the harder I tried to control the cursor the more I furrowed my brow. Fedele assured me that my performance would improve with training but BrainControl does not come cheap: families can opt for a single payment of €7500 or elect to pay a service fee of €480 per month.

On my return to London in April, I sent Fath and Fedele an email expressing my scepticism about what I had seen in Italy and asking them to respond to Birbaumer's criticisms. Fath replied by sending me a link to a training video showing another patient appearing to make five correct selections in sequence. In a separate email, Fedele told me they were now working with a new patient in southern California who, while not completely locked in, was making good progress with BrainControl.

In response to the specific criticism that the Epoc was recording muscle artefacts, Fedele wrote: "Yes, EEG signal [sic] can capture facial movements and the headset can detect head rotation, but the BrainControl app uses one output only, and that is the EEG signals related to movement imaginations." He added that he and Fath were keen to collaborate with Birbaumer but "just haven't had time". Finally, he said BrainControl had signed an agreement to conduct controlled studies with several Italian hospitals.

One cannot blame the families of locked-in patients for being willing to experiment but the history of medicine is riddled with devices and technologies that seemed to promise miracles only to be shown later to be worthless. That is why medical science, if it is to command confidence, must proceed cautiously and why scientists such as Birbaumer who have been studying assistive technologies all their life are so sceptical of the claims made by Emotiv and BrainControl. As Birbaumer told me: "I do not know of a single case where a completely locked-in patient has been able to communicate with EEG. A lot of the time you have no way of telling whether they are awake or asleep."

However, the apostles of Big Data and wearable tech do not see themselves as scientists so much as innovators and entrepreneurs. The fact that these experiments are now taking place in the privacy of people's homes far from the scrutiny of university ethics committees is a good thing, they argue. Indeed, Le told me that in her view EEG research had been the preserve of "sterile medical clinics" for far too long.

But the problem with this open-source, market-led approach is that it can take no responsibility for the impact – for better or worse – that these devices have on end users, or on the fervent hopes and desires of their families. Instead, it can reduce people like Anselmo to data points in a social and medical experiment – one that, at present, may rest as much on faith as science.

* On 26 August 2014 this article was amended with the correct prices for the BrainControl service