Facebook Augmented Reality device to let you type with your mind

San Francisco: Facebook is developing a brain-computer Augmented Reality (AR) interface device that would help users type with their mind.

At its F8 Developers’ Conference in 2017, the company announced its Brain-Computer Interface (BCI) programme – outlining its goal to build a non-invasive, wearable device that lets people type by simply imagining themselves talking.

Facebook is supporting a team of researchers at University of California, San Francisco (UCSF) who are working to help patients with neurological damage speak again by detecting intended speech from brain activity in real time.

In a paper appeared in the journal Nature Communications, the UCSF team “has shared how far we have to go to achieve fully non-invasive BCI as a potential input solution for AR glasses”, said Facebook in a blog post on Tuesday.

The UCSF team has been able to decode a small set of full, spoken words and phrases from brain activity in real time – a first in the field of BCI research.

The researchers emphasise that their algorithm is so far only capable of recognising a small set of words and phrases, but ongoing work aims to translate much larger vocabularies with dramatically lower error rates.

“The promise of AR lies in its ability to seamlessly connect people to the world that surrounds them and to each other. Rather than looking down at a phone screen or breaking out a laptop, we can maintain eye contact and retrieve useful information and context without ever missing a beat,” Facebook added.

As Chief Scientist Michael Abrash and the team at Facebook Reality Labs (FRL) see it, “we are standing on the edge of the next great wave in human-oriented computing, one in which the combined technologies of AR and VR converge and revolutionise how we interact with the world around us”.

“It is going to be something completely new, as clean a break from anything that has come before as the mouse/GUI-based interface was from punch cards, printouts, and teletype machines,” said Abrash.

The aim of the BCI research programme at Facebook Reality Labs is to develop a non-invasive, silent speech interface that will let people type just by imagining the words they want to say – a technology that could one day be a powerful input for all-day wearable AR glasses.

Ultimately, the researchers hope to reach a real-time decoding speed of 100 words per minute with a 1,000-word vocabulary and word error rate of less than 17 per cent.

Facebook first announced in 2017 that its research lab, Building 8, was working on a computer-brain interface.

The Facebook programme comes on the heels of Elon Musk-led startup Neuralink’s bold research that has revealed tiny brain “threads” in a chip which is long lasting, usable at home and has the potential to replace cumbersome devices currently used as brain-machine interfaces.