Hey Alexa, what's my PIN? Hackers could use voice assistants to work out what users are typing on their smartphones from the sound of their fingers tapping the screen

  • Researchers built their own version of a smart speaker to access the recordings
  • In commercial products the recordings are stored securely for safety  
  • But study shows that if someone can access these they can decipher passwords 
  • When typing in a five-digit code within 20cm of microphone the researchers correctly guessed a password 76% of the time with just three attempts  

By JOE PINKSTONE FOR MAILONLINE

Published: | Updated:

Smart speakers like Google Home and Amazon Alexa could be used by hackers to listen to and decipher a password or PIN being typed in on a nearby phone.

Researchers from the University of Cambridge built their own version of a smart speaker to closely resemble those which are commercially available. 

Sound recordings from the gadget were inputted into a computer for analysis and experts investigated if the sound and vibrations caused by typing on a smartphone screen could be used to guess a five-digit passcode. 

When the phone was placed within 20cm (7.8inches) of the custom-built device, the computer was able to guess the code with 76 per cent accuracy in three attempts. 

Scroll down for video  

This graphic outlines the general flow of the experiment. A user inputs their passcode and the vibrations and sounds are detected by the speaker and could be analysed. If an attacker gets hold of these, they can tap into the audio and unpick the code

This graphic outlines the general flow of the experiment. A user inputs their passcode and the vibrations and sounds are detected by the speaker and could be analysed. If an attacker gets hold of these, they can tap into the audio and unpick the code 

Pictured, the location and layout of microphones on the Amazon Echo (left) and the layout of the custom-built device built by the Cambridge researchers (right)

Pictured, the location and layout of microphones on the Amazon Echo (left) and the layout of the custom-built device built by the Cambridge researchers (right)

But the accuracy of the code-breaking diminished the further the phone was from the microphone. 

For example, when the phone was positioned 50cm (19.6 inches) from the speaker the accuracy plummeted to just 20 per cent. 

Ilia Shumailov, lead author of the study, said: 'Using two different smartphones and a tablet we demonstrated that the attacker can extract PIN codes and text messages from recordings collected by a voice assistant located up to half a metre away.' 

Smart speakers like the Echo and Home are always listening for a wake word which triggers them and activates their functionality. 

Once this is detected the audio is recorded and goes through to be interpreted by artificial intelligence.

However, access to these recordings are heavily restricted, with only the Alexa account holder able to review them in the app. They can also be deleted by the user at any time. 

Therefore, any criminals wanting to hijack a smart speaker with the goal of stealing device passwords would have to tamper with the device physically or hack into the server to access the recordings. 

The level of brazenness in this endeavour may outweigh the benefits to criminals and the study is therefore more useful as an exercise in acoustic application than cybersecurity.  

Sound recordings were inputted into a computer for analysis and experts investigated if the sound and vibrations caused by pressing buttons could be used to decipher a passcode (stock)

Sound recordings were inputted into a computer for analysis and experts investigated if the sound and vibrations caused by pressing buttons could be used to decipher a passcode (stock)

'Right now, it's unlikely that people would use our attack. However, the world changes quickly and sensors only get better,' Mr Shumailov told New Scientist.  

The findings of the new research, called 'Hey Alexa what did I just type? Decoding smartphone sounds with a voice assistant' are published on pre-print site arXiv. 

Last year, the same researchers found a gaming app can steal a banking PIN by using the phone's own microphone to pick up screen vibrations as a user enters the code. 

In a test, their algorithm could correctly guess 31 out of 50 four-digit login pins in just 10 attempts based on recordings made of the participants as they typed.

These potential attacks would likely begin with the accidental download of malicious software — so users should keep themselves safe by only using trusted apps.

Limiting microphone access to only those apps that need it will also help to make your smartphone more secure. 

HOW TO FIND WHAT YOUR ALEXA HAS RECORDED ABOUT YOU

Open the Alexa app which the devices are synced to or go to this link. 

Select the icon in the bottom right corner - often dubbed the 'hamburger'

Press 'Settings' near the bottom of the menu  

Press 'Alexa Privacy' at the bottom of the menu 

In this section a range of options will appear in a different looking menu - select 'Review Voice History'

Here all the entries of all Alexa-enabled devices attached to the user's account will be listed in reverse order, with the most recent at the top. 

Entries can be sorted by time and by device. 

After filtering to the desired settings, a specific entry can be selected. 

Here, users can listen to ;ayback of the instruction and also have the ability to delete the recording. 

By electing 'All History' and 'All devices', users can also choose to remove all recordings ever stored by the device. 

Advertisement
Google

Most watched News videos

Comments 33

Share what you think

  View all

The comments below have not been moderated.

  View all

The views expressed in the contents above are those of our users and do not necessarily reflect the views of MailOnline.

We are no longer accepting comments on this article.