POSTER: Fast, Automatic iPhone Shoulder Surfing

Touchscreen devices increase the risk of shoulder surfing to such an extent that attackers could steal sensitive information by simply following the victim and observe his or her portable device. We underline this concern by proposing an automatic shoulder surfing attack against modern touchscreen keyboards that display magnified keys in predictable positions. We demonstrate this attack against the Apple iPhone—although it can work with other layouts and different devices—and show that it recognizes up to 97.07% (91.03% on average) of the keystrokes, with only 1.15% of errors, at 37 to 51 keystrokes per minute: About eight times faster than a human analyzing a recorded video. Our attack accurately recovers the sequence of keystrokes input by the user. A previous attack, which targeted desktop scenarios and thus worked with very restrictive settings, is similar in spirit to ours. However, as it assumes that camera and target keyboard are both in fixed, perpendicular position, it cannot suite mobile settings, characterized by moving target and skewed, rotated viewpoints. Our attack, instead, requires no particular settings and even allows for natural movements of both target device and shoulder surfer's camera. In addition, our attack yields accurate output without any grammar or syntax checks, so that it can detect large context-free text or non-dictionary words.