Zensei: Augmenting Objects with Effortless User

Recognition Capabilities through Bioimpedance Sensing

Munehiko Sato1 , Rohan S. Puri1 , Alex Olwal1, Deepak Chandra2 , Ivan Poupyrev2 , Ramesh Raskar1
1 MIT Media Lab 2 Google ATAP

Cambridge, MA, USA Mountain View, CA, USA

{munehiko, rohan, olwal, raskar}@media.mit.edu {dchandra, ipoupyrev}@google.com

Figure 1. Zensei enables effortless and uninterrupted user identification and personalization with almost any object, such as a smartphone.

ABSTRACT a password, or any other explicit procedure. There are in-

As interactions with smart devices and objects become increas- teresting opportunities now that everyday objects are being
ingly common, a more seamless and effortless identification augmented with computational power and network connectiv-
and personalization technique will be essential to an uninter- ity, and we envision that implicit identification and automated
rupted user experience. In this paper, we present Zensei, a user customization will be of significant importance for human-
identification and customization system using human body computer interaction.
bioimpedance sensing through multiple electrodes embedded
Zensei is a fundamental sensing technology that enables physi-
into everyday objects. Zensei provides for an uninterrupted
cal objects to identify their user instantly by sensing the users
user-device personalization experience that is difficult to forge
touch behavior and bioimpedance. It makes almost any object
because it uses both the unique physiological and behavioral
capable of effortless identification for example, a smartphone
characteristics of the user. We demonstrate our measurement
that recognizes you as soon as you pick up your phone, a car
system in three exemplary device configurations that showcase
that knows who is sitting in the drivers seat so it can change
different levels of constraint via environment-based, whole-
to custom user settings, or a shared tablet that toggles into the
body-based, and handheld-based identification scenarios.
child-safe mode when a child is holding it. With Zensei, one
ACM Classification Keywords does not need to type in a username or password, scan an ID,
H.5.2. Information interfaces and presentation: User Inter- or even align the correct fingertip with the fingerprint scanner.
faces - Graphical user interfaces; Input devices & strategies.
SYSTEM AND IMPLEMENTATION
Author Keywords Bioimpedance and capacitive sensing technologies have been
User identification; bioimpedance sensing; customization used for various applications including tomography [1], multi-
INTRODUCTION
user touch surfaces [3], and wearable devices [2]. Unlike
conventional capacitive sensing, Zensei peers into the body
People interact with more and more smart devices every day.
tissue by measuring the bioimpedance between a pair of elec-
This includes personal electronics, such as mobile phones,
trodes. Therefore, Zensei can know who is touching the object
tablets, laptops, desktop computers, and public displays. In
using the differences in body tissue composition. We advocate
scenarios in which user interactions are frequent, such as with
an approach to capture a users electrical characteristics by
personal devices, it is critical that user recognition procedures
implementing a sensor into the physical objects around us.
do not repeatedly interrupt the interaction. The ideal interac-
Our approach shares some similarity with SFCS [5, 4], how-
tion should not be interrupted by toggling a switch, inputting
ever, we measure both the amplitude and phase components
Secondary affiliations: KTH Royal Institute of Technology (Stock- of electrical frequency response and do so among all combi-
holm, Sweden); Google Inc.(Mountain View, CA, USA) nations of six embedded electrodes (Figure 3) with shielded
cables. Our implementation uses an AD5932 wave generator,
custom analog circuitry, and AD8302 RF gain and phase de-
tector to capture the frequency response over a wide range of
Permission to make digital or hard copies of part or all of this work for personal or frequencies (1 KHz to 1.5 MHz in 150 linear steps) (Figure 4).
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full cita-
tion on the first page. Copyrights for third-party components of this work must be
In our approach, we created three prototypes to demonstrate
honored. For all other uses, contact the owner/author(s). Copyright is held by the au- three different form factors that evaluate and exhibit Zenseis
thor/owner(s). versatility. These include static sensing with relatively stable
UIST15 Adjunct, November 811, 2015, Charlotte, NC, USA.
ACM 978-1-4503-3780-9/15/11. and controlled user touch behavior (hand pad), semi-static
http://dx.doi.org/10.1145/2815585.2817786

41
 G G
G

G
G

G G
G

Rotation 1 Rotation 2 Rotation 3 Rotation 4 Rotation 5 Rotation 6

Figure 2. Three prototypes of Hand Pad, Chair, and Figure 3. Electrode demultiplexing and ground electrode rotation.
Smartphone, and their electrode arrangements.

Waveform
Generator
Output
Driver
Fixed Load DMUX DATA ANALYSIS
To evaluate the overall performance of the system, we per-
Gain/Phase
Detector
Return Signal
Buffer
formed a hold-one-day-out validation by training our classifier
DMUX
on 21 days of data and testing on the remaining day for every
Microcontroller ADC Scaling
Amplifiers
combination of days and averaging the results of all combina-
Commands Data
tions. As shown in Table 1, the more constrained arrangements
Electrodes
(hand pad) tend to outperform those with more user variabil-
Personal Computer
ity. Additionally, the chair showed lower performance likely
Control
Software
Data Storage Visualizer Classifier because of the strong influence of the subjects clothing in
the collected signal. Overall, the high accuracies and low
Figure 4. Zensei system block diagram. FAR prove promising considering the realistic long-term and
variable scenarios in which the data was collected.
sensing with somewhat variable touch behavior due to user
Table 1. Classification Accuracy
posture and changes in clothing (chair), and variable sens- Hand Pad Phone Chair !
ing with highly variable user touch behavior (smartphone) as Classification Accuracy (SD) 96.0% (2.41%) 88.5% (5.51%) 78.6% (7.71%) !
shown in Figures 1 and 2 as well as the Video Figure. FAR (SD) 0.37% (0.20%) 1.04% (0.51%) 2.05% (0.83%)
FRR (SD) 3.97% (2.41%) 11.47% (5.50%) 21.30% (7.88%)
To prepare the data for classification, the thirty vectors of 150
frequency response values were first smoothed using a moving DEMONSTRATION
average filter (n=5). It was empirically determined that good In this highly interactive demonstration, we will set up all three
performance was achieved by feeding just this smoothed data of our arrangements next to each other (hand pad, phone, and
into an SVM classifier with Polynomial Kernel (E = 1.0, C = chair) for training and real-time classification on an attached
1.0). We trained our classifier using SMO implementation in laptop computer as shown in the Video Figure, with a mini-
WEKA Toolkit. mum of three participants trained on any given arrangement.
CONCLUSION
APPLICATION SCENARIOS
We have presented a technique to augment objects to enable au-
Various applications can be realized with Zenseis versatile tomatic personalization through wide-spectrum bioimpedance
user recognition technology. Specifically, it is particularly sensing of the human body. We developed a multi-electrode
useful for casual biometrics. When an individual wants to sensing system and evaluated it with three form-factors with
gain access to a system such as an informational kiosk, they six electrodes each. We then proposed diverse interaction
just need to place a hand on a hand-shaped pad. As Zensei scenarios to highlight the capability of sensing technology
uses multiple embedded electrodes for sensing, users could along with a long-term evaluation with promising results. We
grab a doorknob in a certain way to unlock the door (Video hope Zensei will be a useful tool for designing more seamless
Figure). By doing this, we can create a powerful user-specific customized user interactions with a variety of objects.
key that is a combination of physiological and behavioral
features. Furthermore, users can generate multiple temporary REFERENCES
tokens by changing the way in which the doorknob is grabbed. 1. Bayford, R. H. Bioimpedance tomography (electrical
impedance tomography). Annu. Rev. Biomed. Eng. 8
DATA COLLECTION (2006), 6391.
Data on all three arrangements was collected on 12 subjects
2. Cornelius, C., Peterson, R., Skinner, J., Halter, R., and
over a time period of 30 days excluding weekends (22 days of
Kotz, D. A wearable system that knows who wears it. In
data collection, two sessions each day, five samples per session
Proc. MobiSys 14, 5567.
on each arrangement) to evaluate their classification accuracies
(CA), false acceptance rates (FAR), and false rejection rates 3. Dietz, P., and Leigh, D. DiamondTouch: a multi-user touch
(FRR). Subjects were instructed to touch each arrangement technology. In Proc. UIST 01, 219226.
five times in series per session, removing themselves from 4. Harrison, C., Sato, M., and Poupyrev, I. Capacitive
the arrangement between each sample. The first four days of fingerprinting. In Proc. UIST 12, 537543.
results of the hand pad were not used in the analysis because
5. Sato, M., Poupyrev, I., and Harrison, C. Touche:
its circuit board had malfunctioned and had to be switched.
enhancing touch interaction on humans, screens, liquids,
Although a few sessions were missed, each subject participated
and everyday objects. In Proc. CHI 12, 483492.
in at least one session per day.

42