“Just Like Blooming Fireworks, And Match With Function

Perfectly”: Explore and Evaluate User-Defined One-Handed

Gestures of Smartwatch
Lyumanshan Ye* Jiatong Yue Yiwen Wei
School of design, Shanghai Jiao Tong School of design, Shanghai Jiao Tong School of design, Shanghai Jiao Tong
University, shanghai, China, University, shanghai, China, University, shanghai, China,
Shanghai, China Shanghai, China Shanghai, China
[email protected] [email protected] [email protected]

Shuai Liang Danni Chang

School of Mechanical Engineering, School of design, Shanghai Jiao Tong
Shanghai Jiao Tong University, University, shanghai, China,
shanghai, China, Shanghai, China Shanghai, China
[email protected] [email protected]

ABSTRACT In Extended Abstracts of the 2023 CHI Conference on Human Factors in Com-
One-handed gesture interaction is a more convenient input inter- puting Systems (CHI EA ’23), April 23–28, 2023, Hamburg, Germany. ACM,
New York, NY, USA, 9 pages. https://doi.org/10.1145/3544549.3585914
action method on smartwatches for some special scenarios, e.g.
wearing a smartwatch when running or biking. To explore user-
friendly one-handed gestures, what users are thinking when using 1 INTRODUCTION
the gesture, and what characteristic would make the user feel this With the popularity of mobile Internet and smart devices, wear-
one-handed gesture is friendly, we developed a series of one-handed able products (e.g., smartwatches, smart bands) are becoming more
gestures for 6 basic functions of the smartwatch. The end-user elici- and more mainstream [1]. The locations where these devices are
tation method resulted in 12 new one-hand gestures. We compared mounted can provide instantaneous command input, and their con-
these 12 user-defined one-handed gestures with the Apple Watch tinuous contact with users’ skin enables embedded sensors (e.g.,
one-handed gestures. We developed a Wizard of Oz model and IMU[14], microphone[4, 6] and pulse sensors7, etc.) to record users’
evaluated these gestures by using qualitative and quantitative ap- daily activities and monitor their health. Developers can take advan-
proaches. The results show that we generated a set of one-handed tage of such hardware and combine it with computing algorithms
gestures that are more friendly than the existing Apple Watch one- to correctly recognize various one-handed gestures. At the same
handed gestures. Also, during the evaluation process, we collected time, it is obvious that especially in mobile scenarios, users are
quantitative data and interesting user perspectives. We also gave more willing to use watches instead of mobile phones to complete
some design recommendations for one-handed gestures. some lightweight interactions (e.g. checking the time or reading
social media messages while exercising) [8]. Therefore, interacting
CCS CONCEPTS with the watch in a mobile scenario is a frequent scenario.
• Human-centered computing → Human computer interaction Like mobile phones, touch-screen interaction is the main inter-
(HCI); HCI design and evaluation methods; Heuristic evaluations. action method for smartwatches. However, this interaction method
has limitations in mobile scenarios. Specifically, the small screen
KEYWORDS size makes it difficult for users to tap accurately [15]. At the same
time, some tasks require a series of click actions [9]. The whole pro-
Gesture generation, User-defined gestures, One-handed gestures
cess may take only a few seconds, but for the user to continuously
ACM Reference Format: move their eyes in a mobile scenario to make a series of clicks on
Lyumanshan Ye*, Jiatong Yue, Yiwen Wei, Shuai Liang, and Danni Chang. the watch screen, requires an expensive behavioral cost.
2023. “Just Like Blooming Fireworks, And Match With Function Perfectly”: To explore user-friendly one-handed gestures, we recruited a
Explore and Evaluate User-Defined One-Handed Gestures of Smartwatch. group of end-users to create 168 new one-hand gestures. After our
consolidation and screening, we got 12 user-defined one-handed
gestures for 6 basic functions of smartwatch, and each function
Permission to make digital or hard copies of part or all of this work for personal or has 2 gestures. We recruited another group of users to choose their
classroom use is granted without fee provided that copies are not made or distributed preferred gesture for each 6 functions between 2 gestures. To focus
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for third-party components of this work must be honored. on user behavior and thoughts, we want to eliminate the influence
For all other uses, contact the owner/author(s). of fluctuations in gesture recognition. Therefore, we developed a
CHI EA ’23, April 23–28, 2023, Hamburg, Germany Wizard of Oz model for users to experience one-handed gesture
© 2023 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-9422-2/23/04. interactions. We designed two daily interaction flows to facilitate
https://doi.org/10.1145/3544549.3585914 users understanding of the usage scenarios of one-handed gestures.
CHI EA ’23, April 23–28, 2023, Hamburg, Germany Lyumanshan Ye et al.

Participants learn 12 one-handed gestures and use them to complete In 2021, Apple presented a one-handed gesture interaction based
the interaction flow on the prototype and the Apple Watch. After on Apple Watch, which includes 4 universal functions: previ-
learning process, participants compare and evaluate the preferred ous/next/confirm/open the menu[16]. Users can operate all ap-
user-defined one-handed gestures with existing Apple Watch one- plications via several simple gestures. Such one-handed gesture
handed gestures. We use Semantics, Effortlessness, Learning Costs, set for smartwatch is simpler and friendlier than previous studies.
and Subjective Willingness 4 dimensions to evaluate the friendli- Hence, we refer to these 4 basic functions of Apple Watch, then
ness of a gesture. The user-defined one-handed gestures perform compare our user-defined collection of one-handed gestures with
better than Apple Watch one-handed gestures. We summarized the Apple Watch’s one-handed gestures. Explore whether the user-
the experiment results and users’ Interesting perspectives of these defined gesture set is more preferred by users than Apple’s existing
gestures. And we also propose some design suggestions as provide one-handed gestures.
references for future one-handed gesture development.
3 METHODOLOGY AND EXPERIMENT
DESIGN
2 RELATED WORK The experiment included two stages. In stage 1, we used an elici-
One-handed gesture interaction only requires the user to use one tation study to generate and filtrate gestures. In stage 2, in order
hand to complete related interactive actions[24–26, 29, 23Schäfer,], to help users experience the real feeling of gesture interaction, we
while the other hand can complete other tasks at any time[1]. The use the experimental method of the Wizard of Oz and build a set
one-handed gesture is a type of gesture without touching any tan- of a prototype that can simulate the effect of gesture interaction.
gible interface[7], similar to air gestures[3]. Some mid-air gestures Another group of participants evaluated these gestures by using
require two-handed cooperation[17, 18], while one-handed gestures a prototype, quantitative surveys, and scales. Participants in stage
do not. Hence, one-handed is more simple than two-handed, and 1 (gestures generation) and stage 2 (gestures evaluation) did not
users are more inclined to learn simple gestures[2]. Some on-skin overlap.
one-handed gestures allow the dominant hand to perform related
gestures on the skin of another hand (for example, tap, draw, pinch,
etc.) [5]. But some previous gestures are difficult for users to re- 3.1 Participants
member and distinguish from each other[28]. We publish information and recruit users on three social platforms
The one-handed gesture is useful for not only disabled users but (Weibo, WeChat group, and Xiaohongshu). We pay ¥10 for each
also average users. One-handed gestures allow users with upper participant. We recruited 41 participants. The average age was 23.5
limb disabilities to use electronics without having to touch the years. All participants had previous experience with the Apple
display or watch the crown. For the average user, there is also Watch. Participant occupations included students, engineer, and
a rising demand for interaction methods that can be used easily product manager. In stage 1 of the generative gesture experiment,
and quickly in mobile scenarios. For example, when running, the there were 12 participants, and 7 were female. In stage 2 of the
constant motion prevents runners from accurately tapping and gesture evaluation experiment, there were 29 participants, including
swiping on the small screen of smartwatches. 14 males and 15 females.
There have been many previous studies proposing various tech-
niques for recognizing one-handed gestures[11, 12, 22, 32–35], but
few studies have focused on the user perspective of one-handed
3.2 Stage 1: Developing the User-Defined
gestures. For eample, what one-handed gestures make users feel One-hand Gesture Set
more user-friendly, and make users more willing to interact with We want to collect users’ preferences and opinions of one-handed
the watch theough one-handed gestures. How users understand and gestures for basic features in the watch. Therefore, in stage 1, we
remember a one-handed gesture. In order to explore user-friendly asked 12 participants to design gestures for the seven most basic
gestures, several previous studies have used user-defined[10] ap- functions: confirm, previous Button, next Button, open Dock, return
proaches to develop gestures [19, 20, 30, 31], which is a interesting to Home page, return to previous interface, and open Control Center.
way to develop gestures. Shimon et al. used user-defined approach Each participant designed two different gestures for these functions.
to explore non-touchscreen gestures for smartwatches[21]. This We got 168 gestures, and each function heve 24 kinds of gestures.
study proposed 18 gestures. Some gesture can only be used in a Stage 1 Procedure: 1. Introduction: We describe the back-
specific software, for example, turning on the microphone in video ground and goals of the one-hand gesture design. 2. Designed
conferencing software. But remembering so many set of match- gestures: Participants designed the gesture they thought were nat-
ing relationships between the gesture and the relevant function ural and readily comprehensible for each function without a time
of certain application is a big challenge for users. Fong-Gong Wu limit. 3. Self Evaluation: Participants used rated the gestures they
proposed a smaller set of one-handed gestures only for browsing just designed by three 5-point Likert-scale questions. The Likert
on touch phone[27]. To match the human short-term memories, scale includes three questions: The gesture I designed matches the
they found 5 gestures refer to 7±2 items refer to Miller’s law[37]. expected function very well. I think this gesture is easy to perform.
Therefore, selecting universal functions that can be used in all ap- I think it’s a little complicated to use this gesture to perform this
plications can reduce memory costs of memory cost of gesture function. 4. Selecting Gestures: We use the scores of Guessability
sets. and Agreement to filter gestures. These two scores refer to the
“Just Like Blooming Fireworks, And Match With Function Perfectly”: Explore and Evaluate User-Defined One-Handed
Gestures of Smartwatch CHI EA ’23, April 23–28, 2023, Hamburg, Germany

Figure 1: Experimental procedures

We implemented a Processing4 application1 to control an ESP32-

DevKit-Board and a 1.3-inch 240X240 HD LCD display as a watch
screen. When a participant performs a gesture, the experimenter
will control the appropriate interface on the watch. We finally
selected 12 gestures for 6 functions.
Scenario design: In stage 2, we recruited 29 participants. Par-
ticipants learn 12 gestures generated in stage 1, then use these
gestures to complete two tasks. Participants compare the user-
defined gestures set with the Apple Watch gestures set, select their
favorite gesture and explain their reasoning. Then complete eval-
uation scales. To help participants imagine the scenario of using
Figure 2: 1.3-inch LCD screen and Interface in Processing. one-handed gestures, we designed two one-handed gesture task
flows in two scenarios. The reason for setting these two scenarios
is that wearing a watch for exercise is a common scenario for using
Wobbrock et al. [2, 13, 36]. The two scores are defined as follows: a watch, and one-handed gestures can be more convenient than
𝑃𝑠 tapping the watch screen during exercise (e.g., doing anaerobic
𝐺= .100% (1) exercise).
𝑃
 2 Stage 2 Procedure: 1. Participants watched a tutorial video on
Í |𝑃𝑠 | one-handed user-defined gestures. 2. Participants imitated each
𝑃𝑠 ⊆𝑃 |𝑃 |
𝐴= .100% (2) gesture at least three times and used the prototype to complete
1 the two task flows. Each user-defined gesture will be used in the
In equation 1 and equation 2, 𝑃 is the amount of all user- task flows. 3. Participants selected the most preferred user-defined
generated gestures for a same function, and 𝑃𝑠 is the amount of gesture and explained why. 4. Participants chose a task flow and
similar user-generated gestures. Based on prior work, guessability completed it with an Apple Watch. 5. Participants use scales to
means the quality of symbols which allows a user to access intended rate the user-defined gesture they preferred and the Apple Watch.
referents via those symbols despite a lack of knowledge of those The scales have 4 questions and ranged from 1 to 7, from "strongly
symbols. Hence, the higher G means the more participants have disagree" to "strongly agree", 4 is neutral. The questions were as
the same understanding for its semantics. The A means the the follows, containing 3 forward and 1 negative question: 1. I think
agreement among symbols the semantics of this gesture match with the function well. 2. This
proposed by the participants. The A will be high when the different gesture was easy for me to perform. 3(negative question). I think
kinds of gesture for a function are unique, and it will be when they the learning cost of this gesture is very high. 4. I am willing to
are unique. The function with low A maybe too abstract to indicate use this gesture to manipulate the interface. These four questions
its sematics only by gesture. assess semantics, effortlessness, learning costs, and subjective will-
Selection Result: We calculated the Agreement of all functions. ingness. Cronbach’s 𝛼 coefficient of the scales is between 0.7 and
The scores is: Confirmation(0.1875)> Previous Button(0.1597)> 0.9, showing good internal consistency. Scales have good content
Next Button(0.1458) Opening Dock(0.1181)>Return to Home- reliability.
page(0.1024)> Return to the Previous interface(0.0702)>Open the
Control Center(0.0556). We remove the functions with the lowest 4 RESULT
scores. For each function, we combine similar gestures and calcu-
lated the Guessability. We selected the top two gestures with the 4.1 Data analysis
highest Guessability. The scores of these four questions in all gestures were normally
distributed, after the Shapiro-Wilke test. We calculated the aver-
3.3 Stage 2: Evaluating The User-Defined age score of each question, then used matched T-test to analyze
One-Handed Gestures Set whether statistically significant difference between user-defined
gestures and Apple Watch gestures on four dimensions (semantics,
Apparatus: In developing a user-defined gesture set, we want to
eliminate the influence of fluctuations in gesture recognition on 1 Allnecessary codes can be found at https://anonymous.4open.science/r/1035-
users’ behavior. Hence, we employed a Wizard of Oz approach. supplementary-files-E5B3/1035-supplementary%20files.zip
CHI EA ’23, April 23–28, 2023, Hamburg, Germany Lyumanshan Ye et al.

Figure 3: Task flow1 in Scene1- Open the Sports & Health app with one hand, then return to the Music app.

Figure 4: Task flow2 in Scene2- Open the Timer app with one hand, then return to the Music app.

|
Gesture Preferred Semantics Effortlessness High Subjective
number Learning Willing-
Costs ness
Average T-text Average T-text Average T-text Average T-text
User- 15 P 5.89 0.01<P<0.05 5.53 P>0.05 2.41 P>0.05 4.53 P>0.05
defined
Gesture 1
Apple —— 4.29 4.94 2.94 4.35
Watch
Gesture

Figure 5: Comparison data between user-defined gestures and Apple Watch gestures in the “Previous Button” function

effortlessness, learning costs, and willingness). In questions 1, 2, does one pinch represent the next button and two pinches rep-
and 4, a higher mean score means a friendlier gesture. In question resent the previous button? I will easily confuse the gestures of
3, a lower mean score means a friendlier gesture. these two functions with each other. I can’t find a good reason
to explain the difference in the number of repetitions. So I had
to force myself to remember the correspondence between the
4.2 "Previous Button" Function number of times and function"
Participants mentioned many advantages of the semantics of
Among the two user-defined gestures, 15 participants preferred ges-
user-defined gesture 1, "My palm moves upward with the watch
ture 1, more than the 14 participants who chose gesture 2. Obviously,
screen button move in the same direction, this interaction ef-
in the "Previous Button" function, the average score of user-defined
fect with a good sense of manipulation let me feel reasonable",
gesture 1 performs better than the Apple Watch gestures on all four
and"The action is really like swiping a short video in TikTok".
dimensions. There is a significant difference between the scores
Regarding the effortlessness, participants generally felt that “Al-
of user-defined gesture 1 and Apple Watch gesture on semantics,
though the finger pinch seems to be a slight action, in fact,
while there is no significant difference in the scores of the other
this movement is not as easy to do as the palm-up movement” .
three dimensions.
Some participants criticize pinching, saying "Repeating this ac-
In the qualitative interviews, almost all participants mentioned
tion makes me feel a muscle cramp " and "Pinching twice is
that the Apple Watch gesture has confusing semantics, "For me,
really tiring". For palm-upward, the participants said "It is easier
it’s hard to associate this gesture with any function", and"Why
“Just Like Blooming Fireworks, And Match With Function Perfectly”: Explore and Evaluate User-Defined One-Handed
Gestures of Smartwatch CHI EA ’23, April 23–28, 2023, Hamburg, Germany

Gesture Preferred Semantics Effortlessness High Learning Subjective

number Costs Willingness
Average T-text Average T-text Average T-text Average T-text
User-defined Gesture 1 16 P 5.82 P<0.01 5.35 P>0.05 2.65 P>0.05 4.82 P>0.05
Apple Watch Gesture —— 4.65 5.88 2.47 5

Figure 6: Comparison data between user-defined gestures and Apple Watch gestures in the “Next Button” function

Gesture Preferred Semantics Effortlessness High Learning Subjective

number Costs Willingness
Average T-text Average T-text Average T-text Average T-text
User-defined Gesture 1 17 P 5.53 P>0.05 6.47 P>0.05 2.17 P>0.05 5.18 P>0.05
Apple Watch Gesture —— 5.71 6.18 1.76 5.24

Figure 7: Comparison data between user-defined gestures and Apple Watch gestures in the “Confirmation” function

than pinching" and "This gesture is very sprawling and easy 4.4 "Confirmation" Function
to understand meaning" There are 17 participants who preferred user-defined gesture 1.
Participants admitted that pinching of user-defined gesture 1 was
4.3 "Next Button" Function less laborious than the fist of the Apple Watch gesture. However,
More participants preferred user-defined gesture 1 than user- the average scores of user-defined gesture 1 show that it performs
defined gesture 2. User-defined gesture 1 of the “Next Button” func- slightly worse than the Apple Watch gesture in terms of semantics,
tion has a much higher average of semantics than the Apple Watch learning cost, and subjective willingness. None of these scores were
gesture, and has a significant difference. It also got higher average significantly different. Obviously, making a fist is more effortful,
scores of subjective effortlessness and willingness, and has a slightly because it requires people to move the entire palm and all fingers.
lower average score of learning cost. In these three metrics, there While pinch only needs two most commonly used fingers, that is
are no significant differences between the two gestures. Consider- the index finger and thumb. However, participants said that the fist
ing that gestures of the “Next Button” function and the “Previous gesture was "relaxed", "easy to do" and "can be done quickly", and
Button” function are a set of gestures, here omitted some qualitative the pinch was "Easy to use, but still more difficult than making
comments similar to the “Previous Button” function. Interestingly, a fist ". We analyze that making a fist is used frequently enough in
in the comparison between user-generated 1 and user-generated daily life. So that participants are already extremely familiar with
2, participants mentioned that "Wrist rotation downward is too it, and such familiarity brings a psychological feeling of ease that
easy, which has no sense of control ", and "Though turning outweighs the actual physical exertion.
palm up and down is a bigger set of gesture, more comfortable Participants state many semantic advantages of user-defined
and easier to understand than wrist rotation". These reasons gesture 1: "Pinch looks like OK gesture", "It brings the feeling
are similar to the next "confirmation" function. Though participants of grabbing something", "It is like snapping, and also like a
commented that the user-defined gesture 1 is more effortless than ‘Bingo’". While participants who preferred the Apple Watch ges-
the Apple Watch gesture, they still preferred the latter, a gesture ture mentioned that "Making a fist has a determined feeling
with greater action. of confirmation function" and "The fist-clenching is stronger
CHI EA ’23, April 23–28, 2023, Hamburg, Germany Lyumanshan Ye et al.

Gesture Preferred Semantics Effortlessness High Learning Subjective

number Costs Willingness
Average Average Average Average
User-defined Gesture 1 23 P 6.35 6.59 1.53 6.23

Figure 8: Comparison data between user-defined gestures and Apple Watch gestures in the “Opening Dock” function

Gesture Preferred Semantics Effortlessness High Learning Subjective

number Costs Willingness
Average T-text Average T-text Average T-text Average T-text
User-defined Gesture 1 19 P 5.24 P<0.01 6.06 P<0.01 2.18 P<0.01 5.71 P<0.01
Apple Watch Gesture —— 3.24 3.18 5.12 2.71

Figure 9: Comparison data between user-defined gestures and Apple Watch gestures in the "Return to Homepage" Function

and has more certainty ". Both two gestures have good semantics, perfectly and is easy to remember ", "It feels like all the ap-
but participants felt that the semantics of the Apple Watch gesture plications are unfolding and displayed", "This gesture is very
was stronger than the user-defined gesture 1. Our Analysis is that similar to the touchpad gesture to open the launch pad when
the "Confirmation" function is so frequently used and important, I use Mac".
so that the corresponding gesture needs a strong sense. Another
reason is, in a daily interface interaction flow, "Confirmation" usu- 4.6 "Return to Homepage" Function
ally means the end, staying in the current interface and without A total of 19 participants preferred user-defined gesture 1, and 10
continuing interaction, this end meaning also requires a strong participants chose user-defined gesture 2. As the previous function,
gesture. there is no one gesture for the "Return to Homepage" function. Users
need to use a combination of gestures to complete this function,
the interaction process is 1. Open the control menu by making a
4.5 "Opening Dock" Function fist twice, then the “Return to Homepage” button has been selected;
Apple Watch does not define a specific gesture for the "Opening 2. Then make a fist once which is the “Confirmation” gesture to
Dock" function. And there is also no combination of gestures that click the “Return to Homepage” button.
can achieve this function, so this user-defined gesture 1 can not be Comparing the average scores of user-defined gesture 1 and the
compared with the Apple Watch gesture. This "Opening Dock" user- Apple Watch combination of gestures, gesture 1 performs better in
defined gesture 1 got the best scores among all the user-defined ges- all four dimensions, and there are significant differences between
tures. And the difference between the number of people choosing all these four sets of scores. Participants said, "When I use a smart-
user-defined gestures 1 and user-defined gestures 2 is also the most watch, I frequently need to back to the homepage to click other
dramatic. Many participants expressed their love for the gesture, apps. It’s too troublesome to use this combination of gestures
offering a variety of interesting interpretations and associations. every time". Participants mentioned many semantic advantages of
"It makes me imagine the blooming fireworks, fireworks user-defined gesture 1 " Slowly close all fingers, just as closing
also have a semantic meaning of opening", "There is no any all applications and returning to the home page", "This gesture
separation between the gesture and the operation, it matches and the last Opening Dock gesture 1 looks like in the same
“Just Like Blooming Fireworks, And Match With Function Perfectly”: Explore and Evaluate User-Defined One-Handed
Gestures of Smartwatch CHI EA ’23, April 23–28, 2023, Hamburg, Germany

Gesture Preferred Semantics Effortlessness High Learning Subjective

number Costs Willingness
Average T-text Average T-text Average T-text Average T-text
User-defined Gesture 1 15 P 4.88 P<0.01 5.18 P<0.01 2.44 P<0.01 4.56 P>0.05
Apple Watch Gesture —— 1.56 1.94 6.38 1.31

Figure 10: Comparison data between user-defined gestures and Apple Watch gestures in the "Back to the previous page" Function

system" and "This gesture is simple. And it is very effortless, the "confirm" function explained that, "because it looks like ok” . in
well suited for such a function which be often used". stage 2, many participants said, “I think pinching looks like OK
gesture and means confirmation” . We found that participants
4.7 "Back to the previous page" Function can spontaneously and actively understand the semantics of the
There is no specific gesture of this function in the Apple Watch. Can gesture to help them remember the mapping between gesture and
be replaced with a combination of gestures, the interaction flow function. Gestures with strong semantics were easier to remember
is: Keep using the “Previous” button gesture until the "Back to the and less likely to be confused between functions. There was a high
previous page" button in the upper left corner of the interface has degree of commonality in people’s perception of semantics.
already been selected, then use the “Confirmation” gesture to click Less effort is not always better: The level of comfort is influ-
the button. 14 participants chose user-defined gesture 1, which got enced by the user’s experience and proficiency in using gestures.
better scores than the Apple Watch gesture in all four dimensions, Good semantics help users remember and increase their willingness
with a significant difference. to use gestures. For frequently used functions, users prefer gestures
Among all user-defined gestures, it has the largest average score with well-matched semantics and function, over effort-saving ges-
beyond Apple Watch. Many participants agreed that this gesture is tures. And good semantics can cover up the lack of effort-saving.
reasonable, as they said "Thumb outward gives a sense of getting A typical example is the previous button function and the next
out of the current interface", and "The direction of the thumb button function. Apple proposed an extremely effortless solution of
is the same as the direction of returning to the previous page a simple finger pinch, but a large number of participants complained
on the phone. Easy to remember". that the gesture was semantically weak and difficult to remember.
Interestingly, user-defined gesture 2 of the "Back to the previ- Another similar example is the confirmation function, where par-
ous page" (palm flip-down twice) and Apple Watch gesture of the ticipants felt that squeezing the finger saved effort, but clenching
"previous button" (pinch fingers twice) all need to repeat a gesture the fist was more consistent with the semantics of confirmation.
twice. However, about user-defined gesture 2 of the "Back to the Shortcuts could be considered instead of a basic gestures
previous page", repeating a gesture twice was not as criticized as combination interaction flow: In addition to the most basic ges-
pinching fingers twice. One participant offered this understanding, tures, users also need some gestures that correspond to the short-
"Returning to the previous page means switching the entire cuts function. This is because it is cumbersome for users to use
interface. The previous button means selecting the button in several combinations of gestures to perform a commonly used func-
the current interface. I think two repeated actions represent tion. Some participants mentioned, "For many scenarios, simple
the interaction with a big change. So it is reasonable” . shortcuts can be useful, such as lock screen".
Avoid designing gestures that required repeated multi-
ple times: We found that compared to touch screen interactions,
5 DESIGN RECOMMENDATION FOR
one-handed gestures are a challenge for the user’s memory. Be-
ONE-HANDED GESTURES IN cause there are no visual cues to suggest how to operate. Of all
SMARTWATCH the gestures evaluated in stage 2, users attach importance to the
Based on qualitative and quantitative results, we present some semantics of gestures because the good semantic means "easy to
design recommendations for the one-handed gesture. remember". However, "times" had no semantics and were therefore
Users prefer gestures that are simple and easy to associate, difficult to remember. In the "previous button" function, partici-
and user-defined gestures may have better semantics: There pants commented that a gesture with more than one action would
was some overlap between the inspiration from participants of stage be very problematic, "If I want to go forward to the fourth song,
1 and the understanding of the stage 2 participants. For example, in then I need to do eight pinches and that’s too tiring". A gesture
stage 1, several participants who designed a pinching gesture for
CHI EA ’23, April 23–28, 2023, Hamburg, Germany Lyumanshan Ye et al.

that needs more than one action may result in too many repetitive 13, 1 (Jan 2006), 52–55. https://doi.org/10.1109/LSP.2005.860538
[5] Kim J , He J , Lyons K , et al. The Gesture Watch: A Wireless Contact-free
actions in an interaction flow. Gesture based Wrist Interface[C]// IEEE International Symposium on Wearable
Gestures for “Continuous function”: Some participants sug- Computers. IEEE, 2007.
gested, "I am tired of doing multiple gestures (such as selecting [6] Gupta S , Morris D , Patel S , et al. SoundWave: using the doppler effect to sense
gestures. ACM, 2012.
the last song in the list), I think it would be good to add a [7] Loclair, C., Gustafson, S., & Baudisch, P. (2010, September). PinchWatch: a wear-
gesture for continuous “next button” function". able device for one-handed microinteractions. In Proc. MobileHCI (Vol. 10).
[8] Caroline C. 2017. Here’s How People Are Using Their Smartwatches. https:
Transfer of experience: Consider transferring people’s experi- //www.businessinsider.com/mostused-smartwatch-features-chart-2017-8
ence of using flat gestures on a phone or computer to one-handed [9] Arif, A. S., & Mazalek, A. (2016, July). A survey of text entry techniques for
gestures in the air. For example, participants mentioned "swiping smartwatches. In International Conference on Human-Computer Interaction (pp.
255-267). Springer, Cham.
the index finger outward, similar to going back to the previ- [10] Nielsen M , M Störring, Moeslund T B , et al. A Procedure for Developing Intuitive
ous page on the phone" and "opening the hand, similar to the and Ergonomic Gesture Interfaces for HCI[C]// International Gesture Workshop.
gesture of opening an application with the MAC trackpad". Springer Berlin Heidelberg, 2003.
[11] Jiang, S., Lv, B., Guo, W., Zhang, C., Wang, H., Sheng, X., & Shull, P. B. (2017).
Consider gender differences: Men and women may give dif- Feasibility of wrist-worn, real-time hand, and surface gesture recognition via
ferent ratings about the level of effort cost of a gesture. Almost all sEMG and IMU sensing. IEEE Transactions on Industrial Informatics, 14(8), 3376-
3385.
male users think making a fist is a very effortless gesture, while [12] Lu, Z., Chen, X., Li, Q., Zhang, X., & Zhou, P. (2014). A hand gesture recognition
some female users feel it is effortful. framework and wearable gesture-based interaction prototype for mobile devices.
IEEE transactions on human-machine systems, 44(2), 293-299.
[13] Wobbrock J O, Aung H H, Rothrock B, et al. Maximizing the guessability of
6 LIMITATION symbolic input[C]//CHI’05 extended abstracts on Human Factors in Computing
In this paper, all participants were young, under the age of 30. Systems. 2005: 1869-1872.
[14] Kundu, A. S., Mazumder, O., Lenka, P. K., & Bhaumik, S. (2018). Hand gesture
There may be differences in the understanding and preference of recognition based omnidirectional wheelchair control using IMU and EMG sen-
gestures among people of different ages. Therefore, in future work, sors. Journal of Intelligent & Robotic Systems, 91(3), 529-541.
we will consider participants with different ages e.g. participants [15] Ren, Y., & Arif, A. S. (2021). Stepper, Swipe, Tilt, Force: Comparative Evaluation
of Four Number Pickers for Smartwatches. Proceedings of the ACM on Human-
aged 10-20, 30-50, and above 50. Computer Interaction, 5(ISS), 1-21.
The selection of appropriate and user-friendly shortcut functions [16] Use auxiliary touch on the Apple Watch (Apple Watch SE and Apple Watch
Series 4 and later models only) https://support.apple.com/zh-cn/guide/watch/
is an issue worth exploring. The number of shortcut functions is apdec70bfd2d/watchos
often larger than the basic gestures, which can increase the user’s [17] How to set gesture control for Huawei mobile phone https://consumer.huawei.
memory burden. So developers should choose features that occur com/cn/support/content/zh-cn15801113/
[18] How does tmall Genie open gesture operation https://bot.tmall.com
frequently and have relatively clear semantics. A negative example [19] Piumsomboon, Thammathip, et al. "User-defined gestures for augmented reality."
is the open Control Center function that we removed in stage 1. IFIP Conference on Human-Computer Interaction. Springer, Berlin, Heidelberg,
The meaning of this function is abstract, so people’s understanding 2013.
[20] Ruiz, Jaime, Yang Li, and Edward Lank. "User-defined motion gestures for mo-
of the semantics is diverse. It is difficult to come up with a gesture bile interaction." Proceedings of the SIGCHI conference on human factors in
that is universally understood by most users. computing systems. 2011.
[21] Shimon, S.S.A. et al. 2016. Exploring Non-touchscreen Gestures for Smartwatches.
In Proceedings of the 2016 CHI Conference on Human Factors in Computing
7 CONCLUSION Systems (CHI ’16).
[22] Lu, Z., Chen, X., Li, Q., Zhang, X., & Zhou, P. (2014). A hand gesture recognition
This paper presents a complete study and a complete analysis of framework and wearable gesture-based interaction prototype for mobile devices.
one-handed gestures from generation to evaluation, combining user IEEE transactions on human-machine systems, 44(2), 293-299.
participatory design, qualitative methods, and quantitative meth- [23Schäfer,] A., Reis, G., & Stricker, D. (2022). AnyGesture: Arbitrary One-Handed
Gestures for Augmented, Virtual, and Mixed Reality Applications. Applied Sci-
ods. The main contributions of this paper are: 1. Demonstrating a ences, 12(4), 1888.
new set of simpler and more user-friendly one-handed gestures. 2. [24] Gong, J., Yang, X. D., & Irani, P. (2016, October). Wristwhirl: One-handed contin-
Propose an experimental procedure, experimental prototype and uous smartwatch input using wrist gestures. In Proceedings of the 29th Annual
Symposium on User Interface Software and Technology (pp. 861-872).
scales to develop more user-friendly one-handed gestures. This pro- [25] Delamare, W., Silpasuwanchai, C., Sarcar, S., Shiraki, T., & Ren, X. (2019, Novem-
cess tests the semantics of gestures twice, increasing the possibility ber). On Gesture Combination: An Exploration of a Solution to Augment Gesture
Interaction. In Proceedings of the 2019 ACM International Conference on Interactive
that the semantics of gestures will be understood by most users. 3. Surfaces and Spaces (pp. 135-146).
Gathering interesting user qualitative insights. 4. Extracting some [26] Cannanure, V., Xiang’Anthony’Chen, & Mankoff, J. (2016, June). Twist’n’Knock:
design suggestions for future gesture design. A One-handed Gesture for Smart Watches. In Graphics Interface (pp. 189-193).
[27] Wu, F. G. , & Kuo, J. Y. . (2013). One-Handed Gesture Design for Browsing on
Touch Phone. Proceedings of the 7th international conference on Universal
REFERENCES Access in Human-Computer Interaction: applications and services for quality of
[1] Emily A. Vogels. 2020. About One-in-Five Americans Use a SmartWatch or Fitness life - Volume Part III. Springer Berlin Heidelberg.
Tracker. https://www.pewresearch.org/fact-tank/2020/01/09/about-one-in-five- [28] Alexander, S. , & Gerd, R. , & Didier, S. .(2022). AnyGesture: Arbitrary One-Handed
americans-use-a-smart-watch-or-fitness-tracker/ Gestures for Augmented, Virtual, and Mixed Reality Applications. Applied Sci-
[2] Morris M R , Wobbrock J O , Wilson A D . Understanding users’ preferences for ences, 12(4)
Surface gestures[C]// Proceedings of the Graphics Interface 2010 Conference, May [29] Thewsuwan, S. , & Chiracharit, W. . (2015). One-handed gesture based inter-
31 - June 02, 2010, Ottawa, Ontario, Canada. Canadian Information Processing action for image zoom manipulation. International Conference on Electrical
Society, 2010. Engineering/electronics. IEEE.
[3] Morris, M. R., Danielescu, A., Drucker, S., Fisher, D., Lee, B., Schraefel, M. C., & [30] Wu, H., & Yang, L. (2020). User-defined gestures for dual-screen mobile interac-
Wobbrock, J. O. (2014). Reducing legacy bias in gesture elicitation studies. inter- tion. International Journal of Human–Computer Interaction, 36(10), 978-992.
actions, 21(3), 40-45. [31] Chan, E., Seyed, T., Stuerzlinger, W., Yang, X. D., & Maurer, F. (2016, May). User
[4] K. S. R. Murty and B. Yegnanarayana. 2006. Combining evidence from residual elicitation on single-hand microgestures. In Proceedings of the 2016 CHI Confer-
phase and MFCC features for speaker recognition. IEEE Signal Processing Letters ence on Human Factors in Computing Systems (pp. 3403-3414).
“Just Like Blooming Fireworks, And Match With Function Perfectly”: Explore and Evaluate User-Defined One-Handed
Gestures of Smartwatch CHI EA ’23, April 23–28, 2023, Hamburg, Germany

[32] Babic, T., Reiterer, H., & Haller, M. (2017, October). GestureDrawer: one-handed Proceedings of the 31st Australian Conference on Human-Computer-Interaction
interaction technique for spatial user-defined imaginary interfaces. In Proceed- (pp. 585-589).
ings of the 5th Symposium on Spatial User Interaction (pp. 128-137). [35] Cannanure, V., Xiang’Anthony’Chen, & Mankoff, J. (2016, June). Twist’n’Knock:
[33] Borah, P. P., & Sorathia, K. (2019, March). Natural and intuitive deformation A One-handed Gesture for Smart Watches. In Graphics Interface (pp. 189-193).
gestures for one-handed landscape mode interaction. In Proceedings of the [36] Wobbrock, J. O., Morris, M. R., & Wilson, A. D. (2009, April). User-defined gestures
Thirteenth International Conference on Tangible, Embedded, and Embodied for surface computing. In Proceedings of the SIGCHI conference on human factors
Interaction (pp. 229-236). in computing systems (pp. 1083-1092).
[34] Sei, Y., Funakoshi, M., & Shizuki, B. (2019, December). Expanding One-Handed [37] Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits
Touch Input Vocabulary Using Index Finger on and Above Back-of-Device. In on our capacity for processing information. Psychological review, 63(2), 81.