A Home-Based Functional Electrical Stimulation System For Upper-Limb Stroke Rehabilitation PDF

UNIVERSITY OF SOUTHAMPTON
A Home-Based Functional Electrical

Stimulation System for Upper-Limb
Stroke Rehabilitation
by
Mustafa C. Kutlu
A thesis submitted in fulfillment for

the degree of Doctor of Philosophy
in the
Faculty of Physical Sciences and Engineering
Electronics and Computer Science
June 2017
UNIVERSITY OF SOUTHAMPTON
ABSTRACT
FACULTY OF PHYSICAL SCIENCES AND ENGINEERING
Electronics and Computer Science
Doctor of Philosophy
A HOME-BASED FUNCTIONAL ELECTRICAL STIMULATION SYSTEM FOR
UPPER-LIMB STROKE REHABILITATION
by Mustafa C. Kutlu
Due to an increased population of stroke patients and subsequent demand on health providers,
there is an urgent need for effective stroke rehabilitation technology that can be used in pa-
tients’ own homes. Over recent years, systems employing functional electrical stimulation
(FES) have shown the ability to provide effective therapy. However, there is currently no
low-cost therapeutic system available which simultaneously supplies FES to muscles in the
patient’s shoulder, arm and wrist to provide co-ordinated functional movement. This re-
stricts the effectiveness of treatment, and hence the ability to support activities of daily
living.
In this thesis a home-based low cost rehabilitation system is developed which substantially
extends the current state of art in terms of sensing and control methodologies. In partic-
ular, it embeds novel non-contact sensing approaches; the first use of an electrode array
within a closed-loop model based control scheme; an interactive task display system; and
an integrated learning-based controller for multiple muscles within the upper-limb (UL),
which supports co-ordinated tasks. The thesis then focuses on compacting the prototype
by upgrading the depth sensor and using embedded systems to transfer it to the home
environment.
Currently available home-based systems employing FES for UL rehabilitation are first re-
viewed in terms of their underlying technology, operation, scope and clinical evidence. Mo-
tivated by this, a detailed examination of a prototype system is carried out that combines
low cost non-contact sensors with closed-loop FES controllers. Then potential avenues to
extend the technology are highlighted, with specific focus given to low-cost non-contact
based sensors for the hand and wrist. Sensing approaches are then reviewed and evaluated
in terms of their scope to support the intended system requirements. Electrode array hard-
ware is developed in order to provide accurate movement capability. Biomechanical models
of the combined stimulated arm and mechanical support are then formulated. Using these,
model-based iterative learning control methodologies are then designed and implemented.
The system is evaluated with both unimpaired participants and stroke patients undergoing
a course of treatment. Finally, a home-based prototype is developed which integrates and
extends the aforementioned components. Results confirm the system’s scope to provide
more effective stroke rehabilitation. Based on the achieved results, courses of future work
necessary to continue this development are outlined.
iv
Contents
List of Figures ix
List of Tables xiii
Declaration of Authorship xv
Acknowledgements xvii
Nomenclature xix
1 Introduction 1
1.1 Justification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Research Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Upper Limb Stroke Rehabilitation 11

2.1 Conventional and Robotic Therapy . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.1 Conventional Therapy . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.2 Robotic Therapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Rehabilitation with Electrical Stimulation . . . . . . . . . . . . . . . . . . . . 15
2.2.1 FES Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3 Home-Based FES Technology 21

3.1 BION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.1.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.1.2 Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.1.3 Advantages and Disadvantages . . . . . . . . . . . . . . . . . . . . . . 23
3.1.4 Clinical and Non-Clinical Evaluation . . . . . . . . . . . . . . . . . . . 23
3.2 Bioness Handmaster H200 Wireless . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 Microstim 2v2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.3.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.4 Neuromove NM900 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.4.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
v
vi CONTENTS

3.5 Pulsecure-Pro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.5.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.6 Saebo MyoTrac Infiniti . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.6.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.7 STIWELL Med 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.7.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.8 ShefStim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.8.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.9 Comparison and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4 Goal-Oriented Stroke Rehabilitation Utilising Electrical Stimulation 37

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.2 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.2.1 Task Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2.3 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.4 Control System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.3.1 Intervention Safety Management . . . . . . . . . . . . . . . . . . . . . 48
4.3.2 Intervention Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.3.3 Intervention Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5 Sensor Technologies for the Hand and Wrist 59

5.1 Sensing Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.1.1 Contact Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.1.2 Non-Contact Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.1.2.1 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.1.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.2 Evaluation of Leap Motion and PrimeSense . . . . . . . . . . . . . . . . . . . 71
5.2.1 Sensor Requirement Specification . . . . . . . . . . . . . . . . . . . . . 71
5.2.2 Human Hand Model and Joint Angle Computation . . . . . . . . . . . 72
5.2.3 Experiment Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.2.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
CONTENTS vii
6 GO-SAIL with Improved Hand Functionality 79

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.2 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6.2.1 Task Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6.2.2 System Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6.2.3 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.2.4 Control System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.2.4.1 Biomechanical Model . . . . . . . . . . . . . . . . . . . . . . 86
6.2.4.2 Control Scheme . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.3.2 Unimpaired Participant Results . . . . . . . . . . . . . . . . . . . . . . 92
6.3.3 Stroke Participant Results . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.3.3.1 FES-Unassisted Results . . . . . . . . . . . . . . . . . . . . . 95
6.3.3.2 FES-Assisted Results . . . . . . . . . . . . . . . . . . . . . . 95
6.3.3.3 Clinical Assessments . . . . . . . . . . . . . . . . . . . . . . . 96
6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
7 GO-SAIL Compact System Implementation 105

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.2 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
7.2.1 Task Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.2.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.2.3 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
7.2.4 Control System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
7.2.4.1 Biomechanical Model . . . . . . . . . . . . . . . . . . . . . . 113
7.2.4.2 Control Scheme . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
8 Conclusions and Future Research 121

8.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
A Identification Protocols 127
B Graphical User Interface 131
C Non-Contact Sensors for Hand and Wrist Rehabilitation 135
D Previous Leap Motion Experiment 139

D.1 Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
D.2 Experimental Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
D.3 Leap-Motion Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
E Selection of feedback controller Cc,w and ILC operator Lw 147

viii CONTENTS
F Multichannel Stimulator for GOSAIL Compact 151
Bibliography 153
List of Figures
1.1 Stroke mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Organisation of thesis and steps in development of three systems . . . . . . . 9
2.1 Targeted muscles in upper limb (anterior view) . . . . . . . . . . . . . . . . . 12

2.2 Morphologic features of skeletal muscle (anterior view). . . . . . . . . . . . . 12
2.3 Stimulation types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Electrical stimulation waveforms samples reproduced from (Wysiw and Jack-
son, 2007) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.5 Previous workstations with ILC and FES assistance . . . . . . . . . . . . . . 20
3.1 BION taken from Loeb et al. (2006) . . . . . . . . . . . . . . . . . . . . . . . 22

3.2 BION block diagram taken from Schulman (2008) . . . . . . . . . . . . . . . 22
3.3 Bioness H200 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4 Microstim 2v2 stimulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.5 Neuromove NM900 taken from Neuromove.com (2011) . . . . . . . . . . . . . 27
3.6 OG Giken Pulse Cure Pro and parts taken from Og-giken.co.jp (2002) . . . . 29
3.7 MyoTrac Infiniti taken from Saebo.com (2016) . . . . . . . . . . . . . . . . . 30
3.8 MyoTrac Infiniti reproduced from Saebo.com (2016) . . . . . . . . . . . . . . 31
3.9 Stiwell Med4 taken from Stiwell (2016) . . . . . . . . . . . . . . . . . . . . . . 32
3.10 ShefStim taken from Heller et al. (2013) . . . . . . . . . . . . . . . . . . . . . 33
3.11 ShefStim array reproduced from Heller et al. (2013) . . . . . . . . . . . . . . 34
4.1 The GO-SAIL system components . . . . . . . . . . . . . . . . . . . . . . . . 38

4.2 Task positions (top view) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.3 GO-SAIL software flow diagram. . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.4 Skeletal viewer main screen showing fitted joint locations. . . . . . . . . . . . 41
4.5 Kinect for Windows taken from Microsoft (2012a) . . . . . . . . . . . . . . . 42
4.6 Stimulation input from real-time hardware to stimulator. . . . . . . . . . . . 43
4.7 Kinematic model of SaeboMAS and human arm . . . . . . . . . . . . . . . . . 44
4.8 Block diagram of the ILC scheme . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.9 Anterior deltoid identification with parameters γ, β. . . . . . . . . . . . . . . 47
4.10 GO-SAIL session flow diagram. . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.11 GO-SAIL intervention - contralateral button push . . . . . . . . . . . . . . . 52
4.12 Participant 1 low light switch test results. . . . . . . . . . . . . . . . . . . . . 57
5.1 Outputs of Vicon system taken from Vicon.com (2010) . . . . . . . . . . . . . 60

5.2 MYO taken from Getmyo (2013) . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.3 Freehand with wearable sensor taken from Kim et al. (2012) . . . . . . . . . . 62
5.4 Data Gloves taken from 5dt (2010) and Cyberglove systems (2012) . . . . . . 63
5.5 Goniometers taken from Biometrics (2012) . . . . . . . . . . . . . . . . . . . . 63
5.6 3D sensor technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.7 Kinect 1 taken from Microsoft (2012b) . . . . . . . . . . . . . . . . . . . . . . 67
ix
x LIST OF FIGURES
5.8 Kinect 2 taken from Microsoft.com (2014) . . . . . . . . . . . . . . . . . . . . 67

5.9 PrimeSense taken from PrimeSense (2012) . . . . . . . . . . . . . . . . . . . . 68
5.10 Leap Motion taken from Leap Motion (2013) . . . . . . . . . . . . . . . . . . 68
5.11 Simplified hand kinematic model and joint angles (left hand view) . . . . . . 73
5.12 Experimental hand gestures (left hand view). . . . . . . . . . . . . . . . . . . 74
5.13 Experimental set up of goniometer and non-contact sensors. . . . . . . . . . . 75
5.14 Comparison between Leap Motion and PrimeSense joint angles for three
movements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.1 System components: 1) motion tracking hardware components, 2) interactive

touch table display, 3) FES controller and multiplexor hardware, 4) Saebo-
MAS and perching stool, and 5) electrode-array. . . . . . . . . . . . . . . . . 80
6.2 Task design and graphical backgrounds: a) default (overlaid with task place-
ment geometry), b) bathroom sink, c) coffee table and d) chopping board. . . 82
6.3 Software signal flow diagram. . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
6.4 Experimental hand gestures and identified array elements (left hand view):
a) Starting, b) Open hand, c) Pointing, and d) Pinch gestures. Red and blue
squares present participant 1 and 2’s two element solutions respectively. . . . 84
6.5 Human arm kinematic model. . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.6 Block diagram of the ILC feedback scheme. . . . . . . . . . . . . . . . . . . . 88
6.7 GO-SAIL+ session flow diagram. . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.8 Unimpaired Participant 1’s tracking results for far reach task . . . . . . . . . 99
6.9 Comparison results for assisted tasks vs unassisted tracking. . . . . . . . . . . 100
6.10 Statistical results for range of movement of unassisted tasks . . . . . . . . . . 101
6.11 Stroke Participant 1’s tracking results for FES-Assisted drawer closing task . 103
6.12 Stroke Participant 4’s tracking results for FES-Assisted low light task . . . . 104
7.1 GO-SAIL Compact system overview . . . . . . . . . . . . . . . . . . . . . . . 107

7.2 Software and signal flow diagram. . . . . . . . . . . . . . . . . . . . . . . . . . 111
7.3 Block diagram of the ILC feedback scheme. . . . . . . . . . . . . . . . . . . . 115
7.4 Norm of joint angle error and proportional gain. . . . . . . . . . . . . . . . . 116
7.5 Participant’s wrist data for opening hand. . . . . . . . . . . . . . . . . . . . 117
7.6 Participant’s wrist/hand data for opening hand . . . . . . . . . . . . . . . . . 118
8.1 Leap Motion integrated with Oculus Rift taken from Leap Motion (2014) . . 124
A.1 Anterior deltoid identification with parameters γ, β. . . . . . . . . . . . . . . 128

A.2 Hand and wrist identification in GO-SAIL+. . . . . . . . . . . . . . . . . . . . 128
B.1 Start screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

B.2 Page 1: profile page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
B.3 Page 2: workspace page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
B.4 Page 3: stimulation page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
B.5 Page 4: test page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
B.6 Page 5: results page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
C.1 Duo3D taken from Duo3d (2013) . . . . . . . . . . . . . . . . . . . . . . . . . 135

C.2 DepthSense 325 taken from Softkinetic (2013) . . . . . . . . . . . . . . . . . . 136
LIST OF FIGURES xi
C.3 Intel Creative Senz3D and RealSense F200 taken from Intel (2014) . . . . . . 136
C.4 Pmdtechnologies pico and pico S taken from Pmdtec (2015) . . . . . . . . . . 137
D.1 Leap Motion hand kinematic model and global coordinate frame . . . . . . . 139
D.2 Verification Model for Experiments . . . . . . . . . . . . . . . . . . . . . . . . 141
D.3 Finger angles for static position . . . . . . . . . . . . . . . . . . . . . . . . . . 144
D.4 Finger flexion angles for grab movement . . . . . . . . . . . . . . . . . . . . . 145
D.5 Abduction angle for grab movement only thumb and index . . . . . . . . . . 145
D.6 Index finger flexion angle for separated grab movement . . . . . . . . . . . . . 145
F.1 Electronics design of Ilić et al. (1994) . . . . . . . . . . . . . . . . . . . . . . . 152

List of Tables
3.1 BIONs clinical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Comparison of home-based devices . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1 Participant demographics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

4.2 Stroke participant clinical assessment data . . . . . . . . . . . . . . . . . . . . 52
4.3 Regression slopes and p-values for FES-unassisted tasks over the 18 sessions . 53
5.1 Leading examples of contact sensors . . . . . . . . . . . . . . . . . . . . . . . 64

5.2 Comparison of non-contact sensor devices . . . . . . . . . . . . . . . . . . . . 69
5.3 Non-contact sensors software-middleware comparison . . . . . . . . . . . . . . 70
5.4 Comparison of PrimeSense and Leap Motion results . . . . . . . . . . . . . . 75
6.1 Stroke participant demographic characteristics . . . . . . . . . . . . . . . . . 92

6.2 Unimpaired participants assisted results. . . . . . . . . . . . . . . . . . . . . . 94
6.3 FES-assisted tracking results for stroke participants taken mid-way during
intervention (session 9) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.4 Stroke participant clinical assessment data . . . . . . . . . . . . . . . . . . . . 96
7.1 GO-SAIL+ and GO-SAIL Compact comparisons . . . . . . . . . . . . . . . . 119
xiii
Declaration of Authorship
I, Mustafa C. Kutlu, declare that the thesis entitled
A Home-Based Functional Electrical Stimulation System for Upper-Limb Stroke Rehabil-

itation
and the work presented in the thesis are both my own, and have been generated by me
as the result of my own original research.
I confirm that:
• this work was done wholly or mainly while in candidature for a research degree at this
University;
• where any part of this thesis has previously been submitted for a degree or any other
qualification at this University or any other institution, this has been clearly stated;
• where I have consulted the published work of others, this is always clearly attributed;
• where I have quoted from the work of others, the source is always given. With the
exception of such quotations, this thesis is entirely my own work;
• I have acknowledged all main sources of help;
• where the thesis is based on work done by myself jointly with others, I have made
clear exactly what was done by others and what I have contributed myself;
• Some of this work has been published before submission can be found in introduction.
Signed:.......................................................................................................
Date:.........................................................................................................
xv
Acknowledgements
This thesis could not have been possible without the support of many people.
I would like to express my appreciation and gratitude to my main supervisor Dr. Christo-
pher T. Freeman for supporting me both technically and morally. I followed his path
through experimental works. Thanks to him I tried multiple gadgets which helped me
to win hackathons. I would also like to extend my gratitude to my supervisors Dr. Dina
Shona Laila for her technical and Dr. Ann-Marie Hughes for her clinical guidance. I would
like to thank them for answering my questions, giving suggestions, and for their painstaking
review of this thesis.
I have enjoyed all the time that I have spent in the Vision Learning and Control (VLC)
group in ECS. I have benefited greatly from my colleagues Dr. Timothy Exell, Dr. Katie
Meadmore, Emma Hallewell within the GO-SAIL Project.
Thanks to my friends and colleagues Dr. Daisy Tong, Dr. Shaobai Li, Dr. Jing Liu, Dr.
Abeer Al Gburi, Dr. Halil Yetgin, Dr. Engin Hasan Copur, Ahmet Gelgec, and Yiyang
Chen who helped me to win a prize at the re:cognition hack 2016 with my Signect project
which uses norm optimisation to detect sign language from Kinect v2 position data and
convert it to text.
Special thanks to Dr. Oliver Brend who became a good friend of mine and joined our soup
breaks.
Thanks to the participants who joined our study and spent their time with us during clinical
studies, and their encouragement and hard work. We laughed and cried together after they
reached their goals.
I would like to thank my beloved wife Burcu for her patience and support, and our baby
Alper who brings joy to our life.
Finally, I would like to thank the Ministry of National Education of Turkey, not only pro-
viding me funding for my study, but also for stimulating us to attend conferences where I
met so many interesting people.
xvii
Nomenclature
i:j Sequence of indices i,...,j

β Rotation angle around z axis
γ Rotation angle around x axis
φ1 ...φ19 Joint angles of upper-limb
θ1 ...θ5 Joint angles of gravitation support
λ Phase-lead
B(·) Inertial matrix
C(·) Coriolis matrix
Cc Feedback controller
F (·) Frictional matrix
G(·) Gravitational matrix
G Electrode array solution matrix
g(·) Moment produced through FES
hh Vector of externally applied force and torque
hs Vector of externally applied force and torque caused by spring
h Vector of externally applied force and torque due to interaction with objects
hi (ui , t) Hammerstein structure incorporating static non-linearity
hIRC,i (t) Hammerstein structure incorporating isometric recruitment curve
hLAD,i (t) Hammerstein structure incorporating linear activation dynamics
J(·) System Jacobian matrix
K(·) Spring constant matrix
l Learning gain
L ILC operator
P Static mapping of hand corresponding to stimulation
u Stimulation applied by stimulator
W Array mapping matrix
(·)∗ Adjoint operator
(·)† Pseudo-inverse
(·)a arm component
(·)w wrist and hand component
xix
Acronyms
AC Alternative Current
AD Anterior Deltoid
ADL Activities of Daily Living
ARAT Action Research Arm Test
DC Direct Current
ECRB Extensor Carpi Radialis Brevis
EDC Extensor Digitorium Communis
EIP Extensor Indicis Proprius
EMG Electromyography
EMMSAC Estimation-based Multiple Model Switched Adaptive Control
EPB Extensor Pollicis Brevis
ES Electrical Stimulation
ESD Early Support Discharge
ETMS EMG Triggered Muscle Stimulation
FDS Flexor Digitorium Superficialis
FES Functional Electrical Stimulation
FIM Functional Independence Measurement
FMA Fugl-Meyer Assessment
FPL Flexor pollicis Lungus
ILC Iterative Learning Control
IRC Isometric Recruitment Curve
LAD Linear Activation Dynamics
LE Lower Extremity
MAS Modified Ashworth Scale
MCP Metacarpophalangeal
NMES Neuromuscular Electrical Stimulation
PC Pulsed Current
PID Proportional Integral Derivative
PIP Proximal interphalangeal
RF Radio Frequency
RMS Root Mean Square
ROM Range of Motion
RT Robotic Therapy
RETS Reciprocal EMG triggered stimulation
SBS Sensory Barrage Stimulation
TCP Transmission Control Protocol
TES Transcutenous Electrical Stimulation
UE Upper Extremity
VGA Video Graphic Array
xxi
Dedicated to my family.
xxiii
Chapter 1
Introduction
This chapter motivates the programme of research, outlines the specific aims and objectives
which underpin it and summarises the main findings. The original contributions made to the
body of knowledge related to development of stroke rehabilitation technologies arising from
this research are outlined. The thesis structure is explained and the publications resulting
from the programme of research are listed.
1.1 Justification
Stroke is the result of either the rupture or blockage of blood vessels in the brain, and is the
second biggest cause of disability globally. In 2010 stroke survivors numbered 33 million,
with most occurring in low and middle income countries (Feigin et al., 2014). Effects of stroke
include the loss of motor control, sensory disorder, cognitive impairment and loss of speech
function (Cuccurullo, 2004). Approximately 10%-20% of strokes are due to hemorrhage (a
burst blood vessel), while 80%-90% of strokes are ischemic, meaning that they are caused
by an embolism (a blocked blood vessel). Figure 1.1 shows these mechanisms (Zhang et al.,
2008).
Age is one of the substantial risk factors of stroke. The ageing of the world population means
that a growing number of people are at risk, with associated impacts on the individual, their
carers and broader society. In the UK there were 9.8 million people aged 65 or over in 2007,
and this number is projected to be 16.1 million by 2032 (Dunnell, 2008). Furthermore, stroke
is the most prevalent disability in the UK with approximately 152,000 people suffering a
stroke every year and over 300,000 people disabled as a result (Townsend et al., 2012). It
costs the UK up to £8 billion per year, as well as £1.7 billion community costs, including
nursing home care costs (National Audit Office, 2010). Due to the ageing population, the
cost to the National Health Service (NHS) from stroke is predicted to increase 315% by 2050
(Hughes et al., 2011).
1
2 Ch. 1. Introduction
Figure 1.1: Stroke mechanisms

(1) Hemorrhage caused by a burst blood vessel, (2) ischemic stroke caused by a blockage in a blood vessel
reproduced and modified from National Geographic (2014)
Approximately 70% of survivors experience altered arm function after a stroke, and 40% of
them are left with a non-functional arm (Intercollegiate Stroke Working Party, 2012). The
upper extremity is fundamental to activities of daily living (ADLs) such as eating, bathing,
dressing, and work place activities (Desrosiers et al., 2003). The ability to reach and grasp is
required in over 50% of these activities (Ingram et al., 2008). The capacity to achieve ADLs
has a direct impact on independence, and hence is closely linked to the social and financial
burden of stroke. To restore this function upper limb rehabilitation is vital. However such
rehabilitation must be cost effective, and must ideally also reduce the time patients stay
in hospital. The NHS has therefore explored assistive technology that supports patients in
their own home (Langhorne et al., 2007), and can enable early supported discharge (ESD).
Given that only 38% of stroke survivors regain dexterity and 11.6% of them regain moderate
function at 6 months after a stroke, the efficiency of ESD is still being debated (Langhorne
et al., 2009; Kwakkel et al., 2003).
1.2. Problem Statement 3
1.2 Problem Statement
The research problem addressed in this thesis involves developing more effective home-
based upper-limb assistive rehabilitation technologies for stroke patients. Robotic therapy
(RT) and functional electrical stimulation (FES) are two types of assistive technologies that
can address this problem. RT comprises mechanical devices which support patients while
performing tasks, while FES is based on artificially stimulating muscles using electrical
impulses. Both have been well documented for training of arm-hand skills, however, RT
systems are generally ill-suited for transferring to the home. In contrast, FES has a high
potential for use in effective home-based rehabilitation systems because it is inherently
low cost, can be realised in form of compact devices, and provides intensive assistance to
support lost function (Kowalczewski and Prochazka, 2011). However, existing home based
upper-limb FES systems are either expensive or have very limited scope. The lack of home
rehabilitation technology leads to ineffective physiotherapy during ESD, thus it reduces the
capacity to achieve ADLs.
A carefully orchestrated sequence of stimulation using FES to different muscle groups is

able to produce a natural movement. However, upper-limb muscles have to be stimulated
precisely in order to achieve functional movements. Although FES has shown potential
to facilitate rehabilitation, it has not been able to fully support ADLs. This is partly
because of a lack of low-cost or effective sensors and partly due to a lack of adequate control
mechanisms.
The sensors used in the most sophisticated FES systems are either expensive or hinder
natural movement (e.g. are difficult to don-doff or require markers to be attached). With
increasingly cutting-edge sensing technology becoming available, rehabilitation systems have
the potential to become more suitable for the home environment. However, all current home-
based upper-limb FES devices remain open-loop. Modern control methods such as model-
based FES controllers have not yet been realised in portable or home-based systems. Model-
based control is critical to reduce the effect of noise or disturbance, significantly increase
accuracy and enable complex functional tasks to be performed. Their use is therefore even
more crucial in the home environment. In general, the more functionality the FES system
has, the more complex it becomes to operate. This leads to additional set-up requirements
which may necessitate the involvement of engineers and physiotherapists. Moreover, the
time needed to set-up a complex system also increases with the variety and number of
protocols. No system simultaneously applying FES to the combined shoulder, arm, wrist
and hand has been clinically trialled. Some systems stimulate the hand and wrist, including
the commercially available Handmaster. Only Bion and Neuromove apply FES to the arm
and shoulder. Implanted ‘Bion’ electrodes have provided the best overall function, but have
involved costly surgery.
Based on the above motivation, the aim of this thesis is to first develop technology that
integrates advanced sensors, and a model-based controller in order to enable patients to
perform ADLs. Then focus will be on transferring it to their own homes.
1.3 Research Contribution
The research addresses the limitations in home-based technology outlined above. The results
are built on developing and integrating improvements in non-contact sensing, real-time
hardware and software technologies. The philosophy adopted has been to assess the most
advanced technologies available, before identifying components that most directly enhance
performance and usability. Technological development was carried out in parallel with
experimental testing, so that the significant challenges encountered in employing leading
edge technologies were addressed in a rigorous manner.
This research builds on a rehabilitation platform termed SAIL (Stroke Assistance through
Iterative Learning). This platform which has pioneeringly applied advanced controllers, has
instrumented a rehabilitation robot. It did not employ any advanced sensors. The clinical
results in this and previous studies were published (Hughes et al., 2009; Meadmore et al.,
2012). Within Hughes et al. (2009) the feasibility of model-based FES controllers has been
established for the elbow joint and this was extended to the shoulder in Meadmore et al.
(2012). This motivated the further extension of model-based controllers to address ADLs,
and in particular to show feasibility of extending the hand and wrist. Both these systems
employed iterative learning control (ILC), which is a control scheme formulated to support
repeated tasks. In Chapter 4 significant system extension is achieved by extending FES and
ILC to wrist joint to support fully functional ADLs.
Specific contributions are summarised as follows:
• Feasibility: The developed systems has been clinically tested with six stroke partic-
ipants. This was achieved in combination with a physiotherapist (Emma Hallewell).
Thus, the feasibility of the first system using FES and ILC for the whole arm and
hand movement has been established.
• Sensing: Non-contact sensors have been integrated in a model based control scheme.
In Chapter 4, low cost non-contact sensors have been assessed and adopted for closed-
loop control of the UL. Results establish that non-contact sensors can be used for
home-based rehabilitation. This is followed by Chapter 5 which identifies possible
sensing approaches to extend FES to control the hand and fingers. Then, two sensors
for measuring hand and wrist joints were assessed with the purpose of implementing
them in the system.
1.3. Research Contribution 5
• Electrode Array: Electrode arrays have been developed and implemented within a
home-based system. These are able to precisely produce hand and wrist movements
when tested with stroke patients, using the first model-based controllers to have been
employed clinically in Chapter 6. To improve usability in a home-based system, use
of fabric electrode-arrays is discussed in Chapter 7.
• Task Design: A selection of tasks to train ADLs have been designed and used during
clinical trials. They combine real world tasks with virtual reality (VR) to create
an augmented reality (AR) environment using an interactive touch table. These are
described in Chapters 4 and 6.
• Size and Cost Reduction: The developed system has been compacted for home
use. The model-based FES control system has been implemented within embedded
hardware and preliminary tests have been performed. These confirm that the tech-
nologies developed in this theses are suitable for home use. This is detailed in Chapter
7.
These contributions show that an effective low-cost system integrating leading edge compo-
nents is feasible for home-based rehabilitation. The following papers were published based
on the above contributions:
• Meadmore, K., Exell, T., Hallewell, E., Freeman, C., Hughes, A-M., Kutlu, M., Bur-
ridge, J., Rogers, E. (2013). Upper limb stroke rehabilitation using functional electrical
stimulation mediated by iterative learning control. In 4th Annual Conference of the
International Functional Electrical Stimulation Society, Southampton, UK.
• Meadmore, K., Exell, T., Hughes, A-M., Hallewell, E., Freeman, C., Kutlu, M., Bur-
ridge,J., Rogers, E. (2013). Electrical stimulation and iterative learning control for
functional recovery in the upper limb post-stroke. In IEEE International Conference
on Rehabilitation Robotics, Seattle, US, 6pp.
• Exell, T., Freeman, C., Meadmore, K., Kutlu, M., Hallewell, E., Hughes, A. M.,
Burridge,J., Rogers, E. (2013). Goal orientated stroke rehabilitation utilising electrical
stimulation, iterative learning and microsoft kinect. In IEEE International Conference
on Rehabilitation Robotics, Seattle, US, 6pp.
• Hallewell, E., Exell, T., Meadmore, K.L., Freeman, C.T., Kutlu, M, Hughes, A-M.
and Burridge, J.H. (2013) Goal-orientated functional rehabilitation using electrical
stimulation and iterative learning control for motor recovery in the upper extremity
post-stroke. In UK Stroke Forum Conference, North Yorkshire, UK, 4pp.
• Exell, T., Freeman, C., Meadmore, K., Kutlu, M., Hallewell, E., Hughes, A-M., Bur-
ridge,J., Rogers, E. (2014). Iterative learning control as an enabler for goal oriented
upper limb stroke rehabilitation. In IEEE American Control Conference, Portland,
US, 6pp.
• Kutlu, M., Exell, T., Hallewell, E., Meadmore, K.L ., Freeman, C.T., Hughes, A-
M. and Laila, D.S. (2014) Goal-orientated upper limb stroke rehabilitation utilizing
functional electrical stimulation and advanced sensing and control technologies. World
Congress for NeuroRehabilitation, Istanbul, Turkey.
• Meadmore, K.L., Exell, T., Hallewell, E., Hughes, A-M., Kutlu, M., Benson, V.,
Burridge, J.H., Rogers, E and Freeman, C.T. (2014) The application of precisely
controlled functional electrical stimulation to the shoulder, elbow and wrist for upper
limb stroke rehabilitation: a feasibility study. Journal of Neuro Engineering and
Rehabilitation, 11, (105).
• Hughes, A-M., Meadmore, K.L., Freeman, C.T., Kutlu, M., and Hallewell, E. (2014)
Electrical stimulation and iterative learning control combined with real objects and
simulated tasks to assist motor recovery in the upper extremity post-stroke. In Interna-
tional Functional Electrical Stimulation Society Conference, Kuala Lumpur, Malaysia,
5pp.
• Hughes, A-M., Freeman, C.T., Kutlu, M., Meadmore, K.L., and Hallewell, E. (2014)
Combining electrical stimulation mediated by iterative learning control with move-
ment practice using real objects and simulated tasks for post-stroke upper extremity
rehabilitation. Neurosonology and Cerebral Hemodynamics, 10, (2), 117-122.
• Kutlu, M., Freeman, C.T., Hallewell, E., Hughes, A-M. and Laila, D.S. (2015) Electrode-
array based functional electrical stimulation system for upper-limb stroke rehabilita-
tion with innovative sensing and control approaches. In Technically Assisted Rehabil-
itation Conference, Berlin, Germany, 3pp.
• Kutlu, M., Freeman, C.T., Hallewell, E., Hughes, A-M. and Laila, D.S. (2014) Goal-
oriented upper-limb stroke rehabilitation utilising functional electrical stimulation with
advanced sensing and control. In Set for Britain 2015, London, UK.
• Kutlu, M., Freeman, C.T., Hallewell, E., Hughes, A-M. and Laila, D.S. (2015) Upper
limb stroke rehabilitation combining electrode-arrays with low-cost sensing and ad-
vanced control. In 5th Annual Conference of the International Functional Electrical
Stimulation Society, Sheffield, UK.
1.4. Thesis Structure 7
• Kutlu, M., Freeman, C.T., Hallewell, E., Hughes, A-M. and Laila, D.S. (2015) FES-
based upper-limb stroke rehabilitation with advanced sensing and control. In IEEE
International Conference on Rehabilitation Robotics, Singapore, 253-258
• Kutlu, M., Freeman, C.T., Hallewell, E., Hughes, A-M. and Laila, D.S. (2016) Upper-
limb stroke rehabilitation using electrode array based functional electrical stimulation
with sensing and control innovations. Medical Engineering and Physics 38 (4), 366-
379.
• Freeman, C.T., Yang, K., Tudor, J. and Kutlu, M. (2016). Feedback control of elec-
trical stimulation electrode arrays. Medical Engineering and Physics.
http://dx.doi.org/10.1016/j.medengphy.2016.07.002
• Kutlu, M., Freeman, C.T., Hughes, A-M., Spraggs, M. (2017) A home-based FES
system for upper-limb stroke rehabilitation with iterative learning control. In the
20th World Congress of the International Federation of Automatic Control, France,
6pp.
• Kutlu, M., Freeman, C., Spraggs, M. (2017) Functional electrical stimulation for home-
based upper-limb stroke rehabilitation. In Technically Assisted Rehabilitation Con-
ference, Berlin, Germany, 3pp.
• Kutlu, M., Freeman, C., Spraggs, M. (2017) Functional electrical stimulation for home-
based upper-limb stroke rehabilitation. Current Directions in Biomedical Engineering,
3(1), pp. 25-29. doi:10.1515/cdbme-2017-0006
1.4 Thesis Structure
The technology developed in this thesis has been physically realised within three self con-
tained rehabilitation systems, called GOSAIL, GOSAIL+, and GOSAIL Compact. Figure
1.2 illustrates how the technology is partitioned within each chapter. The limitations of each
of these systems are used to motivate the next system, and hence the systems in this thesis
are strongly inter-related. In Chapter 2, the justification for this research is discussed by re-
viewing existing stroke rehabilitation technologies, as well as motor learning mechanisms. In
Chapter 3, currently available home-based FES systems for upper-limb stroke rehabilitation
are examined and compared on the basis of cost, complexity and functionality. In Chapter
4 attention is then applied to the most advanced FES rehabilitation system currently in
development, which uses non-contact sensing of the upper-limb, together with model-based
control. Having isolated sensing as a fundamental component in the transference to the
home environment, Chapter 5 reviews the state of the art in contact and non-contact sens-
ing technologies. Then, it focuses on applying two leading types of sensor which are then
examined and compared. From this information, the most appropriate sensor is selected.
In Chapter 6, a prototype rehabilitation system is developed which integrates the sensors;
together with novel control schemes, task display, and electrode arrays. Clinical test results
using the system are presented and discussed. In Chapter 7, a compact prototype is devel-
oped which meet all required specifications and combines all the functionality of the previous
system. Preliminary results from a healthy individual using the system are presented and
discussed. Finally, Chapter 8 evaluates the overall contributions of this thesis and outlines
areas for future work.
1.5 Summary
This chapter highlights current gaps in knowledge related to design and control of stroke re-
habilitation systems. It motivates the need for more advanced home-based FES technologies
and outlines the aim and objectives of this research. It also details the main findings and the
original contributions made to the body of knowledge in upper-limb stroke rehabilitation
system development. The thesis structure is explained and the publications and presenta-
tions resulting from the research are listed. The following chapter presents an overview of
research on upper-limb stroke rehabilitation which underpins this thesis.
Part 1: GO-SAIL
1.5. Summary
Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 8
Upper Limb Stroke Home-Based FES Goal-Oriented Stroke

Rehabilitation Technology Rehabilitation
Utilising Electrical
Stimulation
Part 2: GO-SAIL+ Conclusions

Chapter 5 Chapter 6
Introduction and
Sensor Technologies GOSAIL with
for the Hand and Wrist Improved
Hand Functionality
Future Work
Part 3: GO-SAIL Compact
Chapter 7
Compact System Implementation
Figure 1.2: Organisation of thesis and steps in development of three systems

9
Chapter 2
Upper Limb Stroke Rehabilitation
Every year 15 million people worldwide suffer a stroke (Mackay et al., 2004). In developing
countries such as China, where 2.5 million people have a stroke per year (Wu et al., 2013),
there is predicted to be a tripling in stroke mortality in the next two decades. On the other
hand, in developed countries such as the United States, where approximately 800,000 people
annually suffer from a stroke, the stroke death rate decreased by 34.8% from 1998 to 2008.
This means that the number of stroke survivors is increasing (Roger et al., 2012). Thus,
there is an urgent need to rehabilitate these survivors so that they will be able to live more
active, fulfilled lives (Saka et al., 2009).
Research into the effect and rehabilitation of stroke can first be divided by classifying as
upper-limb or lower-limb. The former encompasses reaching and grasping tasks (Egglestone
et al., 2009), whereas the latter deals with gait, sit-to-stand and general mobility (Teasell
et al., 2009). The focus of this thesis is on upper-limb rehabilitation. This provides a strong
motivation since approximately 70% of stroke survivors experience arm function impairment
after a stroke, and approximately 40% of survivors are left with continuous functional loss
(Houwink et al., 2013).
To understand the needs of rehabilitation, the physiology and anatomy of the arm is now
summarised. Skeletal muscles, which consist of motor neurons and the many correspond-
ing muscle fibres which they innervate, produce functional movements when they receive
stimulation (Lynch and Popovic, 2008). In Figures 2.1 and 2.2, the morphologic features
of skeletal muscles and the principal muscles involved are respectively represented (Taylor,
2012). The shoulder comprises an integral part of the upper-limb and is a critical concern
in upper limb rehabilitation. As can be seen from Figure 2.1(a), the anterior deltoid is the
largest muscle in the shoulder involved in actuating the arm. It produces the largest moment
of all the shoulder muscles throughout arm lifting (Kuechle et al., 1997) and has additionally
the largest cross-sectional area (Bassett et al., 1990). In Figure 2.1(b), the triceps brachii is
shown, which provides torque about the elbow in combination with the biceps. As shown in
Figure 2.1(c), the extensor carpi radialis has an important role in moving the hand about
the wrist.
11
12 Ch. 2. Upper Limb Stroke Rehabilitation
(a) Anterior Deltoid (b) Triceps Brachii (c) Extensor Carpi Radialis
Figure 2.1: Targeted muscles in upper limb (anterior view)

Reproduced from (Taylor, 2012)
Figure 2.2: Morphologic features of skeletal muscle (anterior view).

Reproduced from (Taylor, 2012) and (Baker, 2000)
The locomotive systems comprise two components that are excitable:
• the neuron, a conductor that transmits electrical signals around the body to other
neurons and target organs;
• the muscle fibre that contracts upon excitation, allowing the body to perform complex
movements.
The long process of the neuron (the axon), and the muscle fibre are both surrounded by
a membrane, the polarity of which is determined by the distribution of negatively charged
potassium and positively charged sodium ions along its inner and outer surfaces, respectively.
The body regulates the flow of these ions by opening and closing ion-specific channels, and
13
activating ion pumps along the length of the membrane. A constant membrane resting
potential is achieved when the flow of charge into and out of the membrane are equal.
During voluntary muscle contractions, motor neurons are excited asynchronously (at dif-
ferent times and rates) at 20 - 25 Hz. At these frequencies, the refractory period of each
individual neuron is capable of expiring before the stimulus for the next Action Potential
(AP) is applied. Each nerve produces APs of equal amplitude, meaning that the size of the
response can only be graded in terms of the frequency of APs. Each neuronal AP induces
an AP in the motor unit which it is attached to. The result is a single brief contraction
of the motor unit, which is referred to as a twitch. After the twitch has occurred, muscle
fibres can take over 200 ms to fully relax. The post-AP refractory period of the muscle fibre
membrane has long since expired before this time period has elapsed, which means that fur-
ther APs can occur while the muscle is still in a contracted state. This repeated excitation
results in the summation of the contractile forces produced during each individual twitch.
At excitation frequencies of approximately 25 Hz and above, the allowed relaxation period
is significantly small that a smooth (tetanic) step-like muscle contraction is produced. At
tetany-inducing frequencies, the size of the total contractile force produced by the muscle
is determined by the number and type of motor units that are recruited. Typically, the
body recruits the smaller slow-twitch muscle fibres first. These fibres have high fatigue re-
sistance, which allows contractions to be sustained for a longer period. In order to produce
contractions of greater strength, the larger fast-twitch muscle fibres are recruited. These
muscle fibres have low fatigue resistance and produce relatively fast, powerful contractions
of short duration. There are a number of physiological effects that result in the variation
of the natural response of muscle. In particular, the response of muscle is highly dependent
on its contractile history. Post-activation potentiation is a phenomenon whereby increased
muscle contractile force is observed after a series of initial contractions with the same level
of stimulus is applied. The effect can last up to an hour after the initial ’warm up’ activity
has ended.
Rehabilitation comprises undertaking a set of activities that assist individuals with a disabil-
ity to achieve and maintain function in interacting with their environment (World Health
Organization, 2011). For stroke patients, the most critical period for rehabilitation is during
the first 24 to 48 hours post-stroke which are pivotal in determining their degree of recovery.
After the nerve cells are killed, there is no way to regrow them. Cutting edge stem cell
technology has shown potential to replace cells. However, stem cell research is still at an
early stage and clinical trials to date have only focused on safety against tumours (Stone
et al., 2014). Research performed on rats has shown positive results (Tornero et al., 2013).
However, even after placing new nerve cells, the level of relearning capability is still being
debated. On the other hand, the brain is able to relearn lost skills through practice. This is
due to neuroplasticity, a process by which connections are formed between neurons in the
motor cortex of the brain. New neural connections can be made in the brain, which has
plenty of spare capacity which may be utilised for this process (Winstein et al., 2004).
If a neuron cell is near enough to excite another neuron cell repeatedly, additional pathways
start being formed in one or both of the cells. This mechanism causes a type of learning
called Hebbian Learning (Teo, 2009) and there exists substantial literature explaining the
significant anatomical and neurochemical changes which assist the recovery of lost function
(Frost et al., 2003). Motor learning theory provides a strategic approach for maximising
learning through manipulation of the environment (Wolpert et al., 2001; Winstein et al.,
2004) as motor cortex parts are able to adapt their function. This adaption takes the form
of rapid changes in one area of the motor cortex at the expense of other surrounding areas
under the influence of intensive training. In particular, strong relationships between rapid
changes of activity and intensive versus extensive training have been identified (Teasell et al.,
2008). In addition, Timmermans et al. (2009) highlighted that there is strong evidence for
physical therapy interventions which include intensive, highly repetitive task-oriented and
task-specific training in all phases post stroke. This has been highlighted in Winstein et al.
(2016). This presents a major challenge to healthcare providers and is driving the develop-
ment of rehabilitation technology which can deliver this specific and intense rehabilitation
without using additional resources.
A variety of therapeutic approaches exist to assist patients’ movement, with a comprehensive

review of upper extremity interventions for stroke patients appearing in the Evidence-Based
Review of Stroke Rehabilitation (EBRSR) (Teasell et al., 2008). Rehabilitation approaches
are now discussed focusing on their effectiveness for stroke recovery.
2.1 Conventional and Robotic Therapy
This section focuses on two of the most common rehabilitation methods. Note that neither is
appropriate for home use as either a physiotherapist or engineer is required, or prohibitively
expensive and/or large equipment is needed.
2.1.1 Conventional Therapy
Conventional therapy involves a physiotherapist manually assisting patients’ movement dur-

ing repetitive, cyclical activities. This approach may not encourage maximum voluntary
effort and may not be motivating for the patient depending on the therapist-patient rela-
tionship. Conventional therapy to improve upper-limb function is not as effective as we
might expect (Ernst, 1990). Winstein et al. (2016) guidelines outline whichever therapies
have the strongest evidence and state that intensity is important but difficult and costly for
conventional therapies to provide. However, conventional therapy methods are still widely
used within stroke rehabilitation.
2.1.2. Robotic Therapy 15
2.1.2 Robotic Therapy
Robotic therapy (RT) comprises mechanical devices which support patients in performing
tasks, usually in a virtual reality (VR) environment. The main advantage of robotic systems
is that they provide highly controllable support for patients’ limbs and can provide both
resistance and assistance to complete tasks (Kowalczewski and Prochazka, 2011). Robotic
systems have ability to provide longer, more intense and more precise therapy. According to
the functional independence measurement (FIM) index (Van der Loos, 2002), robotic ther-
apy at present provides no benefit over intensity matched conventional therapy (Mehrholz
et al., 2015). There are a large range of passive and active robotic devices available for upper-
limb stroke rehabilitation including the InMotion Arm (Lo et al., 2010) (Clinical version of
MIT Manus), and ArmeoPower. Their structure can be broadly categorised as end-effector
or exoskeletal, and they have been used in a large number of clinical studies (Prange et al.,
2006). RT is still a new treatment platform compared with conventional therapy. Techno-
logical advances such as the use of soft robotics (Polygerinos et al., 2015b) bring potential
improvement in usability and function, however, currently their disadvantages include:
• They rarely combine reaching and grasping.
• The mechanical power provided, even when the amount of support is adjusted to each
individual, does not encourage maximum voluntary input.
• Performance of virtual tasks precludes the normal haptic feedback experienced during
real tasks.
• RT devices are often large and expensive and thus ill-suited to the home.
2.2 Rehabilitation with Electrical Stimulation
Rehabilitation using electrical stimulation (ES) was first used in the 1960s and 1970s (Loeb
et al., 2006). The first stimulation devices focused on lower-limb gait rehabilitation and
tried to address the problem of drop foot. Upper-limb rehabilitation systems emerged af-
ter the multi-channel stimulator was developed in 1990 (Loeb et al., 2006). ES works by
stimulating nerves, especially those which connect to muscles (as shown in Figure 2.3); affer-
ent (sensory) stimulation, efferent (motor) stimulation, and denervated muscles stimulation,
which respectively target peripheral nerves for pain treatment, motor nerves for actuating
muscles and the effected muscles themselves for reducing muscle loss, are shown. In order to
fully understand electrical stimulation therapy, “paralysed muscle” and “denervated muscle”
have to be defined. Paralysis is the inability of muscle to move because of central nervous
system injury or illness. Usually, the integrity of the peripheral nerve remains. If, however,
after an injury or illness, the peripheral nerve transmission is lost, the muscle has become
“denervated”. Electricity is not applied to muscles directly except in the case of denervated
muscles, because direct stimulation would contribute to muscle contraction from the exte-
rior of muscles which would cause fatigue of myofibrils if used continuously and multiple
times. Furthermore, this approach would not cause motor learning because of there being
no connection between muscles and motor nerves (Ragnarsson, 2007; Nicolás Cuenca and
Lazar, 2008).
+ -
Sensory neuron
Skin
Motor neuron Muscle tissue
Afferent stimulation
+ -
Sensory neuron
Skin
Muscle tissue
Motor neuron
Efferent stimulation
+ -
Sensory neuron
Skin
Motor neuron Muscle tissue
Denerved stimulation
Figure 2.3: Stimulation types
Sensory (afferent) stimulation, motor (efferent) stimulation, and denerved muscle stimulation reproduced from
Dalkilic (2013)
Physiological nerve recruitment frequency of a healthy person is between 6 and 8 Hz. In

ES, a frequency of 20-40 Hz is needed in order to cause muscle contraction (Lynch and
Popovic, 2008). Moreover, there are different signal characteristics for the current used in
ES systems for providing effective stimulation, which are defined as direct (DC), alternating
(AC) and pulsed current (PC). DC is mainly utilised for stimulating denervated muscles
and for directly stimulating muscles. Thus, it does not promote motor learning; rather, it
simply prevents the loss of muscle tissue (Kern et al., 2010). On the other hand, AC and
PC are widely used with different signal waveforms for attaining motor learning results.
There are interferential (Palmer et al., 1999), Russian (Ward and Shkuratova, 2002), pre-
2.2. Rehabilitation with Electrical Stimulation 17
moduled (Ward, 2009), micro (Electrotherapy Standards Committee, 1990) and high voltage
current waveforms. Moreover, initially devices were known as: Russian, Galvanic, Faradic,
Interferential and High Voltage (HV) according to the waveforms used. According to the
shape and phase characteristic of these current waveform, waveforms can be classified as
Direct Monophasic Monophasic

0 0 0
Current Double-spike
Amplitude
Amplitude
Amplitude
0
Alternating Symetric
Current 0
Biphasic 0 Sinusoidal
Phase Interpulse
interval
0
Pulsed Asymetric
0 0
Current Biphasic Pulse
Pulse period
Figure 2.4: Electrical stimulation waveforms samples reproduced from (Wysiw and Jack-
son, 2007)
sinusoidal, rectangular, triangular, square or spiked and also as monophasic, biphasic or

polyphasic (Cogan, 2008). Examples of these waveforms are illustrated in Figure 2.4. Many
current devices have been developed, and produce varied waveforms to suit patients’ needs
during treatment.
ES can also be classified as functional electrical stimulation (FES) whilst used as an efferent
stimulation for rehabilitation. It can also be classified as transcutaneous electrical neural
stimulation (TENS) when used as an afferent stimulation for therapeutic purposes as shown
in Figure 2.3. The least invasive method for applying the electrical stimuli involves the
use of two or more electrodes attached to the surface of the skin. The electrodes are con-
nected to an electrical stimulator, which is used to develop a potential difference between
the attachment points. Electrodes, as the interface between the electronic and biological
systems, are required to be capable of effectively delivering a stimulus to the muscle, while
avoiding other excitable tissues. In this research surface electrodes are employed for FES
because they are easily applied and non-invasive. Thus, high resistive hydrogels are used to
effectively stimulate nerves - most hydrogels used in FES are low resistivity the key func-
tions of the (sticky) hydrogel layer in electrodes are to moisten the skin, thereby making it
more conductive and, to help maintain good contact with the skin. Thin, high resistivity
hydrogels are used in array applications to limit current spread under electrodes. Another
drawback of electrodes is those nerves closer to the surface experience higher stimulation
than deeper, less accessible nerves, causing the fibres to which they are connected to fatigue
faster.
The stimulation pattern commonly used to excite motor neurons is a rectangular pulse train.
The duration of each pulse is typically between 5 µ s and 500 µ s, applied at a frequency
between 10 Hz and 100 Hz. FES therapy has attracted considerable research interest, where
it is used on weak or paralysed muscles to assist movement and promote motor learning.
Clinical evidence exists to support the therapeutic use of FES to improve motor control
(De Kroon and IJzerman, 2008; Coupar et al., 2012). On the other hand, research has
suggested that applying FES on denervated muscles does not support motor learning. It
can, however, protect muscle tissue from fibrosis by stimulating the muscle directly.
FES therapy is the most effective when it produces a natural movement that corresponds
to voluntary intention. Then, by gradually reducing the stimulation level, patients’ volun-
tary effort can be encouraged. However, there are a number of differences between natural
and ES-induced motor response that limits the ability of commercial stimulators to produce
contractions that mirror those produced during voluntary activation. The most significant
difference is the order in which motor units are recruited. During normal physiological ex-
citation the smallest, most fatigue resistant fibres are recruited first. The distribution of
motor units is such that the largest, least fatigue resistant fibres are located closest to the
stimulating electrodes. Additionally, these fibres have the lowest excitation thresholds on
account of their size. Thus, during FES-induced contractions the largest, least fatigue resis-
tant fibres are recruited first, i.e. in an order opposite to that seen in voluntary contractions
which further compounds the fatiguing effect of FES. Another important difference between
natural and FES induced movement is that in a natural movement both sensory and motor
neurons recruited to produce functional movements which means sensory nerves is activated
before movement as they are activated before motor neurons. This allows muscles to prepare
for a contraction. However, as neuroplasticity happens throughout both CNS and brain the
stimuli produced from CNS causes spasticity. Theoretical results from neurophysiology and
motor learning research support clinical research with the conclusion that the therapeutic
benefit of stimulation is maximised when applied coincidentally with a patient’s own volun-
tary intention to move (Burridge and Ladouceur, 2001). As is the case in learning how to
ride a bicycle, the patient can re-learn movement through repeated practice. By repetition,
new nerve connections can be made within the brain (Lynch and Popovic, 2008). A popular
approach to control FES is to use electromyography (EMG) to detect when the patient is
trying to use a muscle and then apply stimulation to aid the desired movement (Hara et al.,
2008). In this manner, the FES can be made to coincide with voluntary intention, albeit
with a delayed response.
The delayed response is not always suitable for neurological conditions due to discoordination
or weakness. FES technology is widely used. Moreover, approximately, 2,500 therapists have
received training for the use of FES in the UK and abroad and 16,000 FES devices have
been sold in the UK as the commercial provider reports (OML, 2017; REF, 2017). This
number is predicted to increase in-line with the number of people living with stroke.
2.2.1. FES Controllers 19
2.2.1 FES Controllers
To accurately control muscles and subsequent movement, a well-designed control system is

required. Ferrarin et al. (2001) investigated open-loop, closed-loop proportional-integral-
derivative (PID), feedback-feed-forward and adaptive controllers to control knee angles,
finding that an adaptive controller provided the best results. Simple open-loop or triggered
controllers, however, do not provide the necessary feedback, as was reported by Lynch and
Popovic (2008). For that reason, feedback-feed-forward, fuzzy logic, adaptive controllers,
among others, were employed in FES systems (Blana et al., 2009; Blaya and Herr, 2004;
Davoodi and Andrews, 1998).
Model based feedback control is critical to reducing the effects of noise/disturbance by

adjusting the FES according to achieved motion, measured using sensor data. This facilitates
a significant increase in accuracy and enables complex tasks to be performed. Although
many model-based FES control strategies have been employed to control movement (Zhang
et al., 2007), most are designed for spinal cord injury (SCI) subjects, and have a strong
focus on the lower limb. The complexity of the musculo-skeletal system and the difficulty
of muscle selectivity and recruitment means that fewer approaches have been applied to the
UE (Lynch and Popovic, 2008). This has led to the great majority of the UE systems that
have been clinically trialled employing feed-forward or triggered control (Mangold et al.,
2009; Pelton et al., 2012). However, a small number of clinically trialled UE rehabilitation
systems employ feedback control (Kurosawa et al., 2005) including an adaptive controller
for the hand and wrist (Nathan, 2005), and proportional-integral-derivative controllers for
the shoulder and elbow joints which are combined with a support that sequentially restricts
motion to a single joint in turn (Klauer et al., 2014). Greater accuracy has been shown
through the incorporation of model information within the controller. For example, artificial
neural network (ANN) approaches have been used to approximate nonlinear components of
the dynamic system within feedback control schemes (Blana et al., 2009), giving rise to
asymptotic tracking capability. However, ANNs require retraining for different tasks and
often lack robust stability guarantees.
While there are many control methods for FES, few model-based approaches have been
applied clinically on the upper-extremity. Some of the most accurate upper extremity task
tracking clinical results have been achieved using iterative learning control (ILC). ILC learns
over repeated attempts of a tracking task what control input minimises the tracking error.
This structure exactly fits the need to relearn movement over repeated attempts. ILC has a
well-developed feed-forward control structure and has been shown to be highly effective in
producing precise tracking of repeated tasks (Freeman et al., 2012). Research combining ILC
and FES commenced in 2005 and focused on planar reaching movements (Freeman et al.,
2007) as shown in Figure 2.5(a). There followed a secondary system in 2008 which extended
ILC to support 3D reaching tasks incorporating explicit reference trajectories for the patient
to follow (Cai et al., 2011). This system was called Stimulation Assistance through Iterative
'
(a) Planar system (b) SAIL system and components
Figure 2.5: Previous workstations with ILC and FES assistance

Components of the SAIL system: (1) ArmeoSpring, (2) surface electrodes on triceps and anterior deltoid, (3) FES
module, (4) real-time processor and interface module, (5) PC, (6) monitor displaying task, and (7) operator monitor.
Learning (SAIL) and is shown in Figure 2.5(b). To target training of activities of daily
living, it assists functional reaching movement, but the need to have a defined trajectory
means they are presented in a virtual environment.
2.3 Conclusions
The rising number of people living with the effects of stroke, and the associated cost, means
there is a pressing need for effective rehabilitation technology. Rehabilitation robotics and
FES both have potential to improve levels of treatment, however, the most effective current
technology is expensive and bulky. This means currently they are in labs or a small number
of hospitals. In addition, three quarters of patients are over 65 years old; and cannot always
move easily to access this facilities (National Audit Office, 2010). It follows that one of the
most important drives in upper-limb stroke rehabilitation is to develop effective home-based
systems.
FES technology is a leading assistive technology for stroke and has proven clinical effec-
tiveness. It has translated into clinical practice in some areas, as well as for home use. To
establish the current state of the art, the next chapter reviews the systems that are capable
of supplying FES based rehabilitation in the home or local stroke clinic.
Chapter 3
Home-Based FES Technology
The previous chapter provided an overview of FES technology in terms of function, ap-
plication, and clinical deployment. This chapter focuses on how this technology has been
expressly developed for a low-cost portable or home-based setting. The main aspects of
a home-based system are (1) the requirement of an engineer is not necessary to set-up or
use, (2) user-friendly in terms of both user interface and to don/doff, and (3) to effectively
support/rehabilitate patients. The small number of upper-limb portable or home-based sys-
tems currently are available. This chapter contains a detailed overview of these systems,
focusing on usability, engagement, scope and effectiveness.
3.1 BION
BION, an abbreviation for Bionic Neuron, is an RF-powered continuous injectable micros-

timulator as shown in Figure 3.1. Initially pioneered by Loeb, they continue to be developed
for both medical and engineering applications (Schulman, 2008). The technology was in-
spired by pacemakers; however, the setup muscle groups are very different (cardio-muscles
and skeletal-muscles for pace-makers and BIONs respectively). In the 1980s and 1990s, a
variety of injected multi-channel devices were developed in order to achieve selective muscle
stimulation and remove the problem of electrode placement. Nevertheless, they had lim-
itations such as long surgery hours, infection risk and requiring large RF antennae. The
novelty of BIONs was to use threshold power to stimulate nerves internally (Loeb et al.,
2006; Heetderks, 1988). BIONs comprise the first injectable RF-powered microstimulator
and were conceived to be used in significant numbers and to stimulate individual muscles.
In principle, this enables more precise control of movement which, even under open-loop
control, may be expected to translate to clinical improvements when used for the purpose
of rehabilitation.
21
22 Ch. 3. Home-Based FES Technology
Figure 3.1: BION taken from Loeb et al. (2006)
3.1.1 Functionality
BION devices can be used for both upper-limb and lower-limb rehabilitation. As an FES
device, it shares some similarities with pacemakers since both of them are invasive devices
and stimulate muscles. The BIONs can be inserted adjacent to different nerves, allowing
the contraction of different muscles. It is possible to achieve several tasks with the BION
stimulator. It can be controlled by a computer and requires a sleeve to be worn during task
practice.
Figure 3.2: BION block diagram taken from Schulman (2008)
3.1.2 Specification
An overview of the system is given in Figure 3.2 (Schulman, 2008). Firstly an alternating
magnetic current induces current in the stimulator, and this field is checked by the controller
3.1.3. Advantages and Disadvantages 23
amplitude buffer to ensure it is correct. As a result of these controlled data, BION devices
make it easier to control muscles from the inside of tissue. Therefore, skin absorbance can
be omitted and the required energy decreases.
3.1.3 Advantages and Disadvantages
There are significant disadvantages to the use of BIONs. First of all, there is the need for
a surgical or injection operation, thus raising the cost of the system. Also, it has to be
placed correctly, its operation needs to be well planned before the injection and it has a long
setup time which involves wearing external hardware. However, BION based rehabilitation
is more efficient than using other devices because it stimulates nerves internally. Thus, the
threshold value of BIONs is 8 Hz while a transcutaenous stimulator’s threshold value varies
between 20 and 40 Hz (Lynch and Popovic, 2008). This means the patient has less fatigue
compared to when using other systems (Popovic et al., 2007).
3.1.4 Clinical and Non-Clinical Evaluation
BIONs have been tested with both stroke patients (Baker et al., 2004; Turk et al., 2008;
Merrill et al., 2011) and unimpaired subjects (Schulman, 2008; Loeb et al., 2006). Using
the same open-loop operation injected stimulation is more effective than surface stimulation
according to both Fugl-Meyer Assessment (FMA) and Action Research Arm Test (ARAT)
scores in a clinical study. There are shoulder subluxation and hand exercise programme
for upper-limb rehabilitation. These exercise programmes include two to three sessions
of approximately half an hour per day, consisting of 3-second training periods of stimuli
separated by a 3-second rest time for those who have subluxation (>6 months) for 6 weeks.
Following the ES training, participants were divided into two groups according to whether
they used implanted stimulation or surface stimulation. The number of patients who have
implemented BIONs and who have used surface stimulation for different exercise studies
is detailed in Table 3.1 (Schulman, 2008). These groups undertook rehabilitation at home
for 6 weeks. Both groups demonstrated similar improvement to using open-loop surface
stimulation via electrodes. The implanted stimulator, however, led to improved patient
comfort and avoided the difficulties associated with surface electrode placement procedures.
Table 3.1: BIONs clinical results

Reproduced from Schulman (2008)
No. of patient No. of BIONs
Study Enrolled Withdrew Recd Implants Implanted Out of Spec Explanted
acute shoulder subluxation 14 0 9 19 0 0
chronic shoulder subluxation 14 0 10 22 8 0
hand contracture 16 1 10 25 7 0
3.2 Bioness Handmaster H200 Wireless
The Handmaster H200 (Bioness Inc., Valencia, CA, USA) is a device which focuses on wrist
hand rehabilitation and utilises electromyography (EMG) (Nathan, 1994). It was designed
and produced for wrist and hand rehabilitation for home-based therapy by using surface
electrodes. In the most recent design, the H200 comprises two different parts; the orthosis
and wireless control unit. The orthosis is placed on the patient's forearm and wrist, and has
two sizes (medium or large) (Bioness.com, 2006).
Figure 3.3: Bioness H200

Taken from (Bioness.com, 2006)
3.2.1 Functionality
The Handmaster trains three exercises and three functional modes: flexor, extensor and
finger muscle stimulation for the motor regulation program; sequential key grip or palmar
grasp and release pattern for functional exercise and neuroplastesis programs (Hara et al.,
2008). This device stimulates extensor digitorum communis (EDC), extensor pollicis brevis
(EPB), flexor digitorum superficialis (FDS), flexor pollicis longus (FPL) and the thenar
muscles (Snoek et al., 2000). The system uses an open-loop controller triggered by a button.
Bioness pads are placed under the orthosis, and this provides patients with a stable pad
placement.
The main advantage of this system is that it uses wireless communication and does not need
any other external processing. The second advantage is that the total weight carried by
the patient’s arm is small (medium orthosis 325g and large orthosis 396g). Furthermore, it
is designed to impact a minimum moment on the forearm. On the other hand, it requires
medical analysis and the user may need botox injections in order to find the correct muscle
3.2.3. Clinical and Non-Clinical Evaluation 25
positions and relax them. It has limitations due to limited tasks and its simplistic controller
which means it does not support motor learning effectively. Moreover, it only focuses on
arm and wrist rehabilitation, neglecting the shoulder. It is also expensive and costs around
£4770.
Snoek et al. (2000) tested and analysed the Handmaster H200. As mentioned, the control
unit allows three exercise modes and two motor regulation modes to be chosen, which
provide repetitive stimulation to improve muscle strength and condition, and a key- and
palmar grasp stimulation pattern. When the specific grasp is chosen via a trigger button on
the control unit, extensor stimulation opens the hand. After a predetermined and modifiable
interval, the flexor stimulation starts in order to achieve the grasp. The flexor stimulation
is maintained for several seconds. Then the trigger button starts the extensor stimulation
to release the object. The extensor stimulation is stopped after a predetermined duration.
With this exercise pattern, patients attempted to perform a task set, which comprised
pouring water from a can, opening a jar, cutting meat and brushing teeth. The results
showed an improvement in hand movements for the 10 participants who were tested, with
4 of these 10 patients being able to perform the task set. Hara et al. (2008) also analysed
the system, and their results demonstrated that the system is effective in reducing hand
impairment. For wrist and finger rehabilitation, the H200 provided good results on Box and
Block, FMA and Light-lift J-T tests in a study which involved using the system for half an
hour, twice per day and 5 days per week for 12 weeks (Alon et al., 2008).
3.3 Microstim 2v2
The Microstim 2v2 neuromuscular stimulator is intended to exercise weak or paralysed

muscle. It is designed to be simple to use with the minimum number of user controls
to improve usability. This device is representative of a large of similar units, which are
commonly used by stroke patients.
3.3.1 Functionality
As a home based FES device, Microstim 2v2 is convenient and has 10 modes of operation.
Its simplicity makes it easy to use. It is easy to set up and does not require much training
or any assistance; however, it only stimulates two muscles with pre-programmed open loop
control and the patient is able to tune only one muscle at a time.
Figure 3.4: Microstim 2v2 stimulator

Taken from Odstockmedical.com (2016)
The main advantage of the Microstim 2V2 is its small size and basic operation which can be
easily tuned by patients. In addition, it is cheaper when compared with other stimulation
devices, with its cost of around £200.
Several studies have been conducted on the MicroStim 2v2 (Mann et al., 2005; Taylor, 2004).
As a result of this research, it was reported that FES increases hand function and sensation.
In the study by Mann et al. (2005), the device was used on triceps and wrist and hand
extensors of 11 of 22 subacute stroke patients for 10 to 30 minutes, twice a day for 12 weeks.
This was followed by a non-intervention period of 12 weeks. The control group patients
did passive stretching exercises. In the study by Taylor (2004) tests were conducted with
twenty subjects, and the control group also had balance therapy with EMG captured from
movement of the non-effected arm. This study only focuses on the forearm for finger, thumb
and finger extensors and flexors. These studies used Jebsen-Taylor and ARAT tests which
show the improvement of functionality (Taylor, 2004)
3.4 Neuromove NM900
The Neuromove NM900 (ZYNEX MEDICAL, Littleton, CO, USA) is a battery-powered

portable EMG triggered muscle stimulator (ETMS). Its main novelty is combining EMG
and neuromuscular electrical stimulation (NMES). EMG signals detect the level of muscle
activity and the stimulator stimulates according to these signals. After the patient’s mus-
cles reach a present threshold level, NM900 continues giving the muscles biphasic waveform
pulses for the purpose of performing tasks (Neuromove.com, 2011). The patients focus on
attempting to move their muscles, and they are able to observe activation in a graphical
3.4.1. Functionality 27
representation on the device’s screen, which automatically displays the number of repeti-
tions. After capturing the patients’ muscle activation, a target is determined for them to
achieve. Achieving these goals is rewarded by stimulation, motivating positive feedback
(biofeedback).
Figure 3.5: Neuromove NM900 taken from Neuromove.com (2011)
3.4.1 Functionality
As an ETMS, the system is only useful for stimulating muscles based upon their electromyo-
graphic threshold value and only operates when someone tries to activate specific muscles.
Therefore, there is no association between UE movement and the required muscles to pro-
duce that movement. However, in light of the work done by Gabr et al. (2005), some stroke
patients have an increment in FMA scores following a program of treatment; moreover, the
home-based usage results are similar to the clinical usage results. To set up the Neuromove
NM900, two different types of electrodes have to be placed at specific points on the UE. This
system mainly focuses on wrist and finger extension and flexion, elbow flexion and shoulder
subluxation/abduction movements. Furthermore, the values of EMG sensitivity and FES
stimulation have to be set on the device (Neuromove.com, 2011). No other equipment is
needed as there is no specific task other than trying to move the hand muscles. Therefore,
the Neuromove NM900 is relatively easy to set up and does not require much training or
any assistance; however, only one muscle is stimulated at any one time. The NM900 uses
open-loop control with EMG triggering; on the other hand, it does not provide specific tasks
for patients (Neuromove.com, 2011).
The main advantage of the NM900 is that it is an EMG triggered device. Thus, it par-
tially supports the Hebbian learning connection between voluntary movement and support.
However, there is no guarantee that accurate function is produced. In addition, when it
is compared with other stimulation devices, the NM900 is more expensive in that it costs
£4576 (Hanneson, 2000).
Several studies have been conducted on the NM900 (Meilink et al., 2008; Gabr et al., 2005).
These have shown that ETMS increases wrist extension; however, the system places restric-
tions on patients who have limited wrist extension. In one study, the device was used twice
daily for 35 minutes for 8 weeks; and, in the other, it was used during a home exercise
program on 12 stroke patients with hemiparesis in the wrist. Patients who received ETMS
gained 7 points on the FMA scale following treatment, but the effects were lost when the
patients were switched to the home exercise program and re-examined after 16 weeks. In
particular, They lost 9 points on the FMA score. There were no changes in ARAT scores.
Patients who first received home exercise gained less than one point on the FMA scale during
an 8 week period (Teasell et al., 2009).
There are other home-based ETMS devices available such as Rehabilicare EMS+2, which is
a low-cost version of NM900 and can be used as an alternative for home use (Sullivan and
Hedman, 2004, 2007).
3.5 Pulsecure-Pro
The Pulsecure-Pro (OG GIKEN, Okayama, Japan) is a highly portable FES device. The
Pulsecure-Pro uses closed-loop EMG to determine the amount of stimulation required to aid
movement simultaneously. This termed as “power-assisted”. The use of EMG for sensory
feedback means no task is specified, and an EMG signal may not correctly represent the
intention of movement as it does normally. Furthermore, any delay in the EMG signal affects
the accuracy of the movement. The difference between the Pulsecure-Pro and other ETMS
devices is that EMG detection and FES application are implemented on the same pad. This
prevents EMG delays and the receiving of incorrect signals. Overall, the Pulsecure Pro is
relatively simple to operate, and if used in conjunction with daily tasks it may be engaging.
OG GIKEN Pulsecure-Pro has been investigated in terms of usability by comparing it with
other developed systems (Hara et al., 2008). Figure 3.6 shows the Pulsecure-Pro and its
components (Og-giken.co.jp, 2002).
3.5.1 Functionality
Pulsecure-Pro has two channels of neuromuscular stimulation, which are controlled using
voluntary EMG recorded from targeted muscles using the same electrode pad. Hence, incor-
rect muscle contraction will be averted (Hara et al., 2008). Pulsecure-Pro is used for wrist,
shoulder and elbow muscle stimulation. Some ADL tasks (washing and changing clothes)
and some material tasks (e.g. grasping, holding) have been tested on patients.
3.5.2. Advantages and Disadvantages 29
(a) OG GIKEN Pulse Cure Pro (b) Pulse Cure Pro Parts
Figure 3.6: OG Giken Pulse Cure Pro and parts taken from Og-giken.co.jp (2002)
(1-2) Gel Electrode Cords, (3) Gel Electrode, (4) Gel Pad
The main advantage of this system is that it uses the same pad to sense EMG and apply
FES. It hence provides FES in direct response to EMG data. On the other hand, this device
has additional complexity compared to the other ETMS, and may be too complex for home
use.
Research has been carried out using the Pulsecure-Pro (Kobayashi et al., 1999; Meida, 2000;
Seki et al., 2002; Hara et al., 2008; Kim et al., 2016). In one of these clinical studies, 20
patients were divided into two groups: one received home-based FES and the other received
physical therapy over a 5-month period. The FES group used the Pulsecure-Pro to get
greater muscle contraction. The deltoid (Del), and extensor carpi radialis brevis (ECRB),
extensor indicis proprius (EIP), extensor digitorum communis (EDC), and extensor carpi
radialis longus (ECRL) were targeted. Patients received FES at home over 30 sessions of
60 minutes for approximately 6 days/week. Before and after the FES sessions, the active
range of motion (ROM) of wrist and finger extension and shoulder flexion, the modified
Ashworth scale (MAS), and root mean square (RMS) of ECRL were assessed. The FES
group improved considerably in terms of RMS of ECRL, active ROM of wrist and finger
extension and shoulder flexion, MAS and functional hand tests.
In the most recent of these studies (Kim et al., 2016), task-oriented training (TOT) was
combined with EMG-stimulation and this improved the function of wrist and hand extensors.
The study focuses on hand and wrist extension muscles. Twenty patients were recruited
and divided into two groups: one received only EMG-triggered stimulation and the other
received TOT combined with EMG-triggered stimulation for 5 days/week, for 4 weeks. In
this study, control group was continuously stimulated for 20 minutes and another group
was temporarily stimulated during TOT with EMG-triggered stimulation for 30 minutes.
The results have shown a greater increase in FMA for the group using TOT combined with
EMG-triggered stimulation (Kim et al., 2016).
3.6 Saebo MyoTrac Infiniti
The MyoTrac Infiniti (SAEBO, Charlotte, US) was designed as a biofeedback electrical stim-
ulation (EMG triggered stimulation, and Reciprocal EMG triggered stimulation (RETS))
system. Similar to other ETMS, MyoTrac Infiniti uses EMG to determine the amount of
stimulation required to aid movement. It includes 65 protocols and able to collect and save
biofeedback session. In addition to that, it offers the EMG Triggered Stimulation for both
the upper-limb and lower limb (Taylor et al., 2012).
Figure 3.7: MyoTrac Infiniti taken from Saebo.com (2016)
3.6.1 Functionality
The MyoTrac Infiniti are controlled using voluntary EMG recorded from targeted muscles
and provide stimulation from two channels. The difference between the EMG triggered
program and RETS is that the latter is ideal for patients with increased muscle tone that
have difficulty in reducing muscle activation, as detailed in Figure 3.8. It involves both
the agonist and antagonist muscles and is triggered upon relaxation of muscle instead of
activation.
The main advantage of this system is the possibility to triggering the stimulation of the
desired muscle group once the client deactivates or relaxes the opposing hypertonic muscle
group. These include, for instance front and back anterior deltoid, biceps and triceps, wrist
3.6.3. Clinical and Non-Clinical Evaluation 31
1 2
EMG exceed EMG falls behind

threshold to threshold
activate unit
Stimulation
Stimulation off triggered for wrist
and finger
extensors
Figure 3.8: MyoTrac Infiniti reproduced from Saebo.com (2016)
extensors and flexors. EMG changes resulting from muscle contractions causes stimulation
to be triggered to the opposing weakened muscle group.
Research has been carried out using the MyoTrac Infiniti (Graham, 2013). In this study,
the Myotrac Infiniti was tested on the lower-limb. The only study which uses Myotrac
Infiniti for upper-limb has been reported by Graham (2014) with a passive hand support
(SaeboFLEX, Saebo, USA). However, the device has not been clinically studied with chronic
stroke patients.
3.7 STIWELL Med 4
The STIWELL Med 4 (MED-EL, Germany) is another ETMS which can facilitate the train-
ing of complex movement patterns. It has two EMG measurement channels for biofeedback
therapy, which actively involves the patient in the treatment process (Stiwell, 2016). The
patient receives visual or audible feedback. The use of the STIWELL Med4 is still complex
for the patient due to the complexity of electrode placement.
3.7.1 Functionality
The STIWELL Med4 is a more sophisticated device that allows stimulation of up to four
muscle groups and comes with two EMG-channels for biofeedback and triggering. The ther-
apist can also programme the unit’s second EMG channel to monitor the antagonist EMG
activity and only deliver the current if the EMG activity of the antagonist remains below
the prescribed threshold. This helps train complex functional tasks that require multi-joint
movements and at the same time training relaxation of antagonist activity and inhibition
Figure 3.9: Stiwell Med4 taken from Stiwell (2016)
of compensatory actions. The main programmes of the STIWELL Med 4 are conventional
electrotherapy, functional movements, biofeedback games, EMG-triggered stimulation, pe-
ripheral lesions and pareses.
Biofeedback training programmes allow symptom-oriented (spasm, muscle degeneration, co-

ordination problems) biofeedback training for central innervation ability, maximum strength,
strength endurance, muscle relaxation and intermuscular coordination. EMG-triggered stim-
ulation combines biofeedback training and stimulation.
The main advantage of this system is that it can stimulate 4 muscle groups with 2 EMG-
channels which provide a variety potential for practising of tasks as it can stimulate several
upper-limb muscles. It provides FES in direct response to EMG data. In addition to that,
it can motivate patients with games and provide visual feedback. On the other hand, this
device has additional complexity compared to the other ETMS, and may be too complex as
it has multiple EMG and FES channels.
The usability of Stiwell med 4 as a home-based FES device has been proven (Hof Rehabil-
itationszentrum Weißer, 2007). However, recent trials with 159 stroke participants aimed
to identify impacts of rehabilitating finger extension via ETMS with the STIWELL Med 4
(Kwakkel et al., 2016). Unfortunately, there is no evidence found applying 1 hour stimula-
tion with ETMS for 3 weeks without voluntary finger extension benefits participants within
3.8. ShefStim 33
the first 6 months after stroke. This is one of the drawbacks of the ETMSs which UE are
limited to patients with some voluntary motor control (Kwakkel et al., 2003; Langhorne
et al., 2009).
3.8 ShefStim
The ShefStim is a 64 channel device which can facilitate a finer movement pattern by using
an array electrode. It is an open-loop stimulator. The pattern may be set using a comma
separated variable from a log file or directly defined using a graphical user interface.
Figure 3.10: ShefStim taken from Heller et al. (2013)
3.8.1 Functionality
The ShefStim is an open-loop device that allows stimulation of up to 64 channels. This means
it is capable of providing ES to multiple muscle groups with a higher precision through use an
electrode-array (see Figure 3.11). The patient or therapist can also programme the device’s
stimulation sites manually to a predefined pattern from the GUI.
The main advantage of this system is that it may stimulate multiple muscle groups precisely
with its 64 channels. It provides FES in a predefined pattern. To the author’s knowledge,
it has never been tested as an FES device for the upper-limb; instead, it has been only used
Figure 3.11: ShefStim array reproduced from Heller et al. (2013)
as a transcutaneous electrical sensory stimulation (TESS) upper-limb device. On the other

hand, this device is open loop and does not provide any feedback during stimulation.
The ShefStim has been clinically studied with chronic stroke patients for both upper (Mufti
et al., 2016; Slovak et al., 2016) and lower limb rehabilitation (Reeves et al., 2010; Heller
et al., 2013; Prenton et al., 2014; Kenney et al., 2016). In lower limb studies, ShefStim was
used as an FES walking aid in trials which investigated improvements of walking speed and
usability. With the upper-limb studies, ShefStim has been used as a TESS device which
stimulates transcutaneous nerves. However, no evidence exists for its usability as an FES
device for the upper-limb.
3.9 Comparison and Conclusions
The difficulty in transferring from a lab to a home setting is reflected by the lack of any
effective home FES technology. In Table 3.2, the current home-based devices are compared
according to their cost, usability, engagement, scope and effectiveness. Without a doubt,
upper limb treatments and therapies should support precise functional motion (Johnson,
2006). A major disadvantage of current home-based systems is that they do not use accurate
controllers, instead employing simple open- or closed-loop controllers. Open-loop controlled
systems mainly work with switches, whereas closed-loop controlled systems use EMG or
other triggering methods in order to adjust stimulation. An accurate controller is vital to
reduce the effect of disturbances and increase accuracy (Lynch and Popovic, 2008). Using
such a controller, the system can assist participants in performing complex tasks. Moreover,
the control problem is heavily coupled with the sensing problem. The controller needs
feedback-feedforward data from sensors to determine the position and stimulation level
in order to support voluntary intention. Thus, the sensors and controllers are symbiotic.
Another relevant limitation is that there is no device which stimulates all upper-limb muscles
to provide more complete assistance of ADLs.
3.10. Summary 35
3.10 Summary
This chapter has investigated available home-based system in knowledge related to their
cost, usability, engagement, scope and effectiveness. It also highlights their advantages and
disadvantages. In the next chapter, the lab-based iterative learning controlled functional
electrical stimulation system’s benefits for the future development of home-based systems
will be discussed (Exell et al., 2013). In light of the research done on accurate feedback
methods, feedback using cheap non-contact sensors is pivotal to develop more accurate
controllers that could possibly be used for home-based rehabilitation systems in the future.
Ch. 3. Home-Based FES Technology
Table 3.2: Comparison of home-based devices

System Cost(£) Usability Engagement Scope Study outcome results
BION N/A Need surgical incision Computer based Aim to work with multiple Better FMA than ARAT
and good control for multiple tasks according muscle groups and can be useful scores, better for
providing stimulation to stimulator position with future wireless charging networks <1 year post stroke
Rehabilicare EMS +2 770 £ Relatively simple Manual and Can be used for different muscles Effective ARAT score (from 27/57
Focus on specific muscles monotonous, can be use However it only has one channel to 42/57) MAS results
in different tasks
Bioness H200 4,770 £ Simple Manual and can be Focus on different muscles No standardized test,
May need botox used in different tasks FDS,EPB,FPL, consumer showed improvement in tasks
before using EDC and thenar muscles
Neuromove NM900 4,576 £ Relatively Simple Manual, repetitive task It has three pad No rehabilitative
and user friendly defined positions, it focuses on improvement, increase active
increasing muscle strength wrist extension
STIWELL Med4 3,787 £ Simple but pad placement Manual, repetitive task Target upper-limb Improved ARAT (from 5.54 to 15.85) and
is complicated with motivating games rehabilitation, it has 4 channels FMA-UE (from 14.98 to 25.04) scores.
triggered by two EMG signals.
OG Giken N/A Simple but pad placement Manual, repetitive task Targeted upper and Significant improvement with FMA ,
Pulsecure Pro is complicated it focus on power assistance lower limbs. It uses same pads to and Jebsen-Taylor Hand
collect EMG signals and stimulate muscles. Function Test, Box and Block Test
Microstim 2v2 200 £ Simple and very easy to use Manual, repetitive task Target upper and lower Improved ARAT and
limbs rehabilitation with very Jebsen-Taylor functionality
low effectiveness. (button triggered)
MyoTrac Infiniti 1,731 £ Simple but pad placement Manual, repetitive tasks Targeted upper-limb It has not tested
is complicated with reciprocal EMG especially hand and wrist in any clinical study
triggering
ShefStim N/A Simple and uses Manual repetitive tasks Targeted for upper and lower limbs It has not tested
electrode array to stimulate automatic stimulation pattern muscles as an FES device in clinical studies on
for identification but mainly used as a TENS it is usually used as TENS
device device for upper-limb, but
can be used as an FES device
36
Chapter 4
Goal-Oriented Stroke
Rehabilitation Utilising Electrical
Stimulation
In the previous chapter a summary of home-based FES technology was presented, and limita-
tions were highlighted. In this chapter, a system is developed that incorporates components
that are capable of addressing these limitations.
4.1 Introduction
Chapter 2 highlighted that one of the most successful approaches in upper-limb rehabilitation
was ILC, which is one of the only model-based FES approaches to be employed clinically
within an intervention. As detailed in Meadmore et al. (2012), it has been shown to achieve
a high level of scope and performance. This system, termed SAIL, used a VR task display
together with an exoskeleton support, and employed ILC to update the stimulation applied
to the triceps and anterior deltoid. In this chapter, substantial modifications are made to
SAIL in order to both increase scope of function and progress it further towards home-based
deployment.
4.2 System Overview
The SAIL system showed that precise FES control led to more accurate and effective re-
habilitation results than other techniques in terms of FMA, tracking accuracy and ROM
test results (Tong, 2012). However, improvements were restricted to those tasks trained, as
reflected by ARAT results which were not statistically significant (while the FMA results
37
38 Ch. 4. Goal-Oriented Stroke Rehabilitation Utilising Electrical Stimulation
were). In addition, the use of VR tasks may introduce a cognitive barrier. The system is
also far from portable and highly expensive. The new system addresses these limitations by
embedding the following features;
1. Adding wrist extension capability to the existing triceps and anterior deltoid stimula-
tion function to support more functional ADLs;
2. Presenting real-world tasks instead of predefined trajectories;
3. Exchanging the exoskeleton support for a low cost end effector alternative;
4. Adding non-contact position sensors to provide joint angle information. These com-
prise not only significant additions in terms of hardware but also fundamental modi-
fications to the dynamic model and control system.
The system named Goal-Oriented Stimulation Assistance through Iterative Learning (GO-
SAIL) and is shown in Figure 4.1.
Figure 4.1: The GO-SAIL system components
(1) Workstation; (2) SaeboMAS gravitational arm support; (3) surface electrodes; (4) Kinect v1; (5)
electrogoniometer; (6) control algorithm hardware and software; (7) operator monitor displaying the GO-SAIL GUI;
(8) stop button.
4.2.1. Task Design 39
The GO-SAIL system uses iterative learning control (ILC) to precisely adjust the FES over
repeated trials. It comprises a graphical user interface, hardware, controller and software.
It assists real-world tasks that require the manipulation of objects using the hand and arm
chosen to correspond with tasks of daily living. Support against gravity is provided for
the participant’s arm using a commercially available passive spring support. Tracking of
the arm and hand movements is achieved using a Kinect v1 (Microsoft, Washington, USA)
motion capture device and wrist electrogoniometer (Biometrics Ltd, Newport, UK). Since
no explicit reference trajectory is shown to the patient, the ILC scheme employs principles
from motor control to deliver the FES assistance. In this section, the components of the
GO-SAIL system will be described and justified in detail.
4.2.1 Task Design
Functional reach and manipulation tasks that are typically performed in everyday life were
designed to offer a range of reaching challenges across the workspace. There are 5 main
tasks; closing a drawer, switching on a light switch, stabilising an object, button pressing
and repositioning an object. The light switch is located at two different heights, and there
are four positions in which the buttons can be located or objects repositioned both in the
sagittal plane and towards the frontal plane (45◦ across the body, 45◦ to the hemiplegic
side or in line with the shoulder). The objects are placed at different percentages of arm
length (60%, 75%, 80% and 95%) from the participant’s glenohumeral joint, as illustrated in
Figure 4.2. The coloured circles denote the location of the far reach, ipsilateral and contra-
lateral reaching tasks. The table is positioned at a distance of 45% of arm length away from
the glenohumeral joint and 350 mm below the arm when the arm is held horizontal to the
shoulder.
4.2.2 Software
The system software is shown schematically in Figure 4.3. It connects the various hardware
components that are described in Section 4.2.3, and is a combination of two different ele-
ments. The first is a modified version of the Microsoft Skeletal Viewer Software Development
Kit (SDK). This is necessary to interface with the Kinect sensor and has been developed in
Visual Studio 2010 to obtain the upper-limb data. The second software is MATLAB (MAT-
LAB, MathWorks Inc., USA), which implements the controller and interfaces with dSPACE
(dSPACE, dSPACE GmbH, Paderborn, DE). In this section, the MATLAB graphical user
interface (GUI) and Microsoft Skeletal Viewer software will be detailed.
95%
80%
75%
60%
Figure 4.2: Task positions (top view)
A personalised workstation template to standardise the reaching tasks for each participant according to the arm
length. Five main tasks include; closing a drawer, switching on a light switch (high and low) stabilising an object,
pressing a button and repositioning an object. The green button is placed at 60% of arm length.
Graphical User Interface
A GUI has been developed to oversee the system inputs and outputs and is responsible for
customising control parameters, implementing the FES control, collecting position outcome
data, selecting the task details to be performed and reviewing performance after each session.
The GUI window is sectioned into six parts comprising the initialization screen, profile page,
Kinect workspace, stimulation, test page and results screen. Further details of these screens
is contained in Appendix B. The initialization screen contains a password to embed data
security. On the profile screen, the patients’ personal data and kinematic parameters are
entered.
The Kinect workspace screen draws participant’s arm using the measured Kinect data.
Thus, the physiotherapist is able to check if the sensor is functioning accurately or whether
repositioning is necessary. Then, it passes data to the electrical stimulation page, which
initialises procedures for defining the threshold values of the three targeted muscles (anterior
deltoid, triceps and wrist extensor). After defining the threshold values, a procedure for
identifying the axis about which the elbow rotates, which is needed in the kinematic model.
The test page then allows the user to select treatment modalities.
4.2.2. Software 41
Microsoft Kinect®
Sensor
Mathematical Kinect Interface

Software (Visual C++)
Environment
Direct
Data Hardware
Processing Access Instrumented
Task Objects
Digital I/O
Real-time Processor
for Physiotherapist
Real-time
Interface
(933MHz)
De-weighting
ADC Spring Support
Wrist Goniometer
PWM
FES Module
Figure 4.3: GO-SAIL software flow diagram.
Skeletal Viewer
Using the Skeletal Viewer Application window, the physiotherapist can observe the patient’s
movements, especially the raw arm joint positions as seen in Figure 4.4. This application is a
modified implementation of the standard SDK and contains additional code which interfaces
with the dSPACE via direct hardware access. Note that this software also provides lower
limb joint position which can be used for future system extensions.
Figure 4.4: Skeletal viewer main screen showing fitted joint locations.
4.2.3 Hardware
The major GO-SAIL system components are now described and comprise the depth sensor,
the real-time controller, the FES module and the task display hardware.
Motion Tracking
The recent release of Kinect has revolutionised non-contact motion capture by providing a
free software development kit and pre-calibrated out of the box hardware, which has vastly
reduced the associated hardware and software cost (Lange et al., 2011). Kinect is a small
(0.30 x 0.08 x 0.06 m3 ), lightweight (1.4 kg) device incorporating a video camera with an
infra-red (IR) source and an IR sensor as shown in Figure 4.5. The IR sensor measures
the reflection of IR light by objects in front of the camera and calculates 3D position data
of those objects. Moreover, recent accuracy tests have indicated that the device is capable
of calculating position data with an accuracy of around 10 mm (Clark et al., 2012). The
software supplied with Kinect uses pattern recognition to detect landmarks of interest, such
as limb segments and joint estimations. In this system, Kinect is used to capture joint centre
locations for the shoulder, elbow and wrist, which are needed for the calculation of shoulder
and elbow joint angles. However, it is not possible to use the Kinect for the calculation of
wrist and hand joint angles due to accuracy limitations in hand tracking when the device
is placed at the distance required to produce a large enough field of view for all the other
segments. At the time of GO-SAIL development, no reliable non-contact hand/wrist sensing
technologies existed. Therefore, an electrogoniometer was implemented within the system.
The goniometer (Biometrics Ltd., UK) comprises dual axis strain gauges, which can detect
wrist posture with 2 channels. The measured angles are flexion/extension and abduction/ad-
duction, with both having ±0.1◦ accuracy (Christensen, 1999)
Figure 4.5: Kinect for Windows taken from Microsoft (2012a)
Real-time Controller
The real-time control platform was selected as dSPACE ds1103 since it provides an extremely
rapid prototype environment which is programmed in MATLAB and therefore directly in-
terfaces with the GUI. This platform was also used in SAIL and hence is a reliable tool with
which to confirm usability.
4.2.4. Control System 43
FES Parameters
FES surface electrodes are positioned on the patient’s anterior deltoid, triceps and wrist and
hand extensor muscles. As seen from Figure 4.6, a series of 5 V, 40 Hz pulses are produced by
the control hardware for each channel and amplified by a four-channel electrical stimulator
(Odstock Medical Limited, Salisbury, UK) to generate a biphasic signal which achieves a
smooth muscle contraction (De Kroon et al., 2002). The stimulator is voltage driven in
order to avoid discomfort if the electrode contact area reduces. This is due to the current
density remaining constant if the electrode contact area reduces. For safety, the maximum
pulsewidth that can be applied to any channel is limited within the control software and also
by the stimulator. Prior to each treatment session, the amplification level for each channel
is set by applying a constant stimulation signal with maximum pulsewidth from the control
hardware and slowly increasing the voltage until the maximum comfortable level is reached.
Stimulator input signal

Vstim= 5 V
tstim=1/40s
Real-Time Stimulator FES signal

Hardware
pulse width=u
Figure 4.6: Stimulation input from real-time hardware to stimulator.
4.2.4 Control System
The GO-SAIL system employs biomechanical model and controller which are significant
modification to those employed in SAIL system which was detailed in Freeman et al. (2012).
In this section, the developed biomechanical model and extended control strategy will be
discussed.
Biomechanical Model
Position values for the shoulder, elbow and wrist joint centres are calculated using the
Kinect. To assist the FES control scheme, a simplified model of the arm is used for the
calculation of joint angles. Figure 4.7 shows the kinematic model of the human arm where
Φ = [φ1 , ...φ7 ]T contains joint angles of the human arm, and Θ = [θ1 , ...θ5 ]T contains joint
angles of the SaeboMAS. Spasticity in stroke patients often restricts flexion of the shoulder in
the anteroposterior plane, extension of the elbow and extension of the wrist and fingers. This
motivated the selection of the anterior deltoid, triceps and wrist and hand extensor muscles
φ1
θ1 θ2
φ2
φ3
θ4
θ3
φ4
φ5
φ7
θ5 φ6
(a) SaeboMAS (b) Human arm
Figure 4.7: Kinematic model of SaeboMAS and human arm
for stimulation. It is assumed that stimulation applied to the triceps produces movement
about an axis perpendicular to the upper and forearm segments and that stimulation applied
to the wrist and hand extensors produces movement about an axes that is fixed with respect
to the forearm. For the anterior deltoid it is assumed that stimulation produces movement
about an axis that is fixed with respect to the shoulder and determined by two rotation
transformations. These comprise rotations around the z -axis by β and γ around the x -axis.
The dynamic model of the human arm is given in the simplest case (excluding a full model
of the hand) by
Bh (Φ)Φ̈ + Ch (Φ, Φ̇)Φ̇ + Fh (Φ, Φ̇) + Gh (Φ) = g(u, Φ, Φ̇) − J T (Φ)hh (4.1)
where Bh (·) and Ch (·) are 7-by-7 inertial and Coriolis matrices respectively, Fh (·) and Gh (·)
are friction and gravitational vectors. Here hh is a vector of externally applied force and
torque comprising components hs due to the spring support and h due to interaction with
objects; matrix J(·) is the system Jacobian. Moreover u1 (t), u2 (t) and u3 (t) represent the
electrical stimulation applied to the anterior deltoid, triceps and wrist and hand extensor
muscles, respectively with u = [u1 , u2 , u3 ]T . Vector g(·) comprises the moments produced
through application of FES, which are of the form
g(u, Φ, Φ̇) = [g1 (φ1 , φ˙1 , u1 ), 0, 0, g4 (φ4 , φ˙4 , u4 ), 0, g6 (φ6 , φ˙6 , u6 ), 0]T (4.2)
The moment around joint axis is generated by electrical stimulation signal u; can be assumed
to be of the form (Freeman et al., 2011)
gi (φ, φ̇, uj (t)) = hi (uj , t) × Fm,i (φi , φ̇i ) (4.3)
The term, hi (uj , t) is a Hammerstein structure incorporating a static non-linearity, hIRC,i (uj ),
that represents the isometric recruitment curve, cascaded with linear activation dynamics,
hLAD,i (t). The term Fm,i (φi , φ̇i ) models the multiplicative effect of the joint angle and joint
angular velocity on the active torque developed by the muscle. The SaeboMAS support
structure has the form
Bs (Θ)Θ̈ + Cs (Θ, Θ̇)Θ̇ + Fs (Θ, Θ̇) + Gs (Θ) + Ks (Θ) = −JsT (Θ)hs (4.4)
where Bs (·) and Cs (·) are 5-by-5 inertial and Coriolis matrices. Respectively, θ1 . . . θ5 , rep-
resenting the angles of the spring support as shown in Figure 4.7(a). In addition, Js (.) is
the system Jacobian, and Fs (·) and Gs (·) are friction and gravitational vectors. The vec-
tor Ks (.) comprises the moments produced through gravity compensation provided by the
spring, which takes the form [k1 (θ1 ), 0, 0, 0, 0]T . The rigid connection between structures
gives rise to bijective mapping between Φ and Θ so that the combined model is given by
B(Θ)Θ̈ + C(Θ, Θ̇)Θ̇ + F (Θ, Θ̇) + G(Θ) + K(Θ) = −J T (Θ)h (4.5)
This model of the arm is next used by the FES control system to produce an input signal
that results in accurate completion of the tasks.
Control Scheme
An advanced control system utilising constrained point-to-point optimisation has been de-
veloped and experimentally verified to enforce tracking of joint reference Φ̂ (Freeman, 2012).
This novel approach to controlling movement embeds results from human motor control and
poses each task as a constrained optimisation problem, such as moving from one point to
another with constraints applied on the joint velocity and acceleration to ensure a smooth
movement. In the ILC framework, these problems are solved iteratively using joint data
from each attempt to update the control input. This improves upon using predefined ref-
erences for each task since the task is tailored to each patient using their underlying arm
dynamics, and new tasks can be added without requiring predefined references to have been
collected. A limitation of implementing this technique is the requirement of a full dynamic
model, which entails a lengthy experimental procedure. Therefore, to minimise cost and
maximise the usability of the system, a simpler approach termed phase-lead ILC was used
to control this system, based on tracking joint references extracted from predetermined
movement data.
Iterative learning
controller
vk +1 = vk + Lek
v Stimulated muscle dynamics

h Human arm and 3D mechanical support
k
Φ̂ ek Φk
Φk = B(Φk )-1 (τ k - C(Φk ,Φ& k )Φ& k - F(Φk ,Φ& k ) - G(Φk ) - K(Φk ) - J T (Φk )h )
+ uk + τ k &&
+ + C g ( u k ,Φ k , Φ& k ) +
- +
Feedback
controller
Electrically stimulated muscle ü
ý Effect of voluntary action
All other muscles þ
Figure 4.8: Block diagram of the ILC scheme
System (4.5) has three inputs and 7 outputs, however each input corresponds to a single
output and can be assumed to have minimal dynamic coupling during slow supported tasks.
Therefore we assume the controlled arm dynamics are given by the linear forms φ1 (s) =
H1 (s)u1 (s), φ4 (s) = H4 (s)u2 (s), φ6 (s) = H6 (s)u3 (s). Then controller C is selected such that
u1 (s) = K1 (s)(e1 (s) + v1 (s)), u2 (s) = K4 (s)(e4 (s) + v4 (s)) and u3 (s) = K6 (s)(e6 (s) + v6 (s)).
The resultant closed-loop dynamics are
Gi : (φ̂i + vk,i ) 7→ φk,i : φk,i (s) = (I + Ki (s)Hi (s))−1 Hi (s)Ki (s)(φ̂i (s) + vk,i (s)), i = 1, 4, 6.
(4.6)
During trials incorporating FES, the controller assists in the tracking about φ1 , φ4 and
φ6 only, and it is assumed that the patient has sufficient control over the remaining axes
to adequately perform the task. Having stabilised the arm dynamics, an ILC scheme is
implemented in order to provide input vk such that the error is minimised, i.e. lim vk = vk?
k→∞
with vk? := minvk kΦ̂ − Φk k2 . This is achieved through the update structure
vk+1 = vk + Lek , v0 = 0, k = 0, 1, · · · (4.7)
ˆ yields
where L is the ILC operator. Inserting (4.6) into (4.7), together with ek and and φ(t)
relationships between successive error and command signals of respectively
ek+1 = (I − GL)ek , vk+1 = (I − LG)vk + L(I − G)Φ̂.
The design of L to satisfy kI − Gi Li k < 1, i = 1, 4, 6, guarantees convergence of φi to zero

error, and many suitable schemes are available, see (Freeman, 2014) and examples therein.
Since the dynamics are considered to be decoupled, the update (4.7) simplifies to
vk+1,i = vk,i + Lek,i , i = 1, 4, 6. (4.8)
Selection of phaselead ILC then results in
vk+1,i (t) = vk,i (t) + lek,i (t + λ), i = 1, 4, 6. (4.9)
where l is a gain parameter and λ is the time delay employed.

Model Identification
The FES control schemes utilise a dynamic model of the combined human arm and me-
chanical support, so that stimulation results in accurate tracking of the reference profiles
calculated for each task. The dynamic model requires the two parameters, β and γ, which
define the anterior deltoid axis. These parameters are determined by applying a ramped
FES signal for 10 s to the anterior deltoid and recording the associated movement of the pa-
tient’s elbow. It is assumed that the spring support cancels the effect of gravity so that the
stimulation applied around the shoulder only produces movement about the anterior deltoid
axis. A plane is fitted to the elbow positions that were collected whilst the stimulation was
applied, which is then used to determine β and γ (see further details in Appendix A). Note
that, in the event that a full dynamic model is replaced by an approximation, a kinematic
model is still vital in order to compute the joint angles from the measured joint positions.
z
y
x
Figure 4.9: Anterior deltoid identification with parameters γ, β.
4.3 Experimental Evaluation
Following ethical approval (EP/G014078/1 and FoHS ETHICS-4009), 5 stroke participants

were recruited with demographic characteristics is given in Table 4.1. Use of the system
during the clinical intervention was as follows: each patient attended 18 x 1 hour sessions
during which the system assists them in performing functional reaching and grasping tasks.
All interventions were completed in 6-8 weeks to accommodate missed sessions. Prior to
each session, participants were positioned at the workstation and the arm being tested
was secured in the SaeboMAS. A physiotherapist placed surface electrodes at the required
positions of the patient’s extensor muscles of the shoulder, elbow, wrist and hand. The arm
support was adjusted to facilitate free range of movement either volitionally or when FES
was applied without the arm being lifted too high causing abnormal posture and allowing
the hand to rest easily on the table top (see Figure 4.1). The physiotherapist used a hand-
held electrical stimulator (MicroStim 2v2, Odstock Medical Limited) when positioning the
electrodes so that they are placed at the selected muscle locations to produce the desired
movement. Maximum stimulation levels were identified for all muscles and used as an upper
limit for participant comfort and safety. Parameters necessary for the model of the arm
were also identified. Timing of these stages are detailed schematically for both engineering
and clinical side of team in the Figure 4.10.
Table 4.1: Participant demographics

Participant Age Gender Time since Side of Behavioural Inattention
ID (years) (months) Lesion Test
Line Star Letter
(36)+ (54)+ (40)+
1 53 M 22 Right 36 53 40
2 42 M 84 Right 36 54 40
3 49 M 52 Right 36 54 40
4 46 F 48 Right 36 54 36
5 48 M 84 Left 36 53 40
Mean (SD) 47.6 (4.04) 58 (26.38) 36 (0) 53.6 (.54) 39.2 (1.79)
Note: The BIT assesses for visual neglect and inattention. Clinical cut-offs for inattention are 34, 51 and 32 for the
line, star and letter cancellation tasks, respectively; + numbers in brackets = maximum score.
4.3.1 Intervention Safety Management
The health, safety and welfare of each participant during the intervention was a primary
concern. The physiotherapist systematically and carefully documented information relating
to application of eligibility criteria, refusals, withdrawals, intervention, loss to follow up,
acceptance of the intervention by the participants and problems or difficulties relating to
the performance criteria used. Potential adverse effects such discomfort or fatigue during the
intervention were monitored throughout the course of the study. If the participant had any
symptoms of discomfort or distress during the trial, the intervention could be terminated
4.3.2. Intervention Design 49
immediately and the symptoms assessed by the physiotherapist (Emma Hallewell) in the
first instance or by a member of the supervisory research team. The experimental officer
allocated to the Biomechanics Lab had first aid certificate and could provide additional
assistance if it was required. The participant was never be left alone. Two helpline numbers
were also be made available for participants in case of distress. Note that these measures
were never actually required.
A variety of other health and safety measures were in place. These were similar to those
used in the SAIL project (S07/04-01 and FoHS Ethics-2010-30). They included:
Software safety measures:
1. Rate limiters: Rate limiters were implemented in order to avoid rapid increase in
stimulation.
2. Cut-off switches: Switches were implemented to automatically stop the stimulation

when participants completed each task.
3. Tasks and stimulation: As has been described, the tasks were selected to be achievable,
and a procedure was applied to set comfortable stimulation limits.
Hardware safety measures:
1. Stimulator: The Odstock stimulator was battery-powered, CE marked, and optically

isolated from all electrical equipment,
2. Gravitational assistance: The SaeboMAS support was CE marked,
3. Pulsewidth: The Odstock stimulator had an in-built maximum level of pulsewidth (350
microseconds) and was current controlled to ensure a comfortable level of stimulation
even when the area of the electrode reduced.
4. Stop button: An emergency stop button was implemented to allow the physiotherapist
to stop stimulation immediately at any time.
4.3.2 Intervention Design
At the beginning and end of each intervention session, four button pushing tasks are per-
formed without any FES being applied. The patient’s performance during the unassisted
button pushing tasks was used to monitor their arm function over time. Following the
unassisted tasks, the physiotherapist selected the tasks to be performed with stimulation,
based on the specific rehabilitation requirements of each patient. Each task was typically
performed 6 times with 20 seconds rest time between each trial. Immediately following each
trial and before the next attempt, the magnitude and timing of the FES applied to each
muscle group was updated by the ILC scheme, based on the error recorded during the pre-
ceding attempt. Sensors recorded within the task objects were used to detect if the patient
had completed the task, at which point the FES was terminated for all muscle groups so
that the patient could return their hand to the starting position. The physiotherapist also
had a safety button that they can press to immediately stop the stimulation during a trial.
The custom graphical user interface described in Section 4.2.2 was used by the therapist to
perform the subsequent tests. During training, the therapist selected the tasks to be trained,
according to the rehabilitation needs of each individual. Tasks were chosen to challenge the
participant but so that completion was not unrealistic. Each task was typically repeated 6
times. Participants always started each task with their hand resting on the red square in
front of their shoulder (see Figure 4.11). During each task, FES was applied to the anterior
deltoid, triceps and wrist extensor muscles in order to assist performance of the movement.
Participants were instructed to initiate the activity and try to move their arm to complete
the task themselves. A key role of the therapist was to provide verbal encouragement; mo-
tivational feedback was available in the form of the number of successful tasks completed
out of each set of six, then reducing the level of support needed from the SaeboMAS and
the percentage of available stimulation used in each task. The FES was mediated by ILC
algorithm (4.9) to facilitate the movement of the arm over the six repetitions of the selected
task. At the beginning and the end of each session, participants also completed five unas-
sisted tasks: four button pushing tasks and one light switch task (at 75% of reach at the
highest location). The unassisted tasks consisted of one trial only. Joint angles, timings
and error magnitudes between the participant’s arm movement and the reference movement
were recorded for each task. These provided a measure of accuracy for each muscle group
for unassisted tasks (i.e., movements without FES) and assisted tasks as seen from Figure
4.12. This illustrates the mechanism by which ILC functions: the stimulation on the first
trial is due to feedback control alone, and typically lags the reference trajectory. The ILC
then adds a correction term which causes stimulation to be applied earlier in time, as it
anticipates the subsequent error. As the ILC trial number increases, the amplitude of the
stimulation signal increases.
4.3.3 Intervention Results
One participant was excluded from this thesis due to a deviation from the protocol (whereby
the number of sessions attended each week and amount of time spent exercising in the
sessions was consistently not met). Data are reported for five participants aged between 42
and 54 (four males, one female). All participants had suffered strokes between 22 months
and 7 years prior to recruitment to the study; four have left hemiplegia and one right
hemiplegia. None have visual neglect or visual field deficits. All five participants complied
4.3.3. Intervention Results 51
GOSAIL
Engineering Side Clinical Side
Initial Stage (10-15 min) Initial Stage (10-15 min)

- Initialise PC, sensors and Real-Time hardware -Place participant to the stall and talk them for
(3 min) comforting (3 min)
- Initialise software (GUI) (2 min) - Check positions of muscle belly of AD, Triceps
- Check stimulator connections (2 min) and hand and wrist extensors for electrode
- If electrode positioned correctly and placement via SENIAM guide (2 min)
participant ready guide physiotherapist which - Place electrodes and check if the placement
channel corresponds which muscle. (2 min) correct via hand held stimulator (2 min)
-Check maximum comfortable level of - Connect OdStock stimulator to corresponding
stimulation for each channel (2 min) channels with guidance (2 min)
- Anterior Deltoid Identification (1 min) - Anterior Deltoid Identification (1 min)
Experimental Evaluation (40-50 min) Experimental Evaluation (40-50 min)

- Record participant unassisted tasks in the - Check participant movements during unassisted
beginning of session tasks observe and take notes.
-Check if the collected data is meaningful - Task selections according to participant
- Set the GUI according to physiotherapist task performance in previous trials, fatigue level and
selection. feeling. Motivate the participant while give him/her
- Start test when participant and physiotherapist are a challenge.
ready. - Order to start test when participant ready.
-Check if the collected data is meaningful and -Check if the participant has enough challenge
electrical stimulation is providing support with during tests.
iteration. -Adjust support level if the participant successfully
-Report back physiotherapist if the gravitational finishes the trial with less effort.
support level is required to be adjusted. - Check the participant movements during
- Record participant unassisted tasks in the end of unassisted tasks in the end of session
session
-Check if the collected data is meaningful and saved
properly
Figure 4.10: GO-SAIL session flow diagram.
with the study protocol, and there was no withdrawal. Participants reported no intervention
adverse effects.
Figure 4.11: GO-SAIL intervention - contralateral button push
Clinical Outcome Measure
The scores from the two clinical outcome measures are shown in Table 4.2. Improvements
were seen in scores; in four participants for the FMA and for all participants for the ARAT.
This improvement was statistically significant for both FMA (t(4) = 2.44, p = .036) and
ARAT (t(4) = 4.49, p = .005). Table 4.2 shows assessment scores for the ARAT and FMA
at baseline and post-training sessions.
Table 4.2: Stroke participant clinical assessment data

Action Research Arm Test Fugl-Meyer
Participant ID Base-Line Post- Base-Line Post-
P1 0 7 15 24
P2 3 7 19 24
P3 4 5 17 21
P4 3 8 21 27
P5 3 8 22 20
Mean (SD) 2.6 (1.52) 7 (1.22) 18.8 (2.86) 23.2 (2.77)
ttests t(4) = 2.44, p = .036 t(4) = 4.49, p = .005
Note: ARAT has a range of 0 to 57, and the motor component of the FMA has a range of 0 to 66.
FES-unassisted performance
Table 4.3 shows that significant reductions were found in the time taken to perform both
the button press at 80% of reach and button press at 75% of reach, 45◦ to the impaired side.
4.4. Discussion 53
Table 4.3: Regression slopes and p-values for FES-unassisted tasks over the 18 sessions
Task Slope t-value p-value
End hand position: Button at 80% 25.62 2.61 .01
End hand position: Button on impaired side 12.08 1.47 .15
Time taken: Button at 80% -.38 2.44 .02
Time taken: Button on impaired side -.29 2.17 .03
Time taken: High light switch .55 5.37 .00
Maximum Extension: High light switch -.08 3.51 .001
In addition, the end position of the hand from the participant in terms of distance in the
direction of the button were found to increase over the 18 sessions (significantly so for the
far button). Taken together these results indicate that participants became more successful
at reaching these buttons and did so in a shorter time over the course of the 18 sessions (see
Table 4.3).
FES-assisted performance
FES appeared to have improved the performance when compared to no FES was provided
(see Figure 4.12). Furthermore, ILC successfully controlled the amount of FES applied
independently to each muscle, facilitating movement patterns more similar to the reference
trajectories over a series of trials. This is illustrated in Figure 4.12, in which the participant
completes the task more quickly in trial 6 compared to trial 1 and their movement more
closely resembles the ideal reference movement. As shown in Figure 4.3.3, participant’s
gravitational support level decreases while the maximum shoulder extension increases over
the course of the 18 sessions. This means their independent movement skills become better
through out the sessions. Furthermore, the participant’s time to complete tasks increases
over the session as seen in Figure 4.13(b).
4.4 Discussion
The main aims of the study were to investigate the feasibility of precisely controlling FES
to three muscle groups in the upper limb to complete goal-oriented movements to facilitate
functional motor recovery post-stroke. Results demonstrate that advanced model-based
FES controllers were able to independently and precisely control stimulation applied to the
shoulder, elbow and wrist and finger extensors of chronic stroke participants to facilitate
coordinated reach to grasp tasks. Thus ILC mediated FES has been successfully extended
to three muscle groups, confirming a substantial development in the feasibility of using such
technology in this area. Importantly, statistically significant improvement was measured in
four different outcome measures following completion of the intervention: an increase in both
FMA and ARAT clinical assessment scores, an improvement in FES-unassisted performance,
and a reduction in the arm support levels. This translated into a clinically relevant change
in the clinical assessment measures (defined as 10% of the value of the scale) for only one
participant. In addition to measured quantitative outcomes, participant feedback provided
positive qualitative responses.
An important finding from this study is that, in this sample of chronic stroke patients both
the primary outcome measures, FMA and ARAT scores, showed statistically significant
improvements from pre to post intervention. Thus, following the intervention, participants
showed reductions in motor impairment and were able to perform more functional motor
activities. The same intervention period of one hour was used to facilitate comparison with
previous work using ILC mediated FES which showed statistically significant improvements
only in the FMA assessment and not the ARAT (Hughes et al., 2009; Meadmore et al., 2012).
This has been attributed to the fact that in these studies wrist and hand extensors were not
specifically trained, with only the triceps and/or anterior deltoid being stimulated. Indeed,
upper limb treatments and therapies are suggested to be location specific (Johnson, 2006).
Training of the shoulder and elbow will only improve motor impairment in the shoulder and
elbow (Hughes et al., 2009; Meadmore et al., 2012), just as training of the wrist and finger
extensors shows improvements in hand function (Mann et al., 2011). As such, to achieve
functional changes the whole upper limb should be considered in training. This study set
out to address this by incorporating ILC mediated wrist and finger stimulation, and the
results are very promising to the recovery of whole arm functional movements.
Nevertheless, despite observing an improvement and participants reporting greater ability

to perform everyday tasks at home, such as lifting, stabilising and pressing light switches, it
was still evident that fine finger movement was required to optimise transfer of the benefits
observed to activities of daily living. There has been significant interest in using wrist ar-
rays in recent years (Micera et al., 2010), with existing control methods embedding simple
rule-based selection of suitable sites in order to produce the greatest level of appropriate
movement, while minimizing undesired effects (Heller et al., 2013; Popović-Bijelić et al.,
2005). Such approaches have demonstrated the potential to generate selective movement,
but are slow and imprecise since they do not exploit an underlying dynamic model linking
FES and resulting motion. To-date, however, there has been no feasibility studies in a clin-
ical rehabilitation setting. ILC, on the other hand, has been shown to provide more precise
control of hand and wrist movement by employing a model of the hand and wrist, and learn-
ing from past experience (Freeman, 2014). This motivates integrating a model-based array
ILC framework into the current system to produce fine finger movements during training
of everyday tasks. This will extend the theoretical and practical implications for stroke
rehabilitation demonstrated in this study so that the effectiveness of therapy is maximised.
Another important finding that supported the observation that participants motor function
improved over the intervention period was that, as the sessions progressed, the amount
of arm support that participants required to complete the FES-assisted tasks was reduced
4.5. Conclusions 55
(Krabben et al., 2012). Motor learning theory suggests that as skill improves, expectations
relating to the performance increase. Accordingly, to generate a challenge for learning, task
difficulty must increase (Guadagnoli and Lee, 2004). In the current study, not only did
arm support levels provide participants with an indication of performance throughout the
sessions, but it also allowed for the progression of training as shown in Figures 4.13(a) and
4.13(b). Each time participants were able to consistently complete a task the arm support
level was reduced. This served to make the task harder; the ILC algorithms would then
adapt to facilitate performance whilst still encouraging increased effort from participants.
In addition, as participants became more able to complete the task, the ILC controllers would
reduce the amount of stimulation needed. Thus, in this way, training difficulty was managed
in a number of ways to facilitate motor learning, progression and motivation throughout the
intervention. The limitations of the study were lack of precise finger tracking, a small
sample size, no control group or follow-up due to the time constraints. A larger sample of
participants in a randomised controlled trial or cross-over study may be designed in which
the effects of no FES (unweighting from the arm support alone) or FES that is not precisely
controlled by ILC are compared with ILC controlled FES. In addition, as mentioned above,
although stimulation of the wrist extensors helped participants to open their hand, fine finger
movements are required for many functional tasks. On the other hand, the proof-of-principle
has been demonstrated.
4.5 Conclusions
The GO-SAIL system utilises ILC to regulate FES applied to extensors of the shoulder,
elbow, wrist and hand joints for ADL task training. The system uses a Kinect low-cost
non-contact sensing method for collecting real-time kinematic data, which provides reliable
feedback even when used in conjunction with a spring support. The system incorporates
a variety of functional tasks that challenge different aspects of reaching and grasping. The
use of ILC to update the FES applied during trials encourages maximum effort from the
patient whilst the movement is performed, which along with the trial repetitions, can assist
with Hebbian learning. The results are positive and indicate that the GO-SAIL system is
a promising rehabilitation modality in the field of upper extremity rehabilitation in chronic
stroke. Clinical trial results indicate that the system is effective in increasing the move-
ment of patients and controlling the FES levels during functional tasks to assist with the
performance of real world reach and grasp tasks.
This chapter has therefore established the potential of new technologies including non-
contact sensors for increasing the performance and scope of FES systems. However, the
primary drawback of GO-SAIL is the lack of precise hand and wrist support which has
limited the degree to which GO-SAIL supports functional tasks. A key obstacle which must
be addressed in order to tackle this problem is how to accurately sense the hand and wrist
joint positions. For that reason, in the next chapter, some of the available sensor technology
for hand and wrist sensing will be highlighted.
80 300
60
200
4.5. Conclusions
40
100
Shoulder (Φ1)
20
0 0
0 2.5 5 7.5 10 0 2.5 5 7.5 10
100 300
80
200
60
40
Elbow (Φ4)
Joint Angle
100
20
0 0
0 2.5 5 7.5 10 0 2.5 5 7.5 10
Stimulation pulsewidth (microseconds)

40 300
20
200
0
Wrist (Φ6)
100
−20
−40 0
0 2.5 5 7.5 10 0 2.5 5 7.5 10
Time (s) Time (s)
Ref Unassisted T1 T6 T1 FES T6 FES
Figure 4.12: Participant 1 low light switch test results.

57
Ch. 4. Goal-Oriented Stroke Rehabilitation Utilising Electrical Stimulation
(a) Participants gravitational support levels and session numbers. (b) Participants time to complete task levels and session numbers
(c) Participants time to maximum shoulder extensions between session numbers
58
Chapter 5
Sensor Technologies for the Hand

and Wrist
The previous chapter has highlighted the significance of accurately sensing the movement
of the hand for the development of controllers to support functional tasks in a home-based
setting. Low-cost and compact sensing has become more advanced and accessible in the last
decade, largely due to improvements in digital computing. Such technologies has numerous
applications such as in rehabilitation and human-computer interaction. The current chapter
hence reviews available technologies that are appropriate for sensing the position of the
hand and wrist. Possible home-based non-contact and contact sensors will be examined
and compared according to their scope, cost, usability and accuracy. The movements used
for experimental comparison will include opening and closing of the hand, moving between
certain hand positions, and grasping objects. Within these tests it is assumed that the
environment under which the sensor is operating can be controlled to a certain extent in
terms of location and background light.
5.1 Sensing Requirements
For the rehabilitation of limbs, physically attached motion sensors can often be used without
obstructing movement since few sensed variables are involved. However, the movements of
the hand are far more complex. In order to develop a home-based system, low-cost sensors
have to be identified. Sensor requirements are an important issue effecting the scope and
accuracy of any control system. The accompanying software needs to be capable of running
on a simple, low-specification and low-power computer. This is especially important if the
rehabilitation device is to be battery powered.
The human hand consists of 27 bones and 15 major joints, and is actuated by two groups of
muscles that control motion; namely intrinsic muscles inside the hand and extrinsic flexors
and extensors in the forearm that control the bending and extension of the fingers. Various
59
60 Ch. 5. Sensor Technologies for the Hand and Wrist
studies have been carried out using simplified hand model, which show that the hand has
at least 24 degrees of freedom in practice (Cobos et al., 2010). Thus, it is computation-
ally expensive to implement a model which is appropriate for online data analysis. Due
to the complexity of the modelling and subsequent control, the majority of current FES
implementations control only the extrinsic muscle group.
Additional difficulty occurs since the hand of a stroke patient is typically held in a fist posture
that cannot be opened without considerable assistance. A detection system ideally needs to
be able to initially identify a hand in a fist, which is difficult as the distinguishing features of
the hand are obscured in that posture. Furthermore, a high precision sensing unit is required
for accurate and robust detection of the patient’s hand in an uncontrolled environment. Due
to the challenges in gaining accurate feedback, model-based FES controllers have not yet
been implemented in portable or home-based rehabilitation systems (Kowalczewski and
Prochazka, 2011). In the remainder of this chapter, contact and non-contact systems will
be investigated with accompanying software as potential solutions in providing the required
feedback.
5.1.1 Contact Sensors
Contact sensors are defined as sensors or sensor parts which are attached to a human limb
to collect the intended data. According to the technology used, they can be divided into: IR
marker based, accelerometer-based, fiber-optic, magnetic, and resistive or image processing-
based systems.
Infrared Marker Systems

Infrared marker systems use high quality IR cameras to detect markers placed on the body.
A leading example of this kind of system is Vicon (Vicon Motion Systems Ltd, Oxford, UK),
which is the most accurate and expensive example of this type (see Figure 5.1). The main
Figure 5.1: Outputs of Vicon system taken from Vicon.com (2010)

5.1.1. Contact Sensors 61
idea behind this system is to use infrared markers placed on the body, which are viewed
by high quality infrared cameras. The information is then processed to extract the marker
locations. In the stroke rehabilitation domain results have confirmed Vicon’s effectiveness by
comparing it with electrogoniometer results (Pomeroy et al., 2006). However, the expensive
price of the system makes it unsuitable for use in a low-cost home-based rehabilitation
system. Nevertheless, there are other alternatives available for this kind of system.
Electromyography Data
Gesture control is one of the most exciting evolutions of human-computer interaction and has
been replacing traditional methods such as fiber-optic-, image processing or accelerometer
based techniques. Electromyography data technology is a prime example of this technology,
and is based on processing EMG signals from different muscles. instead of using camera
data directly. It estimates muscle activity and the corresponding motion of the user, and
can detect changes down to each individual finger. When tracking the position of the arm
and hand, it can detect subtle movements and rotations in all directions according to the
position of the hand and the gesture of the arm. A leading example of this kind of device is
the MYO (see Figure 5.2) produced by Thalmic Labs Inc., Kitchener, Canada, which is one
of several new companies working on gesture control using EMG sensors. The device has
an expandable design to fit forearm circumferences between 190 mm and 340 mm, with 10
mm thickness. This system may be suitable for stroke rehabilitation if enough EMG signals
can be extracted from the patient’s muscles (Getmyo, 2013).
Figure 5.2: MYO taken from Getmyo (2013)

Image Processing
Image processing systems work in the same way as IR-based non-contact sensors with signal
processing. A leading example of this kind of device is Digits (Microsoft), which was devel-
oped by the Microsoft Research Team for controlling computers by using hand gestures (see
Figure 5.3). It has been shown that freehand armband devices can detect the position of
fingers with infrared emissions and inverse kinematic data (Kim et al., 2012). This device
is inspired by the Kinect and is intended to replace data gloves. The device is currently
not released due to the recent improvements of non-contact sensors which have offered sim-
ilar function. Nevertheless, Digits has the potential of being used in stroke rehabilitation
systems.
- IMU
~- IR diffuse LED
1""2--- 1R camera
IR laser line projector
Figure 5.3: Freehand with wearable sensor taken from Kim et al. (2012)
Data Glove Sensing
Glove sensing is used to define hand gestures by detecting finger flexion and abduction
using fiber-optic or resistance cables (Dipietro et al., 2008). Measurements are converted
to finger and wrist angles using data with examples including 5dt (2010) and Cyberglove
systems (2012). Gloves generally detect movements with high speed and accuracy; however,
without any other sensor data, they do not directly provide the absolute hand position in
space. Despite this, data gloves can provide accurate finger and wrist joint angle feedback
for hand rehabilitation. Unfortunately, they are hard to use for stroke rehabilitation because
of spasticity and stiffness, as well as very difficult to don/doff which results in a longer set-up
time. Furthermore, gloves are known to require calibration procedures which makes them
impractical to use at home (see Figure 5.4).
5.1.1. Contact Sensors 63
(a) 5DT Glove (b) Cyberglove
Figure 5.4: Data Gloves taken from 5dt (2010) and Cyberglove systems (2012)
Electro Goniometers
Digital Goniometers are one of the most accurate solutions for detecting joint angles. A
leading example of this kind of device is manufactured by Biometrics (see Figure 5.5). These
systems work with a strain gauge transducer and have been used in various research fields,
including stroke rehabilitation. Exell et al. (2013) used a Biometrics digital goniometer for
measuring the pitch angle of the wrist to provide accurate feedback. However, a digital
goniometer is not suitable for a home-based system because of its cost and since it provides
only pitch and yaw angle values.
Figure 5.5: Goniometers taken from Biometrics (2012)

Comparison of Contact Sensors
In Table 5.1, contact sensors are compared according to their price and speciality. This
illustrates that contact sensors can be very expensive, and commonly also entail lengthy
set-up or calibration times. Thus, they are not generally suitable for a low-cost home-based
system. Some of them also have a don/doff problem which creates an obstacle for home-
based application. Storing the sensors is another issue because of the sensitive design of the
fiber-optic sensing apparatus. Nevertheless, electromyographic data and image processing
have significant potential for use as contact sensors.
Table 5.1: Leading examples of contact sensors

Device Price Speciality
Vicon £23,077 Excellent accuracy and scope, lengthy

set-up and high expense
Goniometer £1,154 Data are accurate however when compared with other
sensors it does not give enough data for creating a full model
5DT Data Glove £765 Good feedback with joint angles of all fingers and
hand. However, reliable position data are not available
Myoget £154 EMG-based and useful for muscle control,

however generally inaccurate
5.1.2 Non-Contact Sensors
Non-contact sensors have recently made breakthroughs which make them candidates for
use in upper-limb stroke rehabilitation. The first major example of this technology was
the Microsoft Kinect. It uses a camera and a depth sensor to track a human body for
input to the Xbox gaming console. In 2011 Microsoft released an integrated development
environment (IDE) to allow official third party development using the Kinect sensor. Using
IDE, many applications were developed to enable the Kinect to provide a gesture control
interface for specific applications. In the light of this innovation, new non-contact devices
quickly started being developed for upper-limb stroke rehabilitation. Therefore, the future
of hand rehabilitation is likely to see expensive contact sensors being replaced with low-cost
non-contact sensors.
5.1.2. Non-Contact Sensors 65
3D Sensor Technologies
There are multiple methods for extracting 3D data from a scene as shown in Figure 5.6.
It can be done with multiple 2D cameras achieve stereoscopy, or it can be achieved with
specific depth cameras to create static models of the scene via depth perception. The two
main manufacturers of depth sensing cameras use depth sensors. Each, however, use different
technologies in order to achieve this, as discussed next.
1 2 3
Figure 5.6: 3D sensor technologies

1) time of flight, 2) structured light 3) stereoscopy cameras
Time-of-Flight
Time-of-Flight (ToF) cameras use the known speed of light to calculate the distance between
the camera and the object. ToF cameras are considered to be scanner-less LIDAR (Light
Detection and Ranging) systems since the entire scene is captured in one light pulse rather
than by a scanning laser.
ToF cameras use a very short pulse of infrared light, typically provided by a diffused laser
to illuminate the scene. During a very short and specific time interval after this pulse, the
sensor takes a series of images of the scene. The intensity of the infrared light across the
series of images is then used to calculate the distance it has travelled, with reflections from
closer objects returning sooner than those from more distant objects (Leonardi et al., 2009).
ToF cameras can provide depth data at high frame rates, typically up to 100 Hz, with a
depth resolution of approximately 10 mm. Their limitation is a low X-Y resolution, typically
a maximum of 320x240 pixels with today’s technology (Schuon et al., 2008).
Noise is also an important factor, potentially obscuring objects in the scene. To overcome
this issue, the ToF camera is combined with a higher resolution RGB camera with the
assumption that changes in depth will align with the RGB camera’s edges. This achieves a
100 fold increase in the spatial resolution (Yang et al., 2007). Also, it has been demonstrated
that multiple depth images with slight offsets can be used to create a super resolution
composite image, giving high accuracy from just one ToF depth sensor (Schuon et al.,
2008). This, however, is not applicable in situations requiring real-time imaging.
Structured Light 3D Scanner
Structured light 3D scanners project a light pattern, normally consist of lines or a grid of
dots, onto the scene which are then captured by a camera. The deformation of the pattern
on the 3D surfaces of the scene can then be used to calculate a depth map. In situations
when it is not acceptable to project a visible pattern onto the scene, it is possible to use
an infra-red projection, as is used by Microsoft’s Kinect. The resolution of these cameras
depends on the resolution of the projected array of dots, and on the proximity of the object
to the camera. On specification sheets, the resolution of the light grid is often not given,
and they instead give the resolution of the receiving camera. This does not give an accurate
representation of the resolution that it can provide.
Hardware
There are a number of companies developing consumer solutions to 3D hand tracking. Whilst
most companies develop both the hardware sensors and the accompanying software, there
have also been several examples of software-only and hardware-only developers in the field.
The current solutions are aimed at natural interface applications for computers, games con-
soles, TVs and other systems that could benefit from contact-less interaction. Although
many of the products have already been established and verified, there has been a consider-
able amount of development taking place in order to improve the technology while shrinking
the size of the device.
Each hardware vendor has their own accompanying software. However, open APIs are also
provided, so it is conceivable that each software suite could work with any other hardware.
There also exist software suites that are not backed by a specific hardware vendor.
Kinect 1 - Kinect 2
As previously mentioned, Microsoft Kinect (see Figure 5.7) is a popular non-contact sensor
that can be used for stroke rehabilitation. Furthermore, it has already started to be utilised
in upper-limb stroke rehabilitation areas such as hand function (Metcalf et al., 2013), upper-
limb movement (Chang et al., 2011), VR therapy (Lee, 2013), and FES assistance (Exell
et al., 2013). Research into Kinect measuring capacity has shown that its depth accuracy
can be up to 12 mm at a distance of 2m. In near mode, the Kinect’s camera can observe
objects from 400mm to 3 metres accurately. Although Kinect is affordable and provides
one-sensor tracking, data drop-out is unavoidable.
While Kinect 1 has been used for the purpose of rehabilitation and can provide satisfac-
tory feedback to the controller, the frames per second (fps) become unbalanced when the
computation load increases. Kinect 2 (see Figure 5.8), is an updated version designed to
IR Emi tter Color Sensor

IR Depth Sensor
Tilt Motor
Microphone Array
Figure 5.7: Kinect 1 taken from Microsoft (2012b)
be able to provide upper-limb data and detailed hand information, released in mid-July of
2014. Kinect 2, with its improved processor and resolution, is more advanced than Kinect 1.
Therefore, it is expected to give superior feedback for model based control within a low-cost
home-based system, even if the frame-rate decreases. Kinect 2 can detect 25 skeletal joints
including the hand and wrist (Microsoft.com, 2014). Moreover, its average depth accuracy
is under 2 mm at the distance of 2m. Kinect 2 can observe objects up to 4 metres but it
reduces accuracy and increase average error beyond 4mm (Yang et al., 2015). Furthermore,
Kinect 2 can be used for detecting heart rate and blood pressure. This means it may provide
condition monitoring during rehabilitation.
Figure 5.8: Kinect 2 taken from Microsoft.com (2014)
PrimeSense Carmine
High profile devices, such as Microsoft Kinect and the Asus Xtion, use hardware developed
by PrimeSense. As seen in Figure 5.9, the device combines a high resolution RGB camera
and a structured light 3D scanner. PrimeSense produces its own reference design sensors,
of which the Carmine 1.09 is aimed at short range, accurate scanning, making it ideal for
gesture recognition. Priced at £154, it was also relatively affordable when released in 2012
(PrimeSense, 2012). However, this product is no longer commercially available.
Figure 5.9: PrimeSense taken from PrimeSense (2012)
Leap Motion
Leap Motion is currently developing advanced motion sensing technology for human-computer
interaction. Their product, the Leap Motion Controller, is a small device aimed at precise
short range hand recognition. The device detects all 10 fingers, or any other object in an
area of eight cubic feet above the sensor, with an accuracy of 200 times that of current ges-
ture control systems. Thus, it has high potential for use within home-based systems. The
controller is very small at 76.2 mm long, is USB powered, and is cheap, with a price of only
£50 in 2016 (Leap Motion, 2013). Furthermore, Leap Motion has collaborated with Oculus
Rift to create virtual and augmented reality. This means it may provide more interactive
rehabilitation to motivate patients (Leap Motion, 2014).
Figure 5.10: Leap Motion taken from Leap Motion (2013)

Comparison of Non-Contact Sensors
In Table 5.2, the non-contact sensors which have been examined and are compared according
to their prices and specifications such as distance, frame rate and resolution. Most of these
devices are suitable for use in a low-cost home-based system when combined with suitable
software. In a home rehabilitation system, the operation must be simple, so ideally a non-
contact sensing system with minimal or no calibration is required.
Table 5.2: Comparison of non-contact sensor devices

Device Price Frame per second Speciality
Kinect for Windows £199.99 40 — 480p Good at working long distances
Leap Motion £50 240 — QVGA Good at working short distances
PrimeSense £154 120 — QVGA Good at working short distances

Carmine 1.09
DepthSense 325 £192 25-30 fps — QVGA Good with hand capturing
50 -60 fps — QQVGA
Intel Creative RealSense £77 60 fps —VGA Great with hand capturing
120fps —HVGA
Duo3D £153 184 fps — QVGA There is no tracking software available
Excellent skeletal tracking for

Kinect v2 £110 30 fps — 1080p both short and long range
including hand and wrist data.
Excellent hand data tracking

Camboard Pico £530 90 fps — QQVGA Great depth resolution
(<4mm @ 700mm (75% reflectivity))
5.1.2.1 Software
The software solutions provided by either the manufacturers of the sensors or third parties
are normally device independent. There are five main gesture development suites available;
OpenNI, IISU, Omek Interactive suite, Leap Motion and Intel RealSense. All software
includes the SDK and the middleware. The middleware is the software containing the
algorithms used to detect the hand and translate its movements to usable data.
The requirements for use in a low-cost rehabilitation system are for the software to be able
to accurately track a hand, which will often be in a clenched state, with minimum cost both
for the software itself and the hardware required for it. The most important requirement
of the software is accurate detection and tracking of the patient’s hand. Unfortunately full
testing of each software suite was not possible, and as such the performance has been judged
subjectively. The valuations were based on technical data, using video demonstration from
developers and third part users. The most convincingly robust demonstration was provided
by the SoftKinetic ISSU suite, with error free tracking of two hands, and similarly robust
tracking of individual fingers and thumb in a fist posture. This was closely seconded by
the OpenNI suite used in conjunction with the 3D Hand Tracking Library middleware,
which demonstrated good tracking of individual fingers and thumb when in a fist. However
the video did not show this working from multiple angles, or for partially occluded hands.
On the other hand, 3Gear system is a recently released middleware system, developed with
OpenNI, providing accurate and fast data. It can be used with both the PrimeSense Carmine
1.09 or the Kinect sensor or sensors. The main difference between OpenNI suit and 3Gear
middleware is the latter saves hand scales and gesture usage to detect precise hand and
finger data. The Omek Interactive Grasp development suite shows very accurate tracking
of two hands, even when partially occluded. However no evidence was found demonstrating
tracking of a fist, therefore its suitability for rehabilitation feedback is unknown.
The metric that is easiest to compare is the cost. OpenNI and Leap Motion SDK are
available free of charge. OpenNI is also potentially the most versatile due to its open source
nature and the number of compatible middleware packages available. One of its middleware
options is 3Gear System, which shows high potential for hand detection with a price of
£250. Note that it is free for academic and personal use. It is also compatible with most
of 3D cameras investigated in this chapter. The 3D Hand Tracking Library middleware
requires a CUDA (Compute Unified Device Architecture) capable NVidia graphics card,
but these are not very expensive. On the other hand, Leap Motion SDK is only designed
for use with the Leap Motion. The free Intel SDK provided by SoftKinetic and RealSense
is limited for use with the DepthSense and RealSense cameras, respectively. SoftKinectic is
the most expensive software considered in this chapter, although it needs no other special
hardware. Of the remaining alternatives, the Omek Interactive Grasp development suite
was free, however the company was bought by Intel and it is not commercially available.
Finally the IISU suit retails for £1154, making it firmly the least competitive.
Table 5.3: Non-contact sensors software-middleware comparison

Software Price(£) Fist Tracing Hardware
OpenNI Free Yes CUDA enabled GPU
SoftKinectic Intel SDK Free Yes DepthSense 325
SoftKinetic IISU Pro 1154 Yes None
Intel RealSense SDK Free Yes RealSense Camera
Omek Suite Beta-Free Yes None
3Gear Systems 250 Yes Kinect, PrimeSense and Camboard Pico
Leap Motion SDK Free Yes Leap Motion
5.1.3. Conclusions 71
5.1.3 Conclusions
For a stroke rehabilitation system to maintain accurate, reliable feedback of hand movement,
a combination of software and hardware, is needed. This needs to be fit for purpose while
keeping costs at a minimum. Of the hardware sensors investigated, all are capable of per-
forming the required function. However, among the older two (i.e. Kinect and PrimeSense
Carmine), the second is cheaper at £154, and is designed for short range hand detection.
The more advanced ToF technology within the SoftKinetic camera is the reason for the
price difference. However due to the very similar claimed abilities, there is no benefit in
specifying this controlled usage scenario. Also, among the newest devices, the Intel Cre-
ative cameras, Kinect v2 and Leap Motion compete with one another with relation to hand
detection and all provide accurate feedback data. Overall, the most potential advantages of
a Leap Motion or a PrimeSense because they combine low-cost, reliability and widespread
and accessible software. Due to the current fast rate of development in this area, further
generation products can be expected in the near future that will offer improvements to these
sensors’ accuracy and value over the current offerings.
5.2 Evaluation of Leap Motion and PrimeSense
In the first part of this chapter, the potential of non-contact based sensing technology has
been established and a number of suitable approaches have been compared. Methods for
sensing and identifying hand kinematics were grouped into computer vision based (Erol
et al., 2007), glove based (Wang and Popović, 2009) and surface marker based systems
(Fu and Santello, 2010). The latter two were found to be both expensive and unsuitable
for use in home-based stroke rehabilitation systems. However recent non-contact sensor
innovations mean that computer vision based systems have become suitable for sensing of
hand and wrist. This opens up a potential avenue that has already been partially explored
in the GO-SAIL system of Chapter 4. In this section, two of the most promising non-contact
sensing technologies will be implemented and evaluated in more detail. In particular the
feasibility of using Leap Motion and PrimeSense with 3Gear middleware in a rehabilitation
system will be assessed. Accuracy will be evaluated through comparison with a validated
goniometer over a range of gestures. Note that the first generation of Leap Motion had
highly restricted joint ranges and initial test results can be found in Appendix D. However,
the second generation solved this issue and is used in the tests reported in this section.
5.2.1 Sensor Requirement Specification
To assess the functionality and usability of Leap Motion and PrimeSense for stroke reha-
bilitation a set of criteria are needed with which to gauge its limitations and accuracy. To
support activities of daily living, the most relevant joints comprise the wrist, finger flex-
ion/extension and thumb flexion/extension. Studies have shown that 42% of the functional
movements of the hand involve the four fingers moving together (Ingram et al., 2008), hence
in some circumstances an average angle is acceptable. It is also not necessary to consider
angles about the distal interphalangeal (DIP), due to their limited range of movement. For a
low level of patient function and simple task set, a joint accuracy of ±10◦ may be acceptable,
but this scales with the required task precision. Within their software, Leap Motion and
PrimeSense use different computational algorithms to extract the same position markers.
Thus, to compare the devices, it is next necessary to develop methods to extract joint angles
from the raw position data provided by each system. There are a wide range of kinematic
hand models in the literature (Cobos et al., 2010).
5.2.2 Human Hand Model and Joint Angle Computation
The model employed by both Leap Motion and PrimeSense is shown in Figure 5.11, and
includes proximal interphalangeal (PIP) and metacarpophalangeal (MCP) finger joint po-
sitions. Note that Leap Motion also provides DIP joint data but this has been excluded
for the sake of comparison, and since it is not necessary for this application. The sensors
also give root (R) and wrist (W) joint positions. The task is therefore to extract the wrist
abduction/adduction and flexion/extension angles. The finger variables PIP, MCP and tip
positions will now be used to calculate the two revolute joints of each finger.
Fingertip,PIP and MCP joints data are denoted as fi , pi and mi respectively for i = 1, ...5.
The cross and dot products are used to calculate the associated joint angle variables. The
angles, φ13 to φ17 , can be calculated using wrist, MCP and PIP data via the expression
W~mi · m~i pi
φi+7 = arccos i = 1, . . . , 5. (5.1)
W~mi |m~i pi |

The angles between PIP and fingertips, φ6 to φ10 , can be calculated from
m~i pi · p~i fi
φi+12 = arccos i = 1, . . . , 5. (5.2)
|m~i pi | p~i fi

In particular, wrist flexion/extension, φ6 , is given by

 
~ · W~m3
RW
φ6 = arccos  . (5.3)
RW~ W~m3
The unit vector from the index MCP joint to the little finger MCP joint is next defined as
5.2.3. Experiment Evaluation 73
R(x19,y19,z19)
φ7
φ6
φ12 m1(x4,y4,z4)
W(x3,y3,z3)
φ11 m2(x5,y5,z5)
φ17
p1(x9,y9,z9) m3(x6,y6,z6)
φ10
φ16 m4(x7,y7,z7)
p2(x10,y10,z10) φ9
φ8 m5(x8,y8,z8)
f1(x14,y14,z14)
f3(x16,y16,z16) f5(x18,y18,z18)
φ15
f2(x15,y15,z15) φ f4(x17,y17,z17)
14
p3(x11,y11,z11) φ13
p4(x12,y12,z12) p5(x13,y13,z13)
Figure 5.11: Simplified hand kinematic model and joint angles (left hand view)
T~ = m5~m3 . The abduction/adduction angle φ7 is then defined as

 
RW~ × W~m3
~ ◦
φ7 = arccos  ~ · T − 90 . (5.4)

~
RW W m3
5.2.3 Experiment Evaluation
Experimental Set-up
To evaluate the joint angle accuracy of each sensing approach, three movements were defined
with each movement starting from a relaxed hand position, as illustrated in Figure 5.12(a).
During the process of data collection, the participant’s elbow was supported by a passive
deweighting mechanism. Defined movements were chosen relevant to activities of daily
living. These comprise open-hand movement for grasping and releasing objects, pointing
for switching lights and pinching movements, and are shown in Figures 5.12(b), 5.12(c) and
5.12(d) respectively. Figure 5.13 shows hand joint angle data collected simultaneously by
PrimeSense and Leap Motion. Wrist, index and thumb joint angles data were collected
by an electrogoniometer for the purpose of comparing open hand, pointing and pinching
gestures, respectively.
(a) Starting gesture (b) Open hand gesture
(c) Pointing gesture (d) Pinching gesture
Figure 5.12: Experimental hand gestures (left hand view).
A goniometer was chosen as a gold standard because it provides an accuracy of ±1◦ (Shiratsu
and Coury, 2003) and ±0.1◦ for 4 different angles (10◦ , 20◦ , 40◦ and 60◦ ) for 6 different static
directions (Christensen, 1999). It is also relatively easy to calibrate and is easy to don/doff,
unlike data gloves. Leap Motion and PrimeSense with 3Gear middleware both were initially
calibrated to increase accuracy and reduce data drop out effects.
5.2.3. Experiment Evaluation 75
(a) Experiment set-up: goniometer (b) Middleware for Leap Motion and PrimeSense
Figure 5.13: Experimental set up of goniometer and non-contact sensors.
Experimental Results
The goniometer, Leap Motion and PrimeSense data are denoted by g, l, p superscripts re-
spectively. For Leap Motion, angular errors for open hand, pointing and pinching movements
are calculated respectively from
elopenhand = φg6 − φl6 ,

elpointing = φg11 − φl11 ,
elpinching = φg12 − φl12 .
Similarly, for PrimeSense, the angular errors are respectively denoted by
epopenhand = φg6 − φp6 ,

eppointing = φg11 − φp11 ,
eppinching = φg12 − φp12 .
Table 5.4: Comparison of PrimeSense and Leap Motion results

Task PrimeSense error(rmse) (deg) Leap Motion error (rmse) (deg)
Open Hand 4.6 3.0
Pointing 1.9 2.6
Pinching 3.2 2.4
Results confirm that Leap Motion and PrimeSense both provide joint angle data with and
accuracy within ±10◦ , as seen from Figures 5.14(a), 5.14(b), 5.14(c) and Table 5.4. Hence
either can be integrated in the GO-SAIL system.
5.2.4 Conclusions
Similar levels of accuracy from both technologies were established during representative
activities of daily living. Among all results, the accuracy of pointing is far more better
than other joints as its defined gesture when it became complex such as wrist or thumb
joints for open-hand and pinching movement accuracy of the short range non-contact sensors
decreases within acceptable levels as seen from Table 5.4. Selection hence focuses on usability
and interfacing issues. Due to the ease in which PrimeSense can be incorporated into the
present software structure, it will be chosen for integration as a component within the further
development of the technology developed in Chapter 4.
5.3 Summary
This chapter has confirmed that non-contact sensors are suitable for home-based systems.
In particular, both Leap Motion and PrimeSense with 3Gear System are ideally suited for
hand rehabilitation given their low cost and accuracy to within 1 mm. To understand the
limitations, accuracy and usability of Leap Motion and PrimeSense, they are also compared
experimentally in the second part of the chapter.
5.3. Summary 77
80
Goniometer
PrimeSense
Joint Angle(φ6) (deg) 60 Leap Motion
40
20
−20
−40
0 1 2 3 4 5 6 7 8 9 10
Time (sec)
(a) Open hand movement experiment
0
Goniometer
PrimeSense
−5 Leap Motion
Joint Angle (φ11) (deg)
−10
−15
−20
−25
−30
−35
0 1 2 3 4 5 6 7 8 9 10
Time (sec)
(b) Pointing movement experiment
Goniometer
PrimeSense
Leap Motion
−10
Joint Angle (φ12) (deg)
−15
−20
−25
0 1 2 3 4 5 6 7 8 9 10
Time (sec)
(c) Pinching movement experiment
Figure 5.14: Comparison between Leap Motion and PrimeSense joint angles for three
movements.
Chapter 6
GO-SAIL with Improved Hand

Functionality
6.1 Introduction
The GO-SAIL system developed in Chapter 4 demonstrated feasibility of using single elec-
trodes to deliver precisely controlled stimulation to the anterior deltoid, triceps and wrist
extensors (Hughes et al., 2009; Meadmore et al., 2014, 2012). However, the system could not
decisively produce complex hand and wrist movement such as pointing, pinching and grasp-
ing. Electrode arrays were proposed as a solution to address this problem, but needed hand
and wrist position data to produce these complex motions. Therefore, appropriate sens-
ing technologies have been investigated and discussed. After identifying suitable sensors,
experimental validation was conducted as reported in Chapter 5.
The system developed in this chapter integrates the sensor technology evaluated in Chapter
5 with substantial developments in the GO-SAIL system of Chapter 4. It also embeds
significant innovations in the stimulation hardware, control algorithms, and task display. In
particular, the system integrates three important new components:
1. An electrode array placed over the wrist and finger extensors that enables functional
hand gestures to be performed.
2. A PrimeSense sensor that measures hand and wrist joint angles, reducing set-up time
and removing constraints associated with contact-based sensors (e.g. goniometers).
3. A touch table that displays each task in an interactive manner.
These new developments are expected to provide more effective rehabilitation and lead to
further reduction in upper limb motor impairments, as reflected by evidence that functional
79
80 Ch. 6. GO-SAIL with Improved Hand Functionality
improvement from training is mostly restricted to the actually trained functions and activi-
ties (Veerbeek et al., 2014). In addition, the touch table is intended to promote adherence by
providing exciting and motivating rehabilitation, as required in long term self-management.
Its combination with inexpensive non-contact sensors (Kinect and PrimeSense) represents
a significant step in the development of technology that is suitable for translation into the
home environment. The new system, named GO-SAIL+ and its components are described
in the next section. Following this, Section 6.3 contains experimental evaluation results,
both with unimpaired participants, and with stroke participants during a clinical feasibility
trial. Section 6.5 provides conclusions.
6.2 System Overview
Figure 6.1: System components: 1) motion tracking hardware components, 2) interactive

touch table display, 3) FES controller and multiplexor hardware, 4) SaeboMAS and perching
stool, and 5) electrode-array.
The system comprises the five components shown in Figure 6.1. Participants sit on a
perching stool in front of a touch table, and a SaeboMAS arm support (Saebo Inc., Charlotte)
is used to de-weight their upper extremity according to individual need. Surface electrodes
are positioned on the anterior deltoid and triceps as in the GOSAIL system of Chapter
4, and an electrode array is placed over the common extensor complex of the forearm.
The PrimeSense is used in combination with the depth camera used in GO-SAIL (Kinect;
Microsoft Washington) to measure the position of joint centres within the shoulder, elbow
and wrist. Data from these sensors are fed into the control algorithm hardware and software,
which updates the FES control signals for each muscle group to provide just enough electrical
stimulation to assist performance of functional tasks. These tasks include picking up a
tube of toothpaste, and moving the item to a different position on a representation of a
sink displayed by the touch table. As in GO-SAIL the therapist uses an operator monitor
displaying a graphical user interface (GUI) to select appropriate tasks and monitor training
progression. The therapist also has an over-ride stop button which can be used to terminate
trials with immediate effect.
6.2.1 Task Design
Functional tasks that are typically performed in everyday life were designed to offer a range
of reaching challenges across the workspace. These extend the level of interaction that was
provided by GO-SAIL. Figure 6.2 shows the four main images used on the touch table
background; a default image, a bathroom sink, a coffee table, and a chopping board. Tasks
comprised reaching, grasping and manipulating real objects relevant to each image. There
are five main tasks; closing a drawer, switching on a light switch, stabilising an object,
button pressing and repositioning an object. The light switch is located at two different
heights (low and high) and there are four table-mounted positions at which the virtual
buttons can be located or real objects repositioned both in the sagittal plane and towards
the frontal plane (45◦ across body, 45◦ to the hemiplegic side or in line with the shoulder)
as illustrated in Figure 6.2. The objects are placed at different percentages of arm length
(60%, 75%, and 90%) from the participant’s glenohumeral joint as shown in Figure 6.2a).
The table is positioned at a distance of 45% arm length from the glenohumeral joint and 35
cm below the arm when the arm was held 90◦ horizontal to the shoulder.
6.2.2 System Software
The software and data flow is shown schematically in Figure 6.3. The system software under-
takes tracking of the participant’s movement in real-time, extraction of kinematic variables,
and subsequent implementation of control schemes to adjust the FES in real-time. A custom
made C++ application interfaces with Kinect middle-ware (Skeletal Viewer) as detailed in
Chapter 4, which in turns receives data from PrimeSense via a Client/Server (Transmis-
sion Control Protocol (TCP)) connection with its associated middle-ware (3Gear Systems)
as partially detailed in Chapter 5. This application directly communicates with real-time
hardware (dSPACE ds1104), which handles all data processing and control implementation,
Figure 6.2: Task design and graphical backgrounds: a) default (overlaid with task place-
ment geometry), b) bathroom sink, c) coffee table and d) chopping board.
and interfaces with the touch table and GUI via direct hardware access. Communication
with the touch table employs Snowflake software (NUITEQ Inc., Sweden) which controls
the task display and touch feedback. The GUI oversees communication with the system
inputs and outputs and is responsible for customising control parameters, implementing the
FES control, collecting and storing position data, selecting the task to be performed and
reviewing performance after each session. The real-time hardware generates pulse-width
modulated (PWM) signals for each of the FES stimulator channels, together with RS232
serial data to control the electrode array. Digital inputs and outputs are also employed to
interface with the instrumented task objects.
®
Mathematical Microsoft Kinect
Software Custom Made Application Sensor
Environment (Visual C++)
®
Data PrimeSense
Digital I/O
Processing Kinect Interface Sensor

(Visual C++)
Real-time Processor
Graphical User
(933MHz)
Real-time
Interface
Client / Server Model Instrumented

Interface
(Visual C++)
Task Objects
ADC
PrimeSense Interface
(Visual C++, Open GL) Virtual Reality
Background
PWM
Touch table Interface

Spring Support
De-weighting
(Visual C++)
Multiplexor
Touch Table Screen FES Module
Figure 6.3: Software signal flow diagram.

6.2.3. Hardware 83
6.2.3 Hardware
The GO-SAIL+ system comprises a real-time controller, motion tracking, FES Hardware,
and arm support. These components are now described.
Real-Time Hardware
GO-SAIL+ employs a similar real-time control environment to that of GO-SAIL, comprising

a dSPACE hardware platform which interfaces with a mathematical software environment.
While GO-SAIL used dSPACE ds1103, in GOSAIL+ a ds1104 real-time control board is
used. This supports the same required functionality but is smaller and a fraction of price.
FES Hardware
The FES hardware comprises single pad electrodes, an electrode array, and electronic com-
ponents to generate and route stimulation signals. The single pad FES surface electrodes
are positioned on the anterior deltoid and triceps muscles, with placement following clinical
guidelines (Levin, 1996). Similarly the electrode array is placed on the forearm to actuate
wrist and hand extensor muscles. The electrode array comprises 4 x 6 elements printed on
a PVC substrate, using a hydrogel interface layer. It is manufactured by Tecnalia-Fatronik,
San Sebastian, Spain, and is described in (Malešević et al., 2012). Each element can be
routed to two of the four available FES channels. This is achieved via custom-made RS232
controlled multiplexor hardware, comprising an Arduino board and shift register array,
which interface with a relay bank. The optimal electrode sites are selected for each required
hand and wrist posture relevant to tasks of daily living, using an optimisation procedure
described in Section 6.2.4.2. These postures are shown in Figure 6.4 and comprise open-
hand and pinching movements for grasping and releasing objects, and a pointing movement
for switching lights and pushing buttons. The array stimulation sites are then fixed dur-
ing subsequent experiments, with the two stimulation channels controlled using real-time
algorithms, as described in Section 6.2.4.2.
For each FES channel, the control hardware produces a 5V, 40 Hz pulse train, where pulse-
width is the controlled variable (0-300µs). These are then fed to a four channel stimulator
(Odstock Medical Ltd., Salisbury, UK) which amplifies the voltage of each channel to a
fixed level which is determined at the beginning of each session. The voltage amplitude is
set by applying a 300 µs signal to the stimulation site and slowly increasing a dial on the
stimulator until the maximum comfortable level is achieved. This procedure matches that
used in the GO-SAIL system of Chapter 4. When setting the voltage amplitude of an array
channel, the stimulation site comprises two adjacent array elements located over the wrist
and hand extensor muscles. (See Appendix A for detailed identification protocol.)
a) b) c) d)
Figure 6.4: Experimental hand gestures and identified array elements (left hand view): a)
Starting, b) Open hand, c) Pointing, and d) Pinch gestures. Red and blue squares present
participant 1 and 2’s two element solutions respectively.
Motion Tracking
The GO-SAIL system employed a Kinect sensor to provide shoulder, elbow and wrist posi-
tions accurately. However, this sensor provides only a single joint position for the hand and
bespoke software developed to extract finger joint location measurements severely restricts
the range of admissible movement (Metcalf et al., 2013). Thus, an electrogoniometer was
implemented in GO-SAIL to collect wrist joint angle, as described in Chapter 4. This so-
lution cannot support functional movements such as pointing and pinching via stimulating
single electrodes. Therefore, the GOSAIL+ system also incorporates PrimeSense to collect
the wrist position data and individual finger joint centre position data via commercial 3Gear
middle-ware. In Chapter 5, the accuracy of Primesense was validated. However, this may
degrade due to interference with Kinect as they are both time of flight cameras. Although
there is evidence that using two Kinects can cause interference, this has been found to have
little effect on measurement and is strongly correlated with the distance between sensors
and observed object (Martin et al., 2014). To examine sensor efficacy, joint error has been
recorded during repeated tests performed using the proposed training task set, and perfor-
mance has been quantified through comparison with a goniometer. A minimum mean joint
error of less than 10◦ has been established with the Kinect placed at 45◦ on the opposite side
of the impaired arm at a −20◦ pitch angle in sitting mode, and the PrimeSense positioned
700 mm above the touch-table.
The kinematic model employed in this research is a substantial development of UE models

used in GO-SAIL as detailed in Chapter 4, and includes a comprehensive hand description
φ1
φ2
φ3
φ7 φ5
φ4
φ6 φ6
φ7
φ12
φ12 φ9 φ8
φ9 φ8
φ
φ φ11 10 φ13
φ17 17
φ11 φ10 φ16 φ15 φ14
φ17 φ13
φ16 φ15 φ14
Figure 6.5: Human arm kinematic model.
in Chapter 5.2. Joint centres and corresponding joint angles are shown in Figure 5.11
and 6.5 respectively. The Kinect is used to capture joint centre locations (xi , yi , zi )> for
the shoulder, elbow and wrist, i = 1, 2, 3 respectively. The PrimeSense captures joint
centre locations (xi , yi , zi )> for the hand and wrist, i = 3 . . . 18, respectively as detailed in
Figure 5.11. Joint angles φ1 , · · · φ5 denote the orientation of the upper arm and forearm
segments, with joint axes that are chosen to align with the motion elicited by FES. The
procedure employed to define φ1 , · · · φ5 is described in (Freeman et al., 2011), together with
the mapping (x1 , y1 , z1 )> , · · · , (x3 , y3 , z3 )> 7→ (φ1 , · · · , φ5 ).

The GO-SAIL+ system embeds an extended control strategy to that of GO-SAIL system
which was detailed in Section 4.2.4. This is based on an extended biomechanical model
to that described in Section 4.2.4. Similarly the data transfer protocol has been extended
to support electrode array. In this section the extended biomechanical model and control
strategy will be discussed.
6.2.4.1 Biomechanical Model
The controller updates the stimulation signal applied to each muscle group with the aim
of reducing the error in performing each task from one attempt to the next. The control
signal is generated using kinematic joint information, in combination with a biomechanical
dynamic model of the stimulated arm, given by
Bh (Φ)Φ̈ + Ch (Φ, Φ̇)Φ̇ + Fh (Φ, Φ̇) + Gh (Φ) = g̃(u, Φ, Φ̇) − J > (Φ)h (6.1)
where Bh (·) and Ch (·) are 17-by-17 inertial and Coriolis matrices respectively, Fh (·) and
Gh (·) are friction and gravitational vectors, h is a vector of external force and torque due to
interaction with objects, and J(Φ) is the system Jacobian. The vectors Φ = [φ1 , · · · , φ17 ]>
and u = [u1 , u2 , u3 , u4 ]> respectively denote joint angles and applied electrical stimulation,
where u1 (t) and u2 (t) represent the electrical stimulation pulse-width signals applied to the
anterior deltoid and triceps muscles respectively, and u3 (t) and u4 (t) represent the electrical
stimulation pulse-width signals that are routed to elements within the electrode array at
time t. The vector g̃(·), comprising the resulting moments produced through application of
FES, has form
g̃(u, Φ, Φ̇) = [g̃1 (u, Φ, Φ̇), · · · , g̃17 (u, Φ, Φ̇)]> . (6.2)
The moments produced through stimulation of the anterior deltoid and triceps take the form
g̃2 (u, Φ, Φ̇) = g2 (u1 , φ2 , φ̇2 ) and g̃4 (u, Φ, Φ̇) = g4 (u2 , φ4 , φ̇4 ), respectively. The components
associated with the unassisted upper arm and forearm joint angles are given by g̃i (u, Φ, Φ̇) =
0, i = 1, 3, 5.
The stimulation signal that is transmitted to the 24 elements of the array is given by
µ(t) = [µ1 (t), · · · , µ24 (t)]> = W [u3 (t), u4 (t)]> where matrix W ∈ R24×2 has elements which
are determined as 0 or 1 by the relay hardware. When stimulated, it is assumed that each
element of the electrode array may produce a moment about any of the hand and wrist joint
axes, leading to the form
24
X
g̃i (u, Φ, Φ̇) = g̃i,j (µj (t), φi , φ̇i ), i = 6, . . . , 17. (6.3)
j=1
Moments generated about φ4 , φ5 can similarly be added to g̃i (·), i = 4, 5 but have been
found to be negligible. For the same reason, the effect of φ4 , φ5 on g̃i (·), i = 6, · · · , 17 has
been neglected. From (Le et al., 2010), the moments in (6.2) generated by stimulation of
the anterior deltoid and triceps take the general forms
g2 (u1 (t), φ2 , φ̇2 ) = h2 (u1 , t) × Fm,2 (φ2 , φ̇2 ), g4 (u2 (t), φ4 , φ̇4 ) = h4 (u2 , t) × Fm,4 (φ4 , φ̇4 ).
(6.4)
where hi (ui , t) is a Hammerstein structure incorporating a static nonlinearity, hIRC,i (ui ),
hLAD,i (t). The term Fm,i (φ, φ̇) models the multiplicative effect of the joint angle and joint
angular velocity on the active torque developed by the muscle. Due to weakness, spasticity
and fatigue, stroke patients commonly experience slow restricted movement in their hand
and wrist. This means the multiplicative effect of angle and angular velocity can be neglected
since it is approximately unity (Bernotas et al., 1986). The moment around axis i due to
stimulation of electrode array element j is accordingly given by
g̃i,j (µj (t), φi , φ̇i ) = hi,j (µj , t), i = 6, · · · , 17, j = 1, · · · , 24. (6.5)
where hi,j (µj , t) is a Hammerstein structure incorporating a static nonlinearity, hIRC,i,j (µj ),
hLAD,i (t). Inserting (6.5) into (6.3) yields
g̃i (u, Φ, Φ̇) = gi (µ(t), φi , φ̇i ) = hi (µ, t), i = 6, · · · , 17 (6.6)
where hi (µ, t) is a composite Hammerstein structure incorporating nonlinearity hIRC,i (µ) =

P24
j=1 hi,j (µj , t), cascaded with linear activation dynamics, hLAD,i (t). The SaeboMAS sup-
port structure has the same form as in Chapter 4, given by
Bs (Θ)Θ̈ + Cs (Θ, Θ̇)Θ̇ + Fs (Θ, Θ̇) + Gs (Θ) + Ks (Θ) = 0 (6.7)
where vector Θ = [θ1 , · · · θ5 ]> contains the joint angles of the spring support as shown
in Figure 6.5, Bs (·) and Cs (·) are 5-by-5 inertial and Coriolis matrices, and Fs (·) and
Gs (·) are friction and gravitational vectors respectively. Vector Ks (·) comprises the mo-
ments produced through gravity compensation provided by the spring, which takes form
[k1 (θ1 ), 0, 0, 0, 0]> . When connected to arm structure (6.1), a bijective mapping between
joint angles, Θ = M (Φ), yields the combined model
B(Φ)Φ̈ + C(Φ, Φ̇)Φ̇ + F (Φ, Φ̇) + G(Φ) + K(Φ) = g̃(u, Φ, Φ̇) − J > (Φ)h (6.8)
where
B(Φ) =Bh (Φ) + M1 (Φ)> Bs (M (Φ))M1 (Φ),

G(Φ) =Gh (Φ) + M1 (Φ)> Gs (M (Φ)),
C(Φ, Φ̇) =Ch (Φ, Φ̇) + M1 (Φ)> Cs (M (Φ), M1 (Φ)Φ̇)M1 (Φ),
K(Φ) =M1 (Φ)> Ks (M (Φ)),
F (Φ, Φ̇) =Fh (Φ, Φ̇) + M1 (Φ)> Fs (M (Φ), M1 (Φ)Φ̇) + M1 (Φ)> Bs (M (Φ))M2 (Φ, Φ̇)Φ̇

dM (Φ) d dM (Φ)
with M1 (Φ) = dΦ and M2 (Φ, Φ̇) = dt dΦ . These relationships result by simply
combining the arm and support structures (6.1) and (6.7) and employing the chain rule.
This model is next used by the FES control system.
6.2.4.2 Control Scheme
The control structure is a direct extension to that formulated in Chapter 4. The com-
bined human arm and support dynamics (6.8) are shown in Figure 6.6, where the mo-
ments due to muscle activation are ga (ua , Φa , Φ̇a ) := [0, g2 (u1 , φ2 , φ̇2 ), 0, g4 (u2 , φ4 , φ̇4 ), 0]>
and gw (µ, Φw , Φ̇w ) := [g6 (µ, φ6 , φ̇6 ), · · · , g17 (µ, φ17 , φ̇17 )]> . Here ua = [u1 , u2 ]> is the stimu-
lation applied to the shoulder and elbow, and uw = [u3 , u4 ]> is the stimulation applied to
the forearm muscles via the multiplexor and electrode array, so that µ = W uw . In addition,
Φa = [φ1 , · · · , φ5 ]> and Φw = [φ6 , · · · , φ17 ]> contain the joint angles of the upper arm and
wrist respectively. As with GO-SAIL, the reach and grasp tasks consist of repeated move-
ments for the participant’s affected arm, with a rest period in between, during which their
arm is returned to a common starting position. The reference Φ̂(t) = [φ̂1 (t), · · · , φ̂17 (t)]>
contains the desired joint angles over the trial duration t ∈ [0, T ]. The feedback controller is
partitioned as Cc = diag{Cc,a , Cc,w } and designed to establish stability and baseline track-
ing over each trial. The requirement to repeatedly perform a set of finite duration tasks
with a fixed initial arm position enables ILC to be utilised to improve tracking performance.
> , v > ]> in an
ILC uses the performance error from each trial to update the input vk = [va,k w,k
attempt to increase the accuracy of the subsequent attempt. On trial k, Φk (t) denotes the
joint angles and the associated error is given by ek = Φ̂ − Φk .
Figure 6.6: Block diagram of the ILC feedback scheme.

Note, the dashed line denotes computer memory updating of ILC signal vk , L is diag{La , Lw } and Cc is
diag{Cc,a , Cc,w }
Identification methods have been developed in previous research to establish parameters

within a model of the form (6.8) containing joints φ1 , · · · , φ5 , with the hand and wrist treated
as a lumped mass. This model information means that controller Cc,a can be selected to
take the form Cc,a = Ma Ka (s) where Ma : x 7→ ua is an input-output linearising controller
which introduces an auxiliary control input x and enforces dynamics φi (s) = Ha (s)xi (s),
1
i = 2, 4, where Ha (s) = s5
. Note that φ2 and φ4 denote the anterior deltoid and elbow joint
angles respectively. Feedback controller Ka (s) = [0, Ka,2 (s), 0, Ka,4 (s), 0]> is then selected
to stabilise the resultant closed-loop dynamics
Ga,i : (φ̂i +vk,i ) 7→ φk,i : φk,i (s) = (I+Ha (s)Ka,i (s))−1 Ha (s)Ka,i (s)(φ̂i (s)+vk,i (s)), i = 2, 4.
(6.9)
Implementation of Ma requires joint angles, angular velocities and estimated muscle states,
with full details given in (Freeman, 2015), which contains experimental results confirming
satisfactory closed-loop performance in the presence of modelling uncertainty. It is also
supported by detailed robustness analysis (Al-Gburi et al., 2013). The uncontrolled joint
angles φ1 , φ3 and φ5 are assumed to be under the patient’s remaining voluntary control,
however analysis in (Freeman, 2015) confirms that interaction with controlled variables does
not destabilise the system given sufficient damping and stiffness. Note that the above control
scheme is an alternative to that employed in GO-SAIL, but the previous simpler approach
which does not require full model information can also be used.
Due to the close proximity of muscles in the hand and wrist, and the difficulty in measuring
forces, it is not feasible to extend the global upper arm model identification approaches to
include the hand and wrist. Therefore a linearised model Hw (s) : µk 7→ Φw,k is identified
about a suitable operating point, as described in Section 6.2.4.2. Feedback controller Cc,w =
Kw (s) is then chosen to stabilise system Hw (s)W to yield the resulting closed-loop dynamics
Gw : (Φ̂w + vw,k ) 7→ Φw : Φw (s) = (I + Hw (s)W Kw (s))−1 Hw (s)W Kw (s)(Φ̂w (s) + vw,k (s)).
(6.10)
Many possible designs for Hw (s) exist, with the form presented in Appendix E providing
guaranteed error tracking properties and stability across all joints, Φw .
Having stabilised the arm dynamics, an ILC scheme is implemented in order to provide
input vk such that the error is minimised, i.e. lim vk = vk? with vk? := minvk kΦ̂ − Φk k2 . As
k→∞
in GO-SAIL, this is achieved through the update structure
vk+1 = vk + Lek , v0 = 0, k = 0, 1, · · · (6.11)
where L = diag{La , Lw }. Inserting (6.9) and (6.10) into (6.11), together with ek (t) =
> >
ea,k (t)> , ew,k (t)> and Φ̂(t) = Φ̂a (t)> , Φ̂w (t)> yields relationships

ea,k+1 = (I − Ga La )ea,k , ew,k+1 = (I − Gw Lw )ew,k ,

va,k+1 = (I − La Ga )va,k + La (I − Ga )Φ̂a , vw,k+1 = (I − Lw Gw )vw,k + Lw (I − Gw )Φ̂w .
(6.12)
which are direct extensions of those of GO-SAIL. For the arm dynamics, design of La,i to
satisfy kI − Ga,i La,i k < 1, i = 2, 4, guarantees convergence of φi to zero error, and many
suitable schemes are available, see (Freeman, 2014) and examples therein. The equivalent
relationship, kI − Gw Lw k < 1, guarantees convergence of Φw to zero error, but cannot be
satisfied due to the restricted range of operator Gw . Instead Lw can be designed to satisfy
kI − Lw Gw k < 1 which guarantees convergence of error ew,k to the limiting solution I −
Gw (Lw Gw )−1 Lw Φ̂w . Appropriate selection of Lw can hence minimise the error norm, and is

detailed in Appendix E. In both cases a robust ILC scheme can deal with dynamic changes
and modelling inaccuracy due to fatigue, spasticity and other time-varying physiological
effects, with robust uncertainty bounds given in Freeman and French (2015).
Model identification
Identification of the upper arm component of (6.8) firstly involves identifying the stimulated
joint axes within the kinematic model described in Section 6.2.4.1. This step is identical
to the procedure used in GO-SAIL and is achieved by applying a ramped 10 s FES signal
to the anterior deltoid and fitting a plane to the resulting movement. Transformations are
then embedded in the kinematic chain to align the φ2 axis to the plane normal, with all the
other axes defined as in Section 6.2.3.
Numerous methods can be used to identify the hand and wrist dynamics Hw (s) : µk 7→ Φw,k
around an operating point which corresponds to the k th trial, including fitting a linear
model to the trial data set {µk , Φk } (see (Freeman, 2016) for analysis of resulting stability),
or selecting a model from a set identified over a range of operating points at the beginning
of the treatment session.
In Appendix E feedback and ILC structures have been proposed for the case where the
hand and wrist dynamics assume the structure of Hw (s) = H̄w (s)P comprising uniform
dynamics, H̄w (s), and a 24 × 12 static mapping, P . In this case P can be identified by
applying a slow triangular ramp profile to each array element in turn and fitting a line to the
resulting joint movement. Dynamics H̄w (s) are then identified by stimulating all elements
with a single input sequence, µ, and fitting a single linear model to the combined data
{Φw , P µ}. Matrix W is then computed by solving minW,u kΦ̂w − H̄w P W uk2 for reference
trajectories corresponding to open hand, pointing and pinching. The matrix W then defines
the electrode array element-stimulator mapping for each gesture type, with representative
identified elements shown in Figure 6.4. Further details of the identification process appear
in (Freeman, 2014).
The system has been tested with both unimpaired and stroke participants. Following ethical
approval (EP/I01909X/1 and ERGO ID 7710), 6 participants (2 unimpaired and 4 stroke)
were recruited with demographic characteristics for the latter given in Table 6.1. At the
beginning of each session they were set-up at the workstation, which took approximately 15
minutes and comprised: 1) participant placement as described in Section 7.2, 2) electrode-
array and single electrode placement and setting of comfortable stimulation amplitude for
each channel as described in Section 7.2.3, 3) dynamic model identification as described
in Section 6.2.4.2. Each unimpaired participant attended a single session, in which they
used the system to perform the tasks described in Section 7.2.1. They were instructed to
provide no voluntary effort, and were not shown the task. Each stroke participant attended
17 intervention sessions. The inclusion criteria were: i) aged 18 years old or over; ii) stroke
causing hemiplegia of at least 6 months duration; iii) impaired upper limb that includes the
6.3. Experimental Evaluation 91
GOSAIL+
Engineering Side Clinical Side

Initial Stage (15-20 min) Initial Stage (15-20 min)
- Initialise PC, sensors and Real-Time hardware -Place participant to the stall and talk them for
(3 min) comforting (3 min)
- Initialise software (GUI and 3Gear) (2-3 min) - Check positions of muscle belly of AD, Triceps
- Check TCP connections and data acquisition and hand and wrist extensors for electrode
(1 min) placement via SENIAM guide (2 min)
- Check stimulator connections (2 min) - Place electrodes and check if the placement
- If electrode positioned correctly and correct via hand held stimulator (2 min)
participant ready, guide physiotherapist which -Place electrode array (1min)
channel corresponds which muscle. (2 min) - Connect OdStock stimulator to corresponding
-Check maximum comfortable level of channels with guidance (2 min)
stimulation for each channel (2 min) - Anterior Deltoid Identification (1 min)
- Anterior Deltoid Identification (1 min) - Electrode Array Identification (4min)
- Electrode Array Identification (4 min) -Check gestures (1 min)
- Check gestures (1min)
Experimental Evaluation (35-40 min) Experimental Evaluation (35-40 min)

- Record participant unassisted tasks in the - Check participant movements during unassisted
beginning of the session. tasks observe and take notes.
- Check if the collected data is meaningful. - Task selections according to participant.
- Select task with physiotherapist's guidance. performance in previous trials, fatigue level and
- Start test when participant and physiotherapist are feeling. Motivate the participant while give him/her
ready. a challenge.
-Check if the collected data is meaningful and - Order to start test when participant ready.
electrical stimulation is providing enough support -Check if the participant has enough challenge
with iteration. during tests.
-Report back physiotherapist if the gravitational -Adjust support level if the participant successfully
support level is required to be adjusted. finishes the trial with less effort.
- Record participant unassisted tasks in the end of - Check the participant movements during
the session. unassisted tasks in the end of session.
-Check if the collected data is meaningful and saved
properly.
Figure 6.7: GO-SAIL+ session flow diagram.
inability to effectively extend the elbow in reaching and impaired opening and closing of
the hand; iv) FES produces movement through a functional range; v) able to comply with
study protocol that includes ability to position arm at start point of each trial; vi) able
to communicate effectively; vii) able to provide written informed consent. The exclusion
criteria for participants were: i) any active device implant; ii) a metal implant in the affected
upper limb; iii) uncontrolled epilepsy; iv) pregnancy and lactation; v) any serious or unstable
medical, physical or psychological condition or cognitive impairment that would compromise
the subject’s safety or successful participation in the study; vi) requirement of an interpreter;
vii) current participation in another study involving physical rehabilitation of the arm.
The set-up procedure also followed steps 1) to 3) above, and was followed by 60 minutes
practising of a subset of FES-assisted functional reach and grasp tasks dictated by clinical
need. At the beginning and end of each intervention session, each stroke participant com-
pleted two tasks without FES assistance (high light switch and button pressing tasks). Note
that detailed flow these stages is shown schematically for both engineering and clinical side
of team in the Figure 6.7. Due to the identification and test of electrode array, there is
a five minutes difference occurred. Clinical assessments, described in Section 6.3.3.3, were
carried out before and after the intervention. The assessments were conducted according to
standard protocol, by qualified physiotherapists who were independent of the study.
Table 6.1: Stroke participant demographic characteristics

Participant Gender Age (years) Side of paresis Time since stroke
1 M 54 L 35 months
2 M 51 L 64 months
3 F 47 L 60 months
4 M 43 L 96 months
Average 48.75 63.75 months
The health and safety measures employed in GO-SAIL+ were similar to those used in the
GO-SAIL projects (EP/G014078/1 and FoHS Ethics-4009). The only difference in safety
management is that the use of an electrode array was incorporated.
6.3.2 Unimpaired Participant Results
The model identification procedure of Section 6.2.4.2 was followed using the simplified Hw (s)
structure. In principle the hand and wrist identification procedure must be repeated for a
range of operating points. However, due to the dominant effect of spasticity and stiffness
outweighing variation in operating point conditions, satisfactory results were obtained by
performing a single identification test at the start of each session. Component Ka (s) was
selected as a proportional controller, yielding Cc,a = Ma Ka (s), where input-linearising con-
troller Ma was implemented using the identified arm dynamics. The feedback controller
Cc,w = Kw (s) = K̄w (s)(P W )† of Appendix E was employed, with K̄w (s) selected as a pro-
portional plus derivative controller. The resulting closed-loop dynamics Ga (s) and Gw (s)
6.3.2. Unimpaired Participant Results 93
are given by (6.9) and (6.10) respectively, where the latter simplifies to the form (Appendix
E.3). These were then used to design ILC operators La and Lw respectively within ILC
update (6.11). For simplicity, phase-lead structures were selected due to their ease of tun-
ing and previous successful use in clinical trials (Meadmore et al., 2014). This produces
La (s) = la diag{0, 1, 0, 1, 0}esλa , and by applying the design described in Appendix E.7, the
form Lw (s) = lw (P W )† esλw . These correspond to the update
 " # 
" # " # 01 " #
va,k+1 (t) va,k (t) l 01 ea,k (t + λa )
= + a 0 , λa , λw > 0


vw,k+1 (t) vw,k (t) ew,k (t + λw )
| {z } | {z } lw (P W )†
vk+1 (t) vk (t)
(6.13)
which is a transparent extension to that used in GO-SAIL (and given by equation (4.9))
Tests comprised 6 trials of each of the far reaching and low light switch tasks. The Saebo-
MAS support was set individually for each participant at the minimum level, which enable
them to satisfactorily complete the task when also assisted by FES. This meant they could
contribute maximum voluntary effort, whilst not being demotivated. In practice, this level
was achieved heuristically by the therapist based on a clinical experience and was adjusted
based on task difficulty, fatigue and spasticity. Results for Participant 1, as illustrated by
Figure 6.8, similar to expectations between trials stimulation shifts to the left while its am-
plitude increases, confirm improved tracking between trials k = 1 and k = 6 with summary
performance measures given in Table 6.2. Here φ2 and φ4 are the controlled shoulder and
elbow axes, and, although all wrist and hand axes were controlled, emphasis has been placed
on φ6 and φ11 (wrist and index finger extension) angles as 42% of the functional movements
of the hand involve the four fingers moving together (Ingram et al., 2008). Reference tra-
jectory Φ̂ was extracted in separate tests with 12 unimpaired participants, as reported in
(Freeman et al., 2015).
For the anterior deltoid and triceps, proportional gains were chosen between 2 and 3. For
the forearm muscles proportional gains were chosen between 1 and 1.2 and derivative gains
were between 0.3 and 0.5. These were selected so that closed loop dynamics Ga (s) and
Gw (s) approximate a pure delay, and hence emphasis was placed on reducing oscillation
and following the shape of a delayed copy of the reference. The pure delays could then be
removed by the ILC algorithm by setting them equal to the phaseleads, yielding la = 0.3 and
λa = 0.8s, lw = 0.3 and λw = 0.8s. The effect of fatigue and moderate to severe spasticity
was addressed by re-tuning the feedback control parameters, and reducing learning gains
λa , λw to sacrifice convergence speed for robustness. Results confirm satisfactory tracking
Table 6.2: Unimpaired participants assisted results.

Norm of Error
Task Trial No
kφ̂2 − φ2 k kφ̂4 − φ4 k kφ̂6 − φ6 k kφˆ11 − φ11 k
1 8.54 8.72 12.76 8.51
Far Reaching
6 3.45 6.34 6.55 5.02
P1
1 9.95 11.33 6.12 3.73
Low Light Switch
6 4.33 3.68 4.95 3.02
1 12.09 6.86 8.49 7.05
Far Reaching
6 7.92 4.68 4.90 4.99
P2
1 7.30 7.03 19.58 12.38
Low Light Switch
6 2.03 6.42 6.60 8.17
accuracy of the FES-controlled joint angles φ2 , φ4 , φ6 and φ11 , and hence feasibility of
the system in supports those joints most relevant to stroke participants. Note that, since
these unimpaired participants supplied no voluntary input this assistance did not necessary
translate into accurate completion of overall task (which would be expected in the case of
stroke participants since they typically maintain control over the unassisted joint angles.
6.3.3 Stroke Participant Results
For each stroke participant the set-up procedure steps 1) to 3) were performed at the be-
ginning of each of the 17 intervention sessions, using the same identification procedure and
control structures as the unimpaired case. Identification of the anterior deltoid axis, and the
hand and wrist dynamics, was performed each session, since these components are highly
dependent on electrode placement and hence vary widely betweens sessions. However, the
arm and shoulder dynamics took far longer to identify but varied less widely. Note that
their dependence on support level can easily be accounted by prior characterisation of Ks (θ)
for different support settings without recourse to re-identification. Therefore, due to time
constrains their identification was not repeated each session unless performance was deemed
unsatisfactory, which occurred only once. The four chronic participants each completed the
intervention over 6-8 weeks. They all had a right cerebral vascular event causing left paral-
ysis. With gravitational support, participants had varying degrees of volitional proximal
activity but all demonstrated an increasing deficit in activeness distally.
Changes in the FES-unassisted and FES-assisted performance recorded across the sessions
were analysed by calculating best-fit linear regression slopes of performance against session
number collapsed across all participants. One-tailed t-tests were then applied to assess
significance, these having been employed in previous studies including GO-SAIL (Hughes
et al., 2009; Meadmore et al., 2012, 2014). As shown in Figure 6.9, the trajectory tracking of
FES-assisted and unassisted tasks are completely different and a participant tries to perform
near reaching task with a compensated movement if there is no stimulation applied to the
6.3.3. Stroke Participant Results 95
shoulder and elbow. For the hand and wrist, the participant could not compensate the
movement of fingers and wrist with other joints given the severe spasticity.
6.3.3.1 FES-Unassisted Results
During FES-unassisted tasks each participant was only supported by gravitational support.
The level of support was set by the physiotherapist at a constant level during the first
treatment session according to each participant’s needs. The range of movement, defined as
the difference between maximum and minimum joint angles, was calculated for each FES-
unassisted task. Results in Table 6.4 demonstrate improved range of movement at all four
stimulated joints over the intervention. In particular mean improvements over the course
of the intervention were 5◦ in shoulder flexion (high light switch), 13◦ in elbow extension
(contralateral reach), 42◦ in wrist extension (near reach), and 34◦ in index finger extension
(far reach)(See Figure 6.10). Conforming to prior studies (Hughes et al., 2009; Meadmore
et al., 2012, 2014), one-tailed t-tests were applied to best-fit linear regression slopes of range
of movement against session number, and yielded p-values < 0.05 in all cases.
6.3.3.2 FES-Assisted Results
When assisted by FES, each participant was supported by both gravitational support and
FES according to their clinical need. The level of gravitational support was varied between
tasks based on physiotherapist observations and participant voluntary action. For all par-
ticipants the level reduced over the intervention. As shown in Figure 6.11 and Figure 6.12
improvements were seen in mean tracking accuracy for all four joints as was the case with
unimpaired participants. The results demonstrate the success of the control system for im-
proving movement accuracy during reaching and grasping tasks. As shown the stimulation
input changes while shifting. If between trials more input is required, it increases, other-
wise it decreases according to the error in previous trial. Participant 4’s low light switching
task is one of the examples in which there is no stimulation for the shoulder as the error is
negative. This means he/she is capable of lifting his/her shoulder without assistance of the
stimulation. However, the participant’s shoulder trajectory profiles aim to follow the refer-
ence between trials (due to the stimulation of the other muscles and adaptation mechanism).
This means he/she can perform more natural movement. Summary performance measures
are given in Table 6.3 and confirm that tracking accuracy increased between the first and last
ILC trials. For example, the norm of tracking error for all joints (last column) increased by
61.1%, attaining an accuracy on trial k = 6 which confirms that the overall movement was
performed to a sufficient level of precision to support functional movement. Since the first
trial (k = 1) corresponds to vk = 0, these results clearly show the improvement compared
with using proportional and proportional plus derivative feedback controllers alone.
Table 6.3: FES-assisted tracking results for stroke participants taken mid-way during
intervention (session 9)
Norm of Error
Task Trial 17
P
No kφ̂2 − φ2 k kφ̂4 − φ4 k kφ̂6 − φ6 k kφ̂11 − φ11 k kφ̂i − φi k
i=1
Drawer 1 16.6 10.8 14.9 8.47 192.8
P1
Closing 6 10.1 7.42 6.28 4.9 72.74
Far 1 24.9 9.94 9.77 7.78 196.68
P2
Reaching 6 19.1 5.01 5.36 6.52 89.12
Near 1 16 5.42 30.4 12.5 186.84
P3
Reaching 6 12 3.24 13.5 6.21 105.88
High Light 1 25.3 8.68 12.9 9.55 217.96
P4
Switch 6 37.6 5.37 5.42 5.52 125.64
6.3.3.3 Clinical Assessments
The primary outcome measures were the motor component of the FMA and ARAT, with
maximum scores of 66 and 57 respectively. Scores are shown in Table 6.4 with improvements
seen in both; for two participants the ARAT and for three participants the FMA improved.
A paired t-test was applied to the FMA scores but the p-value of 0.0596 is not deemed
significant.
Table 6.4: Stroke participant clinical assessment data

Action Research FES-Unassisted Range of Movement
Fugl-Meyer
Arm Test φ2 φ4 φ6 φ11
Base- Base- Base- Base- Base- Base-
Participant Post- Post- Post- Post- Post- Post-
Line Line Line Line Line Line
P1 8 9 26 33 6 14 30 33 30 50 11 23
P2 6 5 28 28 7 13 10 23 65 130 15 42
P3 7 3 30 35 2 16 16 29 59 73 6 44
P4 10 13 23 38 3 8 30 35 40 76 32 45
ttest p-value 0.0596 0.0054 0.0041 0.0322 0.0364
6.4 Discussion
The development of GO-SAIL+ is motivated by the findings of previous research described

in Chapter 4. This indicated that training multiple muscle groups simultaneously, produced
significant improvements, with participants reporting greater capacity to perform everyday
tasks at home, such as opening a drawer, stabilising and moving objects, and pressing light
switches (Meadmore et al., 2014). GO-SAIL+ system addressed the problem that current
home-based systems still stimulate too few muscles and use non-selective, large electrodes.
They also do not employ position feedback or model based control algorithms. This leads
to inadequate support during functional activities.
6.4. Discussion 97
The aim of this study is using GO-SAIL+ to establish the feasibility of combining state-of-
the-art technologies to enable people with stroke to practise goal-oriented functional tasks.
The system developed in this chapter incorporates VR, FES hardware, advanced sensing,
control and passive support. Compared with previous systems, this demonstrates a sub-
stantial development in the scope of technology for upper-limb rehabilitation.
A critical element of the system reported in this chapter is an electrode array and advanced
model-based control scheme that supports activities of daily living. Use of such arrays is
an emerging area, however existing control schemes are slow and imprecise since they do
not have a dynamic model connecting FES to resulting motion. In contrast, ILC has been
shown to provide precise control of hand and wrist movement by employing a model of the
hand and wrist, and learning from past experience (Freeman, 2014; Freeman et al., 2011;
Freeman, 2015). In this study the model-based array ILC framework is integrated into
the system to produce fine finger movements during training of everyday tasks. While the
study confirmed acceptance and positive outcomes, limitations included the small sample
size, absence of a control group or follow-up (due to time constraints).
A number of further developments are required before the system can be transferred to
patients’ own homes:
1. Expensive components (dSpace and touch table) should be replaced with affordable
alternatives.
2. The control scheme should be made autonomous to reduce set-up time, enable use
without an engineer, and adapt to physiological changes such as fatigue and spastic-
ity. A major challenge is to eliminate the need to perform model identification tests,
which are not feasible for the home environment, especially without the assistance
of a carer. A possible solution is to employ the multiple model adaptive framework
established in Brend et al. (2015) which incorporates a large number of ’candidate’
model representation which feasibly represent the system dynamics. It evaluates the
fitting accuracy using a Kalman filter bank, and switches the controller corresponding
to the most accurate model into closed loop. Robust performance banks are reported
in Freeman and French (2015), and initial results when applied to FES control of the
UE are given in Brend et al. (2015). Using this approximation, the models identified
in this chapter therefore can be used to populate the candidate model set.
3. If necessary, the SaeboMAS should be replaced with a more affordable alternative (e.g.
Bakx Magic Arm, Focal Meditech Balancer, Sammons Preston Stable Slide). Their
dynamics can also be represented in the form (6.7), and the new combined structure
(6.8) is used to construct the candidate model set.
These developments will be expanded upon in the next chapter.

6.5 Conclusions
Results confirm that FES, mediated by ILC, successfully assisted participants in completion
of functional tasks, and training transferred to tangible changes in motor performance. The
key findings were significant improvements in FES-unassisted performance with different
metrics. In addition, participants reported that the system was usable, enjoyable and mo-
tivating, and importantly that the intervention was effective in reducing weakness, leading
to changes in everyday activities at home.
This chapter has established the feasibility of using low-cost, user-friendly sensing ap-
proaches and an arm support mechanism that can be used in conjunction with FES-assisted
tasks. The next chapter will focus on translating the GO-SAIL+ system into participants’
homes, prior to conducting a randomised control trial.
Shoulder Elbow Wrist Index Finger
6.5. Conclusions
100 50
60 0
40
50 0 −20
20
0 −40
Joint angle (φ2)

Joint angle (φ4)
Joint angle (φ6)
0
Joint angle (φ11)

−50
0 5 10 15 20 0 5 10 15 20 0 5 10 15 20 0 5 10 15 20
Time (s) Time (s) Time (s) Time (s)
300 300 150 150
200 200 100 100
100 100 50 50
Pulsewidth (μs)
0 0 0 0
Pulsewidth (μs)
Pulsewidth (μs)
Pulsewidth (μs)
0 5 10 15 20 0 5 10 15 20 0 5 10 15 20 0 5 10 15 20
Time (s) Time (s) Time (s) Time (s)
20 30 15 15
20 10 10
10
||e2||
||e4||
||e6||
||e11||
10 5 5
0 0 0 0
1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6
Trial Trial Trial Trial
trial 1 trial 2 trial 3 trial 4 trial 5 trial 6 reference
Figure 6.8: Unimpaired Participant 1’s tracking results for far reach task
99
Ch. 6. GO-SAIL with Improved Hand Functionality
Figure 6.9: Comparison results for assisted tasks vs unassisted tracking.

100
6.5. Conclusions 101
(a) highlight switch task shoulder angle
(b) contra-lateral reach task elbow angle
(c) lateral reach task elbow angle
Figure 6.10: Statistical results for range of movement of unassisted tasks

(d) near reach task wrist angle
(e) far reach task index angle
Figure 6.10 (cont.): Statistical results for range of movement of unassisted tasks
6.5. Conclusions
Figure 6.11: Stroke Participant 1’s tracking results for FES-Assisted drawer closing task
103
Ch. 6. GO-SAIL with Improved Hand Functionality
Figure 6.12: Stroke Participant 4’s tracking results for FES-Assisted low light task
104
Chapter 7
GO-SAIL Compact System

Implementation
7.1 Introduction
In Chapter 6 the feasibility of an upper-limb stroke rehabilitation system with advanced

control and sensing technologies has been established. In particular, mean tracking errors
of less than 10◦ were produced, which translated into improved functional ability during a
clinical trial. However, there are inherent complications with the use of multiple sensors
due to mutual interference and subsequent degradation of accuracy. Furthermore, some of
the components are costly and large and thus remain ill-suited to transferring to the home.
Here principal examples are the PC, the array electronics, the touch table and the real-time
control board. Another example is that the real time hardware did not support wireless
communications which further adds to a difficulty in setting up the system for the patient or
carer. Finally, it is aimed to maximise performance before making the control system usable
without an engineer or physiotherapist. This chapter details the development of a prototype
system which addresses all these issues except the autonomous control system. Thus, the
system embeds the same functionality as GO-SAIL+, but in a compact and affordable form
that is suitable for deployment in the home setting. This system, termed GO-SAIL Compact,
has been realised by undertaking the following developments to the previous system:
• The touch-table has been replaced by a standard tablet PC which is a platform com-
monly used by older people.
• Kinect and Primesense have been replaced by a single, more convenient, sensor unit
with no loss of functional scope.
• The Odstock stimulator and bulky relay electronics have been eliminated by developing
a multichannel stimulator that provides all necessary stimulation channels without the
105
106 Ch. 7. GO-SAIL Compact System Implementation
need for switching.
• The dSPACE real-time control hardware has been exchanged for a far more affordable
platform with a small footprint, and wireless communication capability.
• The software has also been simplified by eliminating the need for MATLAB, with the
GUI being translated to a C# programme. This single programme replaces the two
C++ and MATLAB programmes used by GO-SAIL+ while retaining its functionality.
• An additional innovation has been to replace the polycarbonate electrode array with
a fabric electrode array of identical dimensions. The advantage of the latter is that is
is breathable and more flexible, thereby providing greater comfort to the user.
The aforementioned components represent a significant step towards producing a system

that patients can use independently in their own home.
7.2 System Overview
The GO-SAIL Compact system comprises the six principal components shown in Figure
7.1. The same arm support as GO-SAIL+ is used to de-weight the participant’s upper
extremity according to individual need. Note that this support could be replaced with a
more affordable alternative as discussed in Section 6.4, with examples including Ergorest
and SaeboMiniMAS. As with GOSAIL+, surface electrodes are positioned on the anterior
deltoid and triceps, and an electrode array is placed over the common extensor complex of
the forearm. The system trains the same task set as GO-SAIL+, comprising both real and
virtual movements. The GUI runs on the tablet PC/laptop and is used to assign parameters
and visualise results. It hence can be used by the carer, physiotherapist, or patient (via the
unimpaired arm) to select tasks and monitor training progression. The GUI presentation is
identical to GOSAIL+, however the presence of a single screen means that when performing
a virtual task, the interactive virtual task representation replaces the GUI for the time
taken to perform the task. This interactive virtual task representation is the same as that
developed for the GO-SAIL+. The depth camera is used to measure the position of joint
centres and is placed in front of the participant. The depth camera has been placed over
the laptop at a -45◦ angle to the user. Combined stimulator electronics and compact real-
time hardware are attached to the passive support as seen in Figure 7.1. In the future
development of the system it is intended that this hardware may be integrated in the users’
own clothing to maximise convenience. As with GO-SAIL+, there is also an over-ride stop
button connected to the real-time hardware which terminates trials with immediate effect.
The components in the system are now described and design choices justified.
Figure 7.1: GO-SAIL Compact system overview

1) Single sensor unit, 2) laptop/tablet PC, 3) fabric electrode array, 4) arm support, 5)
compact real time hardware and combined stimulator electronics.
7.2.1 Task Design
Similarly to GO-SAIL+, the system supports training of functional tasks which are aligned
with ADLs and comprise reaching, grasping and manipulating real objects. There are five
main tasks; closing a drawer, switching on a light switch, stabilising an object, button
pressing and repositioning an object. The position of each object relative to the patient
is the same as described in Section 7.2.1. The interactive virtual task representation is
used to present the button pressing tasks, but can also be employed to provide a graphical
background to aid in positioning the real objects. The area of the tablet PC/ laptop surface
is a limitation in terms of the scope of task that can be trained.
7.2.2 Software
In GO-SAIL+, the Kinect v1 sensor and Skeletal Viewer middleware software programme
computed the upper arm joint angles. Similarly, the PrimeSense sensor and EchoClient
middleware programme computed the hand and wrist joint angles. Both were programmed
in C++ since this platform was required to interface with the dSPACE hardware (via C
libraries). Furthermore, EchoClient was only available in a 64-bit architecture, while Skeletal
Viewer ran only on a 32-bit architecture. This meant that a TCP socket connection was
implemented so that data could be sent between the two programmes.
The change to a single depth sensor (Kinect v2) is discussed later in this chapter, and
is motivated by the fact that this hardware is able to be used with both the (updated)
Skeletal Viewer and EchoClient middleware programmes. The switch to this updated form
of Kinect sensor therefore entirely eliminates the PrimeSense unit. Moreover, accuracy is
also increased due to the greater resolution of the sensor which is detailed in Section 7.2.
However, this change impacts the software in a substantial manner, as does the replacement
of dSPACE hardware and MATLAB software with myRIO and LabVIEW respectively (as
discussed in Section 3.2.3).
Firstly a new version of Skeletal Viewer is required to interface with Kinect v2. This is
provided in both C++ and C# language formats and supports 64-bit architecture. This
middleware programme provides the same joint positions as the previous Skeletal Viewer,
with the addition of the thumb position. These joint positions hence match the definitions
given in Section 6.2.3, and the corresponding joint angles can therefore be calculated as
described in that section. As discussed, EchoClient interfaces with the Kinect v2, and
therefore the remaining joint angles can be computed as also described in Section 6.2.3.
Since both programmes support the same architecture and share mutual languages (either
C++ or C#), they can be integrated within a single software application. C# was chosen
since it provides a greater level of abstraction compared with C++ and embeds a greater
level of error checking at the cost of slightly reduced memory management performance. To
ensure application resembled the previous MATLAB GUI, the Microsoft Forms template
was selected within the Windows.NET Framework programming environment.
The next issue is communication with the real-time software running on the real-time hard-
ware platform. This communication is primarily required to kinematic data from the sensors
to the real-time device, but it is also necessary to send control parameters and also receive
data (such as the level of stimulation applied). As a first step in reducing the amount of
data to be sent wirelessly, it was decided to perform the joint angle calculations within
the C#/C# application. As detailed in Section 7.2, the platform chosen is myRIO. This
supports a number of communication protocols including TCP, as was used in GO-SAIL+.
However, the operating system used by myRIO is linux based and only supports TCP li-
braries defined in C#. Therefore this fact meant that the programming language employed
had to be C#.
Having defined the properties of the GUI application running on the patient’s laptop or
tablet, attention can now be paid to the software running on the MyRIO real-time hard-
ware platform. MyRIO can be programmed in LabVIEW, which is graphical programming
7.2.2. Software 109
environment similar to Simulink, but differing in that it was originally created for simple
data acquisition purposes rather than complex matrix computation. An important feature
of programmes developed using LabVIEW is that they can also be deployed as ‘standalone
applications’ which automatically load and run on the device after it is reset. This means
that once developed and tested, no software other than the GUI is required to be loaded in
order to commence using the system.
Having selected the LabVIEW environment, the issue of converting the previous real-time
code for use in this framework is now considered. The GO-SAIL+ GUI was programmed
in the Matlab ‘m-code’ language, and interfaced with the real-time hardware via read and
write functions supplied by dSPACE (in a library termed ‘c-lib’). The control scheme was
designed in Simulink then compiled to the dSPACE hardware with the m-file functions
controlling the data flow between Simulink and Matlab. The process of transforming this
to the LabVIEW environment is not straightforward since LabVIEW is not text-based,
and instead implements graphical programming structures (e.g. switch-case, for and while
loops). These aid visualisation of logical programming structures, but greatly lengthen the
time taken to translate code between environments. In addition, nested loops are extremely
difficult to convert to grapgical equivalents in LabVIEW. However, LabVIEW supports
mathscript blocks, which enable direct use of m-code. Therefore the philosophy employed
in developing the LabVIEW programme was:
1. Implement major MATLAB functions as components of a LabVIEW switch-case struc-

ture. Major functions here correspond to self-contained set-up procedures (e.g. de-
termining stimulation amplitude levels) and the running of experimental tests (e.g.
to identify the anterior deltoid axis, to identify model parameters, or to run an ILC
trial).
2. Within each component of the switch-case structure, employ LabVIEW ‘MathScript’

blocks to embed the remaining m-file computations. MathScript blocks support the
direct use of m-file functions, which enables the direct use of previous MATLAB code.
3. Implement each block of the previous Simulink layout within the relevant MathScript
block. The Simulink layout was previously used in conjuction with MATLAB, and
defined the real-time control structure compiled into the dSPACE hardware. This
structure therefore ran continuously, meaning that various switches were required
to ensure only the required control loops were in place at any one time. Since the
aforementioned switch-case structures govern each test in the LabVIEW structure, all
components of the Simulink programme are now absorbed within the aforementioned
Mathscript blocks in each switch-case structure.
In addition to this, the TCP code required to communicate with the C# application was
implemented using the TCP block library supplied by LabVIEW. The design used to create
a single threaded communication protocol in LabVIEW followed these steps:
1. Labview opens a TCP connection using a ‘TCP Open Connection Function’ block
specifying the address of the real-time device, and an available port number. It is
assumed that a TCP server within the Microsoft.NET application is ‘listening’ to this
port, and will accept the connection within a specified time interval (60 seconds).
2. The Microsoft.NET application sends data packages of either joint angles, control
parameters, or instructions to perform given tests. Each is preceded by a string which
denotes the LabVIEW switch-case structure which must handle the data which follows
this instruction. Accordingly, the LabVIEW programme next contains a ‘TCP Read’
block to extract the identification string, and then sends the subsequent data to a
‘INITIALISE’ switch-case structure which has a case corresponding to each possible
string. These cases place the data in the appropriate LabVIEW variables, and if the
string specifies that an experiment is required to be performed, they set a parameter
which activates the corresponding component of the ‘RUN’ switch-case structure (this
is the switch-case structure previously described).
3. If a test has been run, then the LabVIEW ‘RUN’ switch-case structure that has
been activated will count the number of samples, and terminate once they have been
concluded. On the final sample, a flag will be raised which denotes that the data must
be sent back to the Microsoft.NET application on the following sample. Once raised,
the flag activates a third switch case structure, which has a case for each possible test.
Each case uses a ‘TCP Write’ LabVIEW block to send back the variables associated
with the test that has just been run.
4. Finally, the LabVIEW programme closes the TCP connection if the programme stops.
Clearly the Microsoft.NET application implements the TCP structures and code necessary
to interface with those of the LabVIEW programme described above. In particular, having
sent a TCP command instructing that an experimental test be carried out, it also waits for
the requisite number of samples, before reading in the data it expects to be sent. During
this waiting period, it continues to send joint angle data every sample. The structure of the
LabVIEW and Microsoft.NET GUI applications is summarised in Figure 7.2.
7.2.3 Hardware
The GO-SAIL and GO-SAIL+ systems have been shown in previous chapters to provide
the necessary functionality to deliver rehabilitation. It is therefore important that the GO-
SAIL Compact system returns the same functionality as GO-SAIL+ despite its elements
being replaced by alternatives with minimal cost and size. The rationale for the choice of
hardware components is now discussed.
7.2.3. Hardware 111
1) Tablet PC/ Laptop 2) RT Hardware and

Stimulator Electronics
Wireless Card
Custom Made Data Transfer
Application Wireless
Interface(Visual C#) Module
3Gear

Mathematical
Software
Environment
(LabVIEW)
TCP
Single Sensor Unit

Interface
Pulse-width
Kinect
2 Modulation
1 Stimulator
De-weighting
Spring Support
(a) Software and signal flow
Sensor input
(point cloud)
1 2
READ INITIALISE RUN
Joint
Acquisition ! Controller u
Joint
Loop Message
Control c
TCP TCP Parameters
Parameters Control
! e
(c) Parameters
c x STORE
User
Data x Start Test CASE STOP
(p) GUI p e&u

Start
e&u
Input
Test
(x) GUI Output C#
e&u
SAVE
DATA
LabVIEW
(b) Detailed data acquisition
Figure 7.2: Software and signal flow diagram.
Real-Time Hardware
The real-time hardware affects all other components since it connects both the GUI appli-
cation and the stimulator electronics. There are a number of real-time platforms available
and the chosen real-time control hardware must support the following functions:
1. embed wireless communications via TCP/IP protocol,
2. run real-time programmes at least 20Hz sampling frequency,

3. support at least 26 PWM output channels to feed to the stimulator electronics,
4. support MATLAB scripting and library functions in order to expedite development,

and
5. be as compact and affordable as possible.
It is clear that the previous real-time device (dSPACE) could not provide the latter two
specifications. Real-time hardware selection for the GO-SAIL Compact system is based
on finding the best accuracy to utilize data and produce suitable signals, the ability to
implement complex control schemes with the signals, its portability, and price. There are a
variety number of devices able to support the GO-SAIL Compact system such as Arduino
and Raspberry Pi, however MyRIO delivers for greater processing power and function.
MyRIO is a real-time controller that meets the systems specification and can be programmed
in the graphical programming environment LabVIEW described above. Manufactured by
National Instruments, MyRIO embeds Reconfigurable Input/Output (RIO) architecture in
a small footprint. It is based on four components: a processor, a reconfigurable field-
programmable gate array (FPGA), inputs and outputs, and the LabVIEW graphical design
software. MyRIO is a member of National Instruments’ larger RIO range, which all have
similar architecture. Importantly, the device can produce suitable PWM signals that can be
amplified by the stimulator to provide the required FES. FPGA enables the user to adapt
input output functions according to special needs. This is a significant advantage since it
allows up to 40 channels of PWM to be supported, whereas the default FPGA ‘personality’
only provides 8 PWM channels. By reprogramming the FPGA, the GO-SAIL Compact
system is able to support the required 26 FES channels. Finally, an even smaller version of
the MyRIO is available in the form of a System on Module (SoM). This measures only 76.2
mm by 50.8 mm and is therefore suitable for future migration into a more compact or even
wearable form. In the future, smaller single chip solutions, e.g. ARM processors, may be
feasible, but these are not suitable for development.
Motion Tracking
As previously stated, the GO-SAIL Compact system replaces the Kinect v1 and PrimeSense
with a single Kinect v2 sensor unit. This is possible since the latter is able to support the
demands of both middleware programmes: EchoClient requires high resolution and Skeletal
Viewer requires long range. Kinect v2 meets the needs of both, and provides high depth
resolution of 512 x 424 at 30 frames per second within a long range capability. In addition,
its accuracy has been confirmed in Sharp et al. (2015), which demonstrated precise real-
time articulated hand tracking accuracy combined with fast reinitialization and model fitting
(Shotton et al., 2016). In GO-SAIL Compact, the Windows.Net Form application embeds
the necessary classes and data structures in EchoClient and Skeletal Viewer so that the
functionality of both is supported. This has previously been described in Section 7.2.2.
FES Hardware
The GO-SAIL+ used a four channel Odstock stimulator, and employed a multiplexor circuit
to route two of these channels to arbitrary, but separate, sets of array elements. Although,
this approach has been shown to be effective, it entails bulky hardware and also reduces
accuracy since each electrode element can only have one of only two levels of (non-zero)
stimulation. Therefore in GO-SAIL Compact, each electrode array will have an entirely
separate channel of stimulation that can be arbitrarily specified. This is expected to lead to
far higher accuracy since no constraints are placed on the achievable stimulation patterns.
Thus, a new stimulator is designed at the University of Southampton which is detailed in
the Appendix F.
The GO-SAIL Compact system embeds exactly the same control strategy as the GO-SAIL+
system which is detailed in Section 6.2.4. As previously described, the control algorithm
and data transfer protocol have been implemented using LabVIEW. In this section the
differences in terms of control parameters will be discussed.
7.2.4.1 Biomechanical Model
The input stimulation signal is now given by uw (t) = [u3 (t), · · · , u26 (t)]> since it includes
the 24 elements of the array. Since the multiplexor is omitted, the matrix W is set equal to
the identity matrix so that µ(t) = uw (t). As was assumed in Section 6.2.4.1, each element
of the electrode array may produce a moment about any of the hand and wrist joint axes,
which now leads to the simplified form
26
X
g̃i (u, Φ, Φ̇) = g̃i,j (uj (t), φi , φ̇i ), i = 6, . . . , 17. (7.1)
j=3
where hi (ui , t) is again a Hammerstein structure incorporating a static nonlinearity, hIRC,i (ui ),
hLAD,i (t). As in Chapter 7, the term Fm,i (φ, φ̇) models the multiplicative effect of the joint
angle and joint angular velocity on the active torque developed by the muscle. The moment
around axis i due to stimulation of electrode array element j is accordingly given by
g̃i,j (uj (t), φi , φ̇i ) = hi,j (uj , t), i = 6, · · · , 17, j = 3, · · · , 26. (7.2)
where hi,j (uj , t) is a Hammerstein structure incorporating a static nonlinearity, hIRC,i,j (uj ),
hLAD,i (t). Inserting (7.2) into (7.1) yields
g̃i (u, Φ, Φ̇) = gi (u(t), φi , φ̇i ) = hi (u, t), i = 6, · · · , 17 (7.3)
where hi (u, t) is a composite Hammerstein structure incorporating nonlinearity hIRC,i (u) =

P24
j=1 hi,j (uj , t), cascaded with linear activation dynamics, hLAD,i (t). Any appropriate form
of passive support structure used with GO-SAIL Compact can be assumed to take the form
Bs (Θ)Θ̈ + Cs (Θ, Θ̇)Θ̇ + Fs (Θ, Θ̇) + Gs (Θ) + Ks (Θ) = 0 (7.4)
where vector Θ = [θ1 , · · · θ5 ]> contains the joint angles of the support mechanism, Bs (·)
and Cs (·) are 5-by-5 inertial and Coriolis matrices, and Fs (·) and Gs (·) are friction and
gravitational vectors respectively. Vector Ks (·) comprises the moments produced through
gravity compensation provided by the spring, which take the form [k1 (θ1 ), 0, 0, 0, 0]> . When
connected to arm structure (6.1), a bijective mapping between joint angles, Θ = M (Φ),
yields the combined model
B(Φ)Φ̈ + C(Φ, Φ̇)Φ̇ + F (Φ, Φ̇) + G(Φ) + K(Φ) = g̃(u, Φ, Φ̇) − J > (Φ)h (7.5)
which has the same general form as that employed in both GO-SAIL and GO-SAIL+.
7.2.4.2 Control Scheme
In GO-SAIL Compact, ua = [u1 , u2 ]> is the stimulation applied to the shoulder and elbow,
and uw = [u3 , · · · , u26 ]> is the stimulation applied to the forearm muscles via the electrode
array. In addition, Φa = [φ1 , · · · , φ5 ]> and Φw = [φ6 , · · · , φ17 ]> again contain the joint
angles of the upper arm and wrist respectively. The simplified control scheme is shown
in Figure 7.3, in which the feedback controller is partitioned as Cc = diag{Cc,a , Cc,w } and
designed to establish stability and baseline tracking over each trial. As with GO-SAIL+,
the requirement to repeatedly perform a set of finite duration tasks with a fixed initial
arm position enables ILC to be utilised to improve tracking performance. ILC uses the
> , v > ]> in an attempt to
performance error from each trial to update the input vk = [va,k w,k
increase the accuracy of the subsequent attempt. On trial k, Φk (t) denotes the joint angles
and the associated error is given by ek = Φ̂ − Φk .
As with GO-SAIL+, it is not feasible to extend the global upper arm model identification
approaches to include the hand and wrist. Therefore a linearised model Hw (s) : uw,k 7→ Φw,k
is identified about a suitable operating point, as described in Section 6.2.4.2. Feedback
controller Cc,w = Kw (s) is then chosen to stabilise system Hw (s) to yield the resulting
vk +1,a
évk +1,a ù évk,a ù éLa 0 ù
êv ú = êv ú + ê úek vk +1,w h
ë k +1,w û ë k,w û ë 0 Lw û
Iterative learning controller vk,a Stimulated muscles dynamics
Human arm, hand and support model
ek
+ ua
Φ *+ éCc,a 0 ù +
é ga (ua ,Φa ,Φ& a ) ù τk
vk ,w && = B(Φ ) -1 æç τ - C (Φ ,Φ& )Φ& - F (Φ ,Φ& ) - G(Φ ) - K (Φ ) - J T (Φ )h ö÷ Φ k
+
ê 0 C ú ê & ú
+ Φ
- ë c,w û
+ uw k k k k k k k k k k k
+
ë gw (uw , Φw,Φw )û + è ø
Linearising feedback controller
Electrically stimulated muscle ü

ý Effect of voluntary action
All other muscles þ
Figure 7.3: Block diagram of the ILC feedback scheme.

Note that the dashed line denotes computer memory updating of ILC signal vk , L is diag{La , Lw } and Cc is
diag{Cc,a , Cc,w }
closed-loop dynamics
Gw : (Φ̂w + vw,k ) 7→ Φw : Φw (s) = (I + Hw (s)Kw (s))−1 Hw (s)Kw (s)(Φ̂w (s) + vw,k (s)). (7.6)
Many possible designs for Hw (s) exist, with the form Hw (s) = H̄w (s)P presented in Ap-
pendix E providing guaranteed error tracking properties and stability across all joints, Φw .
Also following the procedure used with GO-SAIL+, an ILC scheme is then implemented
in order to provide input vk such that the error is minimised, i.e. lim vk = vk? with
k→∞
vk? := minvk kΦ̂ − Φk k2 . This is achieved through the update structure
vk+1 = vk + Lek , v0 = 0, k = 0, 1, · · · (7.7)
where L = diag{La , Lw }. Inserting the Ga form of (6.9) and (7.6) into (7.7), together with
> >
ek (t) = ea,k (t)> , ew,k (t)> and Φ̂(t) = Φ̂a (t)> , Φ̂w (t)> yields relationships

ea,k+1 = (I − Ga La )ea,k , ew,k+1 = (I − Gw Lw )ew,k ,

va,k+1 = (I − La Ga )va,k + La (I − Ga )Φ̂a , vw,k+1 = (I − Lw Gw )vw,k + Lw (I − Gw )Φ̂w .
(7.8)
For the arm dynamics, design of La,i to satisfy kI − Ga,i La,i k < 1, i = 2, 4, guarantees
convergence of φi to zero error, and many suitable schemes are available, see (Freeman,
2014) and examples therein. Similarity, kI − Gw Lw k < 1 guarantees convergence of the
wrist and hand joints to zero error. Here a major advantage to the GO-SAIL Compact
system is the far increased range of operator Gw . This means that there is a far greater
chance of a suitable Lw existing to enable perfect tracking. If this is not possible, the
alternative form of kI − Gw Lw k < 1 may be employed to guarantee convergence of error
ew,k to the limiting solution I − Gw (Lw Gw )−1 Lw Φ̂w . Due to the increased range of Gw ,

the norm of this limiting solution will be smaller than that described in Chapter 7. Note
that the explicit design of Lw detailed in Appendix E can again be used, but now employing
W = I.
Following ethical approval (EP/MO26388/1 and ERGO 7710), GO-SAIL Compact has been
tested with a single unimpaired participant. The same experimental setup procedure as em-
ployed with GO-SAIL+ has been used. This included placing of electrodes, setting ampli-
tudes, performing identification tests to find β, γ and the static P matrix. After completing
the set-up procedure, the participant opens the application with his/her unimpaired arm
and starts a variety number of tasks as described in Section 6.2.1. Each task was repeated
a total of 6 times.
The some model identification procedure as GO-SAIL+ was followed using the simplified
model structure. Within the control design, Ka (s) was selected as a proportional controller,
yielding Cc,a = Ma Ka (s), where input-linearising controller Ma was implemented using the
identified arm dynamics. The feedback controller Cc,w = Kw (s) = K̄w (s)P † was employed
similarly to the previous system and a proportional gain was selected based on experimental
results. Here representative results are shown in Figure 7.4 and confirm that a larger value
leads to greater accuracy. However larger values lead to greater high frequency oscillation,
and also reduce performance of the subsequent ILC update. The resulting closed-loop
dynamics Ga (s) and Gw (s) are given by (6.9) and (7.6) respectively. These were then
used to design ILC operators La and Lw respectively within ILC update (7.7).
Norm of Error vs Kp
25
20
Norm of Joint Angle Error (%)
15
10
0
1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2 2.3 2.4
Proportional gain (Kp)
Figure 7.4: Norm of joint angle error and proportional gain.
For simplicity phase-lead structures were selected due to their ease of tuning and previous
successful use in clinical trials described in Chapters‘ 4 and Chapter 7. This produced
La (s) = la diag{0, 1, 0, 1, 0}esλa , and (from Appendix E.7) the form Lw (s) = lw P † esλw .
After defining the feedback controllers for hand and wrist extension and designing the ILC
7.3.1. Intervention Safety Management 117
operators, the drawer closing task was selected. Figure 7.5 illustrates the resulting tracking
over 6 trials for a single joint angle (the wrist), and shows the stimulation level which has
the highest amplitude and therefore can be assumed to most directly actuate this joint. It is
clear that the percentage error norm reduces over 6 trials. Since the first trial corresponds
to proportional control alone, it can be seen that ILC provides superior tracking to feedback
alone.
Joint Angle (Φ6)
50
−50
0 20 40 60 80 100 120 140 160 180 200
Time (s)
40
PWM (u16)
20
0
0 20 40 60 80 100 120 140 160 180 200
Time (s)
norm of error(%)
0.6
0.4
0.2
0
1 2 3 4 5 6
Trial
Reference Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6
Figure 7.5: Participant’s wrist data for opening hand.
The results in Figure 7.5 use a weighting matrix applied to place more emphasis on wrist and
index finger tracking. This weighting matrix was removed and the test repeated. Results
for this case are shown in Figure 7.6 and confirm satisfactory tracking across a wide range
of joint angles. Note that in GO-SAIL+ system, tests were performed with proportional
and derivative controller. Where proportional gains were chosen between 1 and 1.2 and
derivative gains were chosen between 0.3 and 0.5. Due to the increased range of operator
Gw , a proportional controller could enable perfect tracking. Therefore, proportional gains
were chosen between 2 and 3 similar to the anterior deltoid and triceps proportional gains
of the GO-SAIL+ system.
The safety management protocol employed for the GO-SAIL Compact was similar to that
used for the previous systems. The only significant difference between them was the use
of rate limiters. Since GO-SAIL Compact employed electrode array elements to provide a
greater resolution and, in general, a lower level of stimulation, rate limiters were found to
be unnecessary and were not implemented.
Joint angle error Joint angle error Joint angle error Joint angle error Joint angle error Joint angle error
50 300 1
PWM (u4)
200
Wrist
0 0.5
100
−50 0 0
0 10 20 0 10 20 1 2 3 4 5 6
100 300 1
PWM (u )
6
200
Little
50 0.5
100
0 0 0
0 10 20 0 10 20 1 2 3 4 5 6
100 300 1
PWM(u )
10
200
Ring
50 0.5
100
0 0 0
0 10 20 0 10 20 1 2 3 4 5 6
100 300 1
PWM(u )
6
Middle
200
50 0.5
100
0 0 0
0 10 20 0 10 20 1 2 3 4 5 6
100 300 1
PWM(u )
6
200
Index
50 0.5
100
0 0 0
0 10 20 0 10 20 1 2 3 4 5 6
100 300 1
PWM (u )
4
Thumb
50 200
0.5
0 100
−50 0 0
0 10 20 0 10 20 1 2 3 4 5 6
Time (s) Time (s) Trial
ref 1 2 3 4 5 6
Figure 7.6: Participant’s wrist/hand data for opening hand
7.4 Discussion
Previous research described in Chapters 4 and 6 applied FES to a selection of upper-limb

muscles to support ADLs. What remained unclear however is how to implement these
systems in a home-based environment, since no home-based systems existed with which
support wrist, hand, forearm and upper upper movement in a precise manner using sensory
feedback to moderate the control action. Thus, the results in this chapter seeked to develop
a system which could be transferred to patients’ own homes. The initial results of this
case-study confirm its operation and are in keeping with the previous observational studies
conducted with GO-SAIL and GO-SAIL+.
The aim of the experiments described in this chapter was to establish the feasibility of a com-
pact system combining state-of-the-art technologies to enable people with stroke to practise
7.4. Discussion 119
goal-oriented functional tasks at their home. The system developed in this chapter incorpo-
rates VR, FES hardware, advanced sensing, control and passive support. Compared with
the previous GO-SAIL+ system, this demonstrates the same functional scope of technology
for upper-limb rehabilitation, but in a far smaller and more affordable package. While the
study confirmed acceptance and positive outcomes, limitations included the small sample
size, absence of a control group or follow-up (due to time constraints).
Despite limited testing, GO-SAIL Compact improved the usability with its standalone ap-
plication however it still requires the presence of an engineer. Furthermore, the set-up time
is expected to decrease with use of the standalone application. From the compactness points
of view the GO-SAIL Compact has satisfied the requirements for a home-based application
as seen from Table 7.1. It is approximately 100 times smaller than GO-SAIL+ system. To-
gether with the previous results, these figures show that GO-SAIL Compact has important
implications for developing an effective low-cost home-based upper-limb stroke rehabilitation
device.
A number of further developments are clearly indicated following on from GO-SAIL Com-
pact, including:
1. Embedded the hardware in fabric clothing with initial results in this direction reported
in Yang et al. (2014) and Freeman et al. (2016).
2. Replacing the passive support with a more affordable and compact alternative such as
Dunning et al. (2015).
3. Implementation of more autonomous controllers to ease tuning and reduce dependence

on the engineer. For this purpose, an Estimation-based Multiple Model Switched
Adaptive Control (EMMSAC) may be implemented as detailed in Section 6.4.
These avenues are further expanded in the next chapter.
Table 7.1: GO-SAIL+ and GO-SAIL Compact comparisons

System Component W(mm) D(mm) H(mm) Cost
Display Touch Table & Screen 1370 1720 1190 £5172
Processor PC 200 500 430 £1033
GO-SAIL+
Real-Time DS 1104 185 107 50 £9500
Total £15702
Display
Laptop 366 259 25 £689
GO-SAIL C Processor
Real Time myRIO 136.6 86.6 25.6 £500
Total £1189
7.5 Conclusions
This study set out to assess the feasibility of realising GO-SAIL+ in a form suitable for home
use. The findings from this study make several contributions to the current literature. The
developed system implements a compact non-contact sensor, real-time hardware and display
tools. The present case study confirms the previous findings of GO-SAIL+ and contributes
additional evidence that suggests that a home-based upper-limb system which stimulates
anterior deltoid, triceps and, wrist and hand extensors is feasible. One of the limitations of
the study is that the system has not been tested and evaluated with stroke participants at
home due to time limitations, therefore there is no information yet to conclude if the system
can be used easily in the home environment.
Chapter 8
Conclusions and Future Research
The objective of this thesis was to develop an innovative low-cost upper-limb stroke re-
habilitation system for home-based therapy. This was motivated by design limitations of
current home-based FES stroke rehabilitation systems which could not provide assistance to
sufficient muscles in the upper-limb or use the accurate controllers needed to fully support
training of ADLs and thereby deliver effective rehabilitation. Prior studies had shown that
the few model-based FES schemes in the literature yielded improved movement accuracy.
Similarly, the introduction of electrode arrays had demonstrated a potential for greater stim-
ulation resolution and simplified set-up procedures. However, the feasibility of combining
these ingredients had yet to be established since there existed neither model-based controlled
devices designed for home-use nor model based controllers employing electrode arrays.
The primary challenge to achieve the aims in the programme of research undertaken was,
therefore, to combine and assess model-based control with electrode arrays, within a home-
based environment. Inevitably, this required overcoming a range of technical challenges.
The system has developed towards a take home implementation and the principal stages
which have advanced the field to achieve this as are as follows:
1. To implement and assess ILC control schemes in which FES is applied to 3 muscle
groups within the upper-limb;
2. To determine the accuracy and usability of non-contact sensors in a compact form,

while extending the scope of upper-limb stroke rehabilitation;
3. To develop electrode-array hardware and control implementation needed to provide

greater assistance with ease of use;
These advanced the field of stroke rehabilitation as the feasibility of two systems were
clinically evaluated and three different systems implementing the ILC controlled FES on
121
122 Ch. 8. Conclusions and Future Research
multiple muscles have been designed. These systems quantify the clinical performance with
a range of ADLs. That is to say a better rehabilitation for complex tasks is achievable.
Furthermore, developmental stages address the aims of the programme which are aligned
with thesis chapters, and appraisal of each is provided next.
Chapter 2 provided a general review of stroke rehabilitation and highlighted that amongst
the greatest level of tracking performance in upper limb stroke rehabilitation had been
provided using a model-based control technique termed iterative learning. This controller
was integrated within a system called SAIL, which also used an exoskeletal support device
and displayed virtual tasks to the patient. FES was applied to two muscles to mediate
accurate tracking movement without any grasping. Once each task was completed, the
patient’s arm was returned to the initial point and ILC was used to compute the stimulation
to be applied on the next attempt using information from the error on the previous attempt.
ILC was particularly suited to this task as it employed the same philosophy of learning over
repeated task attempts. If the patient showed improvement with each attempt, the level
of stimulation applied on the next trial may be reduced. Therefore, the voluntary effort of
the patients can be maintained. This procedure showed that ILC could provide effective
rehabilitation, however, the functional scope of SAIL was limited and robotic systems are
clearly inconvenient for transferring to patients’ homes.
The thesis used SAIL as a starting point to transferring effective FES-based technology
to the home. The first step was to remove the exoskeleton and replace it with a passive
support. This, in turn, meant reformulating the underlying dynamic model linking applied
FES and resulting movement. Moreover, this led to a requirement of new sensors to replace
those previously embedded within the exoskeleton. An additional technical challenge was
to exchange the predefined virtual trajectory-following tasks with more natural functional
movements. The resulting system developed within this thesis was termed GO-SAIL, and
was detailed in Chapter 4. A particular innovation was to harness the potential of new
non-contact sensing technology with model-based control. This took the form of a long-
range non-contact sensor (Kinect) and an electro-goniometer. The developed system was
applied in a clinical trial with six stroke patients. The trial participants were invited for
18 treatment sessions; their performance was measured using two clinical outcomes (FMA
and ARAT) both pre- and post-intervention, as well as by analysing unassisted tracking
of five tasks at the start and end of each session. The range of unassisted movements of
patients was shown to increase over the treatment sessions. The FMA scores of the patients
have shown clinically significant improvements indicating that their level of impairment
has reduced. The measurements (discussed in Chapter 4) have shown a positive outcome
from the intervention, which confirms that the GO-SAIL system is a promising rehabilitation
method in the field of upper extremity rehabilitation for chronic stroke. However, the system
did not encompass hand and wrist movement, and hence did not fully support functional
tasks.
123
Subsequent development focussed on the ability of FES electrode arrays to improve the
resolution of applied stimulation and potentially provide more accurate hand movement.
However, its potential for closed-loop control was restricted by limited sensing technology
suitable for home-based stroke rehabilitation. Thus, Chapter 5 undertook a review fo-
cusing on sensing technology and comparing and critically evaluating the most promising
technologies. The systems with the greatest potential were selected, and a more detailed
experimental comparison was undertaken, necessitating software implementation and ex-
traction of joint angles. Experiments were designed and conducted to compare short-range
non-contact sensors and results confirmed feasibility and accuracy within a range of ±10◦ .
Thus, this has opened the door to the use of an electrode array to enable to be achieved
using FES. This is a major step in developing a system that can train ADLs.
Chapter 6 then addressed the next critical issue of combining the technical elements into
an experimental system. This led to the second system which integrated a short-range
non-contact sensor (PrimeSense), an electrode array and a touch table. For this pur-
pose (i) a custom made client/server model has been developed in order to provide on-line
hand/wrist data, (ii) custom made touch table interface software has been developed, (iii)
electrode-array identification procedures and control schemes have been designed to sup-
port hand/wrist tracking. The developed system was applied in a clinical trial with four
stroke patients which involved 17 intervention sessions. The outcomes of the study were
also positive. At the end of the trials, one of the participants was able to perform the most
challenging task without any anti-gravity support and stimulation. Hence the feasibility of
electrode array based FES and ILC has been proven and successfully employed in clinical
studies. However, the second system was also not suitable for transferring to patients’ home.
Thus, the final step in the research in this thesis focused on compacting the system as
well as improving it with newer technologies. For this purpose the system (i) replaced
previous sensors with a single non-contact sensor (Kinect v2) which is capable of providing all
upper-limb data with higher resolution, (ii) implemented a compact real-time and stimulator
hardware which support higher precision in electrode-array stimulation, (iii) substituted the
PC and touch table with a tablet PC/laptop which reduced costs. Due to the limitation on
time, the system was only applied on one unimpaired subject to investigate usability and
indicate the performance. Future research will involve the use of this system in clinical trials
with stroke participants.
In summary, the research in this thesis has developed an effective low-cost home-based FES
system by utilising innovative technologies. With the aid of advanced control methods, and
innovative technologies, the feasibility of effective and compact rehabilitation systems for
use in patients’ homes has been shown. However, the research has also highlighted that the
controller tuning and identification procedures are critical in determining accuracy. This
is a limitation of the systems developed in this thesis since it adds time constraints and
requires an engineer.
124 Ch. 8. Conclusions and Future Research
8.1 Future Work
In order to improve the achieved results and increase usability, further research within this
project can be divided into short-, medium- and long-term goals. The short term future
work will focus on further optimisation of the GO-SAIL Compact system and testing it
with stroke participants, firstly in lab-based clinical environment and secondly in patients’
homes. Furthermore, the future developments detailed in Section 7.4 should be implemented
in the short term. The EMMSAC framework could replace the current control scheme, and
electronics could be amalgamated into fabric clothing and a low-cost passive support could
be used for reducing the cost and footprint of the passive support. This will help in further
compacting the system, as the more compact the system, the greater the potential for
transferability to the home and daily use. It will also make controller tuning autonomous
and reduce the need for identification tests.
(a) Oculus Rift mounted Leap Motion (b) Oculus Rift Vision with Leap Motion
Figure 8.1: Leap Motion integrated with Oculus Rift taken from Leap Motion (2014)
The medium-term future work will focus on analysing the feasibility of new technologies
such as smartphone/tablet sensing and portable VR systems. Portable VR systems such
as Oculus Rift could replace tablet PC, as well as higher resolution sensors will provide
better VR and position sensing. These are shown in Figure 8.1. However, there are certain
drawbacks associated with use of Oculus Rift and Leap-Motion. First of all, Oculus Rift and
Leap-Motion require PC connection and power which limits their usability. On the other
hand, Google ATAP (Project Tango) and Microsoft smartphone sensors can be investigated
for this purpose. Using these, patients will be able to use their smartphones/tablets within
the developed rehabilitation system. This tablet can be used with a simple 3D printed
plastic device in similar design with Google cardboard to produce VR. The device will
provide head position data via the gyroscope and accelerometers, relative hand, wrist and
elbow data via its IR sensors and associated algorithm for transferring to the real-time
controller. The only limitation is detecting the shoulder which can be given by gyroscope
and accelerometers of the real-time control hardware which could be amalgamated in fabric
clothing. This will result in a compact and efficient system with a capability to provide
enhanced motivation and feedback to the user. Modification of the system structure and
computational algorithms, as well as the experimental set-up, can also be implemented in
8.1. Future Work 125
the future.
Another medium-term future work will focus on extending usability by replacing the passive
support with a wearable soft-robotic device (Rus and Tolley, 2015; Majidi, 2014). In contrast
to rigid robotics, soft-robotics is transmissible to human motion if it is used appropriately.
This means that an adaptive assistive technology could be developed for upper-limb stroke
rehabilitation. Furthermore, the use of soft robotics has a potential to be the pervasive, low
cost and adaptable. However, wearable soft robotic technology is still under development
and need to be more powerful and more compact. When it becomes compact and affordable,
this technology is ripe for use within wearable fabric-clothing (Polygerinos et al., 2015a,b).
The long-term future work will centre of building and improving on the results from the short
and medium term goals. The most challenging goal is to adaptively implement the system
into patients’ daily living. This would mean that patients not only undertake effective
rehabilitation at home but also perform this rehabilitation during their normal activities
(i.e. combine assistance and therapy).
Appendix A
Identification Protocols
In order to employ the dynamic model given by equations for simulation or control, the pa-
rameters within this model is required to be identified. The method developed here builds on
previous approaches that have been applied to the rehabilitation. First kinematic parame-
ters appearing in the model are established, before both identification of passive parameters
(associated with rigid body dynamics of support) and active parameters (associated with
stimulated muscle).
The identification protocol of the anterior deltoid and the hand and wrist via an electrode
array will be explained in this appendix. Firstly, as it is gradually developed, the AD
identification is to be detailed.
Anterior Deltoid
Identification of the parameters appearing in equation is achieved via a series of tests, which
due to variable electrode placement, spasticity, fatigue, environmental and physiological
conditions, must be performed prior to each experimentation or treatment session. This
involves identifying the stimulated joint axis within the kinematic model (Freeman, 2012).
Thus, the orientation of the anterior deltoid axis has to be defined by rotations about the
fixed frame. This is achieved by stimulation of the anterior deltoid as described next. It is
achieved by applying a ramped 10 s FES signal to the anterior deltoid and fitting a plane to
the resulting movement. Transformations are then embedded in the kinematic chain to align
the φ2 axis to the plane normal. The system developed in this thesis is a following part of the
SAIL project and identification protocol is similar. However, the general coordinate frames
are placed in sensor’s general coordinate frames. It was defined from sensors in ARMEO
for the SAIL as shown in Figure A.1(a) and general coordinate frame of the Kinect for the
GO-SAIL as shown in Figure A.1(b).
The dynamic model of the systems includes rotations with variables to displace the anterior
127
128 Ch. A. Identification Protocols
deltoid axis so as to correspond with stimulated movement. After the maximum comfortable
level of stimulation is set and electrodes are positioned to produce maximum movement in
the required direction. A triangular ramp signal is applied to the anterior deltoid and the
movement of the elbow is recorded. Then a plane is fitted to the points traced out in
Cartesian space.
z
y
(a) SAIL (b) GO-SAIL
Figure A.1: Anterior deltoid identification with parameters γ, β.
Hand and Wrist
Hand and wrist structure is more complex than upper arm structure. Thus, it is unlikely
feasible to extend the upper arm model identification approaches to include the hand and
wrist. Numerous methods can be used to identify the hand and wrist dynamics. One of
the methods is choosing an operating point which corresponds to the k th trial, including
fitting a linear model Hw (s) : uw,k 7→ Φw,k about a suitable operating point to the trial data
set {uw , Φk } (Freeman, 2016), another one is selecting a model from a set identified over a
range of operating points at the beginning of the treatment session.
(a) First element (b) Second element (c) Resulting movement
Figure A.2: Hand and wrist identification in GO-SAIL+.
In GO-SAIL + a linearised model is implemented. The main limitation is the number

of stimulation channels. The linearised model allows us to employ superposition rule to
combine responses of each elements of the electrode array. Then, the combined response
matrix is used to find a solution to perform different gestures using optimisation. The most
129
suitable matches to the reference (initial and final positions of the hand and finger joints
of a gesture) selected to correspond two channels which provides better convergence to the
selected stimulation. An example of this solution corresponding linearised model which is
selected to perform a task is shown in Figure A.2.
Appendix B
This graphical user interface (GUI) realises control of the rehabilitation session of the GO-
SAIL system. It uses a similar structure to the SAIL system and the flow of the pages is
detailed below.
Start Screen
This page enables the data security and when a physiotherapist clicks the start button seen
from Figure B.1, the pop-up window asks for the user name and password.
Figure B.1: Start screen
131
132 Ch. B. Graphical User Interface
Patient details
Records the patients’ personal data and predefined variables for later use such as length of
the arm and side of stroke to set correct data flow e.g. sensor data.
Figure B.2: Page 1: profile page
Workspace
This section provides data from real-time sensor in 3D. The physiotherapist or engineer may
check the data flow from this workspace and SkeletalViewer as detailed in Subsection 4.2.2.
Figure B.3: Page 2: workspace page

133
Stimulation
This section includes two tests. These are 1) setting maximum stimulation values of three
muscle groups at constant pulsewidth (300µs) and 2) identifying the anterior deltoid axis as
explained in Subsection 4.2.4.
After identifying the anterior deltoid axis it is plotted in a graph as seen in Figure B.4.
There is also an experimental electrode array test designed for the future implementation.
Figure B.4: Page 3: stimulation page
Test
The test page includes all stimulation assisted and unassisted tasks. The button groups
divided into
• Task selection which is chosen by physiotherapist decide tasks described in Section

4.2.1,
• Assisted session details including patients Id, control parameter, trial and session
number and
• Unassisted data capture is designed for collecting task without FES in the beginning
and end of sessions.
134 Ch. B. Graphical User Interface
Figure B.5: Page 4: test page
Results
The results page allows the therapist and participants to review performance. This feedback
page is important to provide motivation through knowledge of results.
Figure B.6: Page 5: results page

Appendix C
Non-Contact Sensors for Hand and

Wrist Rehabilitation
In Chapter 5, a detailed review is conducted regarding to available sensor types and short
range sensors for tracking hand and wrist. It also includes evaluation of two chosen sensors.
For simplicity, hence, this appendix is detailed on the various non-contact sensors on the
market at the time. Please note, these sensors are also included in hardware comparison
Table 5.2.
Duo3D
Duo3D uses two PS3 cameras to sense hand position and joint centres. The most important
difference between Leap Motion and Duo3D is that Leap Motion processes circular data
while Duo3D processes joints. The company released Duo3D with synchronised PS3Eye
cameras including both software and hardware. Simultaneous frame capturing with these
cameras results in up to 184 stereo frames per second, or a total of 360 fps at 320x120
resolution (Duo3d, 2013). This was followed by the release of Duo mini which is an ideal
candidate for mobile projects, with a price of £154 in 2013.
Figure C.1: Duo3D taken from Duo3d (2013)
135
136 Ch. C. Non-Contact Sensors for Hand and Wrist Rehabilitation
DepthSense 325
Similar to PrimeSense, SoftKinetic produces both the hardware and software needed for
gesture recognition; in contrast to PrimeSense, however, it uses ToF depth sensors. Their
hardware products are called DepthSense (see Figure C.2), and are devices aimed at different
ranges and resolution requirements. These are available as complete cameras, modules for
integration into other devices, or as individual sensors. The DepthSense 325 sensor is very
specifically targeted at hand gestures, having a short range, and high resolution RGB camera
paired with a ToF depth sensor, with the price of £192 in 2013 (Softkinetic, 2013).
Figure C.2: DepthSense 325 taken from Softkinetic (2013)
Intel Creative Interactive Gesture Camera- RealSense
Intel Creative cameras are small and have a low-cost sensor for hand tracking (see Fig-
ure C.3). They are specifically built for close range tracking with a range from 200 mm to
1200 mm and a diagonal frame of view of 73 degrees. The SDK provides recognition of hand
postures and gestures. Gestures like grab and release can be implemented by examining the
openness of the palm and the fingertip positions according to their area and structure. The
system had a price of £77 in 2014 (Intel, 2014).
(a) Intel Creative Interactive Gesture Camera (b) Intel Creative RealSense
Figure C.3: Intel Creative Senz3D and RealSense F200 taken from Intel (2014)
137
CamBoard pico and CamBoard pico S
The Camboard pico (pmdtechnologies, Germany) is a 3D camera, which is manufactured

with the Infineon 3D Image Sensor, in light of pmd’s ToF depth sensing innovation. The
small structure and low power utilisation enables its integration into electronic devices like
note pads and tablets. The Camboard pico and the significantly more slender edition Cam-
board pico S with a dimension of 85x17x8mm are expected to be soon utilized by other
systems (Pmdtec, 2015).
Figure C.4: Pmdtechnologies pico and pico S taken from Pmdtec (2015)
Appendix D
Previous Leap Motion Experiment
This section evaluates the first generation of a potential hand and wrist sensing technology
of Chapter 5. The first model employed by Leap Motion is shown in Figure D.1, and includes
MCP and PIP finger joints. The kinematic variables provided by Leap Motion are the finger
position fi , direction vectors vi , palm position P , and orientation [xP , yP , zP ].
Figure D.1: Leap Motion hand kinematic model and global coordinate frame
139
140 Ch. D. Previous Leap Motion Experiment
D.1 Calculations
The DIP joints have been neglected due to their limited range and coupled action and the
length of two associated links have been combined for each digit. The thumb is considered
to have 2 revolute joints at its connection with the wrist. The vector from the fingertip to
the PIP joint is denoted vi for i = 1, 2, 3, 4 for the four fingers and i = 5 for the thumb.
The position of the MCP is denoted as ki and is calculated with respect to central palm
position. Likewise the wrist positions wi are calculated using a fixed displacement from the
palm centre. The position of the centre of palm is denoted as P with a coordinate frame
with axes z pointing away from k3 and y orthogonal to z and the palm plane. The variable
θi,1 represents the pitch angle of the hand, θi,2 and θi,3 are the flexion angles of the MCP
and PIP joints for joint, i, and θi,0 is the abduction angle between fingers. For calculating
the θi,2 and θi,3 variables, the dot product of two vectors is used given by,
v1 · v2 = |v1 | |v2 | cos θ (D.1)
then θi,2 and θi,3 can be derived as,
vi · (fi + li,3 vi ) = |v1 | . |fi + li,3 vi | cos θi,3

(D.2)
(fi + li,3 vi − ki ) · (wi − ki ) = |fi + li,3 vi − ki | . |wi − ki | cos θi,2
After the finger flexion/extension values θi,2 and θi,3 are derived, the abduction angles
between fingers can be calculated by using the finger direction vectors. Let
 
vi,x
vi =  vi,y  (D.3)
 
vi,z
then the angle between fingers i and i + 1 can be calculated by projecting vi and vi+1 onto
the plane which passes through the palm. The palm coordinate frame can be written as
[xP yP zP ] Then any vector x in the global frame can be written with respect to the local
palm coordinate frame as x0 using the relationship
x = [xP yP zP ] x0 + P (D.4)
D.1. Calculations 141
Hence in the palm frame vi becomes
vi0 = [xP yP zP ]−1 vi

 0 
vi,x
 0 (D.5)
=  vi,y


0
vi,z
and when projected onto the plane yP = 0, this gives an abduction angle of
0 0
   
vi,x vi+1,x
θi,0 = arccos  0  ·  0  (D.6)
   
0
vi,z 0
vi+1,z
Leap Motion provides the fingertip position fi , and hence inverse kinematics may be used
to check the joint angles derived from the finger vectors and palm vector. The kinematics
of the ith finger are shown in Figure D.2;
Figure D.2: Verification Model for Experiments
 
fi,x
fi =  fi,y  (D.7)
 
fi,z
Let fi0 denote fi when transformed with respect to the wrist coordinate frame, as shown in
0 and f 0 values are used for checking the flexion/extension
Figure D.2. Then transformed fi,z i,y
angles using
0
fi,z = li,1 . cos θi,1 + li,2 . cos (θi,1 + θi,2 ) + li,3 . cos θi,4
0
fi,y = li,1 . sin θi,1 + li,2 . sin (θi,1 + θi,2 ) + li,3 . sin θi,4
02 2
fi,z = li,1 . cos2 θi,1 + li,2
2
. cos2 (θi,1 + θi,2 ) + li,3
2
. cos2 θi,4 + li,1 .li,2 . cos θi,1 . cos (θi,1 + θi,2 )
+ li,1 .li,3 . cos θi,1 . cos θi,4 + li,2 .li,3 . cos θi,1 + θi,2 . cos θi,4
02 2
fi,z = li,1 . sin2 θi,1 + li,2
2
. sin2 (θi,1 + θi,2 ) + li,3
2
. sin2 θi,4 + li,1 .li,2 . sin θi,1 . sin (θi,1 + θi,2 )
+ li,1 .li,3 . sin θi,1 . sin θi,4 + li,2 .li,3 . sin (θi,1 + θi,2 ). sin θi,4
(D.8)
As sin2 θ + cos2 θ = 1, this gives rise to
02 02 2 2 2
fi,z + fi,y = li,1 + li,2 + li,3 + 2.li,1 .li,2 + li,1 .li,2 .(cos θi,1 . cos θi,2 + sin θi,1 . sin θi,2 )
+ li,1 .li,3 .(cos θi,1 . cos θi,4 + sin θi,1 . sin θi,4 ) (D.9)
+ li,2 .li,3 .(cos (θi,1 + θi,2 ). cos θi,4 + sin (θi,1 + θi,2 ). sin θi,4 )
Then using cos(A − B) = cos A. cos B + sin A. sin B
02 02 2 2 2
fi,z + fi,y = li,1 + li,2 + li,3 + 2.li,1 .li,2 + li,1 .li,2 .(cos θi,1 − θi,2 )
+ li,1 .li,3 . cos (θi,1 − θi,4 ) (D.10)
+ li,2 .li,3 . cos (θi,1 + θi,2 − θi,4 )
cos θi,2 and sin θi,2 can be computed from the above equation using sin2 θ + cos2 θ = 1. Then
θ1,2 can be calculated using the arctan 2 function.
θi,2 = arctan 2(cos θi,2 , sin θi,2 ) (D.11)
D.2 Experimental Test Results
Glove calibration is one of the most challenging tasks in the experiment because the calibra-
tion process can only be achieved using its own software. Examination of raw data shows
huge discrepancy with calibrated data. For this reason calibration is vital on the 5DT glove
based system. In this experiment goniometers have been used to provide accurate data with
which to calibrate the glove.
D.3 Leap-Motion Results
The Leap Motion Controller attempts to fit a cylinder to each finger by using a camera
and assuming one circular finger. Leap-Motion data drop out problems were encountered
D.3. Leap-Motion Results 143
during experiments. If the sensor detects two fingers together, it calculates two fingers as
one finger and it does not count it as a finger because of radius constraints of standard
fingers. Thus, static experiments were performed to define the limitation of the technology,
as shown in Figure D.3. These confirm that Leap Motion can provide accurate information
in the natural static posture for both abduction and extension/flexion joint angles. However,
in this static position experiment the glove does not provide accurate data for abduction
due to calibration problems for abduction data (See in Figure D.3(b)). Thus, the abduction
angles were manually approximated.
Grab movement is the hardest movement for Leap Motion to capture, because the system
defines finger tips through their length and width. Fist gestures used therefore difficult to
compute accurately. In the next test grab movement without full hand closing was recorded
and analysed. As seen in Figure D.4, Leap Motion and the 5DT data glove’s movement
paths match each other well. However, there remain differences between Leap Motion’s
joint angles and glove, primarily due to calibration.
In Figure D.5 it is seen that Leap Motion can accurately track the thumb-index abduction
angle. Using Leap Motion, abduction data is pivotal to define when Leap Motion data drop
out occurs, while collecting. As seen in Figure D.5 data does not drop out even when the
fingers are as close as 15-16 degrees abduction.
In Figure D.6 it is seen that Leap Motion tracks the index fingers angles accurately while
the other fingers stay in a natural hand position.
(a) Static joint angles
(b) Static abduction angles
Figure D.3: Finger angles for static position
Turquoise and blue lines represent Leap Motion’s flexion/extension and abduction/adduction joint angles, while the
red line represents glove’s flexion/extension values.
D.3. Leap-Motion Results 145
Figure D.4: Finger flexion angles for grab movement
The turquoise line represents Leap Motion’s flexion angle, while the red line represents the glove’s flexion angle
values.
Figure D.5: Abduction angle for grab movement only thumb and index
The blue line represents Leap Motion’s abduction angle, while the red line represents the glove’s flexion angle values.
Figure D.6: Index finger flexion angle for separated grab movement
Turquoise line represents Leap Motion’s flexion angle, while red line represents glove’s flexion angle values.
Appendix E
Selection of feedback controller

Cc,w and ILC operator Lw
Design of feedback controller Cc,w (s) for system Hw (s) : µk 7→ Φw,k is motivated by the
form of muscle dynamics (6.5), which suggests that on the k th trial a structure comprising a
static 12 × 24 mapping combined with uniform dynamics may be used to represent the hand
and wrist. In practice this assumption is supported by the presence of spasticity, inherent
stiffness of the muscular tendon structure, and the low bandwidth of required movements.
We therefore assume Hw (s) = H̄w (s)P where P is a static mapping and H̄w (s) is a SISO
system, and introduce the control structure:
Proposition E.0.1. The control action Cc,w : ew 7→ uw : uw = K̄w (s)(P W )† ew , where K̄w (s)
is a SISO system, applied to system Φw = H̄w (s)P W uw realises stimulation input
uw = Nw (s)u?w (E.1)
where u?w minimises a weighted norm of the tracking error, ew = Φ̂w − Φw , and the SISO
system
Nw (s) := (I + H̄w (s)K̄w (s))−1 H̄w (s)K̄w (s). (E.2)
The resulting closed-loop dynamics are
Φw = Nw (s)PP⊥W Φ̂w . (E.3)
where 12 × 12 matrix PP⊥W = P W (P W )† is the orthogonal projection onto the range of

PW.
Proof. Consider the weighted tracking error u?w = min uw kΦ̂w − Φw k2Q where weight Q =
147
148 Ch. E. Selection of feedback controller Cc,w and ILC operator Lw
(H̄w−1 )∗ H̄w−1 with (·)∗ the adjoint operator. This has solution
u?w = min uw kΦ̂w − Φw k2Q

= min kΦ̂w − H̄w P W uw k2Q
uw
= min kH̄w−1 Φ̂w − P W uw k2

uw
= (P W )† H̄w−1 Φ̂w . (E.4)
where (·)† is the pseudo-inverse operator. The proposed control action Cc,w = K̄w (s)(P W )†
realises
uw = K̄w (P W )† (Φ̂w − H̄w P W uw )

⇒ (I + K̄w (P W )† H̄w P W )uw = K̄w (P W )† Φ̂w
⇒ (I + K̄w H̄w )uw = K̄w (P W )† Φ̂w
⇒ uw = (I + K̄w H̄w )−1 K̄w (P W )† H̄w H̄w−1 Φ̂w
⇒ uw = Nw u?w
The corresponding closed-loop dynamics Gw (s) are
Φw = H̄w P W uw = H̄w P W (I + K̄w H̄w )−1 K̄w (P W )† H̄w H̄w−1 Φ̂w = Nw (s)P W (P W )† Φ̂w
Control action Cc,w = K̄w (s)(P W )† therefore provides a transparent method to achieve
satisfactory tracking performance for all joints Φw . This requires only that SISO feedback
controller K̄w (s) be selected to stabilise dynamics (E.2) but ensures stability of all 12 joints.
It also enables the following simplified ILC update design.
Proposition E.0.2. When applied to the system of Proposition E.0.1, the ILC update struc-
ture
vw,k+1 = vw,k + Lw ew,k (E.5)
with operator Lw = lw (Nw PP⊥W )† , lw ∈ (0, 1], enforces convergence to the minimum error
norm, i.e.
lim vk = v ? , v ? = min kΦ̂w − Φw k2 . (E.6)

k→∞ v
Furthermore, if K̄w (s) is tuned so that Nw (s) is a pure delay of λw seconds then Lw =
lw (Nw PP⊥W )† = lw esλw (PP⊥W )† = lw esλw (P W )† , and (E.5) is equivalent to the phaselead
update
vw,k+1 (t) = vw,k (t) + lw (P W )† ew (t + λw ) (E.7)

149
Proof. Substitute Lw = lw (Nw PP⊥W )† and Gw = Nw (s)PP⊥W into limiting error solution
I − Gw (Lw Gw )−1 Lw Φ̂w to give (I − PP⊥W )Φ̂w which is the orthogonal projection onto

the nullspace of Φ̂w . This is the minimum achievable error and hence solves (E.6). If
Nw (s) = e−sλw then Lw = lw (e−sλw PP⊥W )† = lw esλw (PP⊥W )† = lw esλw (P W )† with time-
based implementation (E.7).
Appendix F
Multichannel Stimulator for

GOSAIL Compact
The four channel Odstock stimulator amplified each PWM voltage signal using a separate
transformer. This approach cannot be expanded to the case of 26 channels (2 for the shoulder
and elbow, and 24 for the wrist and hand extensors via an array) due to the large size and
weight of each transformer. The alternative method is to generate a high voltage supply,
and then implement 26 switches. Such an approach is described in (Ilić et al., 1994), in
which Zener diodes and resistors are used to also control the current in each output. Here a
bi-phasis design is used to limit the build up in charge, however this can be replaced with a
simplified mono-phasic version if this issue is not deemed important. The final design used
in GO-SAIL Compact closely follows Ilić et al. (1994) shown in Figure F.1, but expands
it to implement 26 channels. In addition, every component has been implemented using
surface mount components in order to minimize the overall footprint. The final module
has a ribbon cable input comprising 26 PWM signals supplied by MyRIO, with 3.3 volt
amplitude. The output is via another ribbon cable comprising 26 FES channels which is
then routed to supply the 24 electrode array elements and 2 single pad electrodes. The
module is powered by a 9 volt battery, and the output is optimally isolated from the input
side via opto couplers. The voltage and current of the output can both be controlled via
potentiometers, and the controlled variable is the pulsewidth of each channel.
151
Ch. F. Multichannel Stimulator for GOSAIL Compact
Control
n:1 winding VC
Error Amplifier Comparator VBB
LS C RP
+ V E
Control LP
+ input
- V Oscillator Power switch
S - VB
VR lR
RS
R iS
Figure F.1: Electronics design of Ilić et al. (1994)
152
Bibliography
5dt (2010). 5DT products. [online] Retrieved from:

http://www.5dt.com/products/pdataglovemri.html. [Accessed: 2013-23-05].
Al-Gburi, A., French, M., and Freeman, C. T. (2013). Robustness analysis of nonlinear
systems with feedback linearizing control. In 52nd IEEE Conference on Decision and
Control, pages 3055–3060.
Alon, G., Levitt, A. F., and McCarthy, P. A. (2008). Functional electrical stimulation (FES)
may modify the poor prognosis of stroke survivors with severe motor loss of the upper
extremity: a preliminary study. American Journal of Physical Medicine & Rehabilitation,
87(8):627–636.
Baker, L. (2000). Neuromuscular Electrical Stimulation: A Practical Guide. Los Amigos

Research & Education Institute, Incorporated.
Baker, L., Palmer, E., Waters, R. L., and Chun, S. N. (2004). Rehabilitation of the arm
and hand following stroke–a clinical trial with BIONs. Conference proceedings : Annual
International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE
Engineering in Medicine and Biology Society, 6:4186–8.
Bassett, R. W., Browne, A. O., Morrey, B. F., and An, K. N. (1990). Glenohumeral muscle
force and moment mechanics in a position of shoulder instability. Journal of Biomechanics,
23(5):405–415.
Bernotas, L. A., Crago, P. E., and Chizeck, H. J. (1986). A discrete-time model of electrcally
stimulated muscle. IEEE Transactions on Biomedical Engineering, BME-33(9):829–838.
Biometrics (2012). Goniometers and torsiometers. [online] Retrieved from:

http://www.biometricsltd.com/gonio.htm. [Last accessed: 2016-17-08].
Bioness.com (2006). Bioness Inc.- H200 for Hand Paralysis. [online] Retrieved from:
http://www.bioness.com/H200 for Hand Paralysis.php. [Accessed: 2013-21-05].
Blana, D., Kirsch, R. F., and Chadwick, E. K. (2009). Combined feedforward and feedback
control of a redundant, nonlinear, dynamic musculoskeletal system. Medical & Biological
Engineering & Computing, 47(5):533–542.
153
154 BIBLIOGRAPHY
Blaya, J. A. and Herr, H. (2004). Adaptive control of a variable-impedance ankle-foot

orthosis to assist drop-foot gait. Transactions on Neural Systems and Rehabilitation
Engineering, 12(1):24–31.
Brend, O., Freeman, C., and French, M. (2015). Multiple-model adaptive control of
functional electrical stimulation. IEEE Transactions on Control Systems Technology,
23(5):1901–1913.
Burridge, J. H. and Ladouceur, M. (2001). Clinical and therapeutic applications of neuro-

muscular stimulation: a review of current use and speculation into future developments.
Neuromodulation: Technology at the Neural Interface, 4(4):147–154.
Cai, Z., Tong, D., Meadmore, K. L., Freeman, C. T., Hughes, A.-M., Rogers, E., and Bur-
ridge, J. (2011). Design & control of a 3D stroke rehabilitation platform. In International
Conference on Rehabilitation Robotics (ICORR), pages 1–6. IEEE.
Chang, Y.-J., Chen, S.-F., and Huang, J.-D. (2011). A kinect-based system for physical
rehabilitation: A pilot study for young adults with motor disabilities. Research in Devel-
opmental Disabilities, 32(6):2566–2570.
Christensen, H. W. (1999). Precision and accuracy of an electrogoniometer. Journal of

Manipulative and Physiological Therapeutics, 22(1):10–14.
Clark, R. A., Pua, Y.-H., Fortin, K., Ritchie, C., Webster, K. E., Denehy, L., and Bryant,
A. L. (2012). Validity of the Microsoft Kinect for assessment of postural control. Gait &
Posture, 36(3):372–377.
Cobos, S., Ferre, M., Ángel Sánchez-Urán, M., Ortego, J., and Aracil, R. (2010). Human
hand descriptions and gesture recognition for object manipulation. Computer Methods in
Biomechanics and Biomedical Engineering, 13(3):305–317.
Cogan, S. F. (2008). Neural stimulation and recording electrodes. Annual Reviev of Biomed-
ical Engineering, 10:275–309.
Coupar, F., Legg, L., and Pollock, A. (2012). Home-based therapy programmes for upper
limb functional recovery following stroke. Technical Report 5, Cochrane Collabration.
Cuccurullo, S. (2004). Physical medicine and rehabilitation board review. Demos Medical.
Cyberglove systems (2012). Cyberglove systems. [online] Retrieved from:

http://www.cyberglovesystems.com/. [Last accessed: 2016-17-08].
Dalkilic, M. (2013). Electrical stimulation: Physi-

cal therapy and rehabilitation. [online]Retrieved from:
http://www.fizyoterapistler.org/images/kataloglar/elektrostimulasyon.pdf. [Accessed:
2013-22-11].
BIBLIOGRAPHY 155
Davoodi, R. and Andrews, B. J. (1998). Computer simulation of FES standing up in

paraplegia: a self-adaptive fuzzy controller with reinforcement learning. Transactions
on Rehabilitation Engineering, 6(2):151–161.
De Kroon, J., Van der Lee, J., IJzerman, M., and Lankhorst, G. (2002). Therapeutic elec-
trical stimulation to improve motor control and functional abilities of the upper extremity
after stroke: a systematic review. Clinical Rehabilitation, 16(4):350–360.
De Kroon, J. R. and IJzerman, M. J. (2008). Electrical stimulation of the upper extremity in

stroke: cyclic versus EMG-triggered stimulation. Clinical Rehabilitation, 22(8):690–697.
Desrosiers, J., Malouin, F., Richards, C., Bourbonnais, D., Rochette, A., and Bravo, G.
(2003). Comparison of changes in upper and lower extremity impairments and disabilities
after stroke. International Journal of Rehabilitation Research, 26(2):109–116.
Dipietro, L., Sabatini, A. M., and Dario, P. (2008). A survey of glove-based systems and
their applications. Transactions on Systems, Man, and Cybernetics, Part C: Applications
and Reviews, 38(4):461–482.
Dunnell, K. (2008). Ageing and mortality in the UK. Technical report, Office for National
Statistics.
Dunning, A., Stroo, J., Radaelli, G., and Herder, J. (2015). Feasibility study of an upper arm
support based on bending beams. In IEEE International Conference on Rehabilitation
Robotics (ICORR), pages 520–525. IEEE.
Duo3d (2013). CL//DUO. [online] Retrieved from:http://duo3d.com/. [Accessed: 2013-21-

05].
Egglestone, S. R., Axelrod, L., Nind, T., Turk, R., Wilkinson, A., Burridge, J., Fitzpatrick,
G., Mawson, S., Robertson, Z., Hughes, A.-M., Ng, K. H., Pearson, W., Shublaq, N.,
Probert-Smith, P., Rickets, I., and Rodden, T. (2009). A design framework for a home-
based stroke rehabilitation system: Identifying the key components. Proceedings of the
3rd International ICST Conference on Pervasive Computing Technologies for Healthcare.
Electrotherapy Standards Committee (1990). Electrotherapeutic terminology in physical

therapy. American Physical Therapy Association.
Ernst, E. (1990). A review of stroke rehabilitation and physiotherapy. Stroke, 21(7):1081–

1085.
Erol, A., Bebis, G., Nicolescu, M., Boyle, R. D., and Twombly, X. (2007). Vision-based hand
pose estimation: A review. Computer Vision and Image Understanding, 108(1):52–73.
Exell, T., Freeman, C. T., Meadmore, K. L., Kutlu, M., Rogers, E., Hughes, A.-M.,
Hallewell, E., and Burridge, J. (2013). Goal orientated stroke rehabilitation utilising
electrical stimulation, iterative learning and microsoft kinect. In IEEE International
Conference on Rehabilitation Robotics (ICORR), 2013, pages 1–6.
156 BIBLIOGRAPHY
Feigin, V. L., Forouzanfar, M. H., Krishnamurthi, R., Mensah, G. A., Connor, M., Bennett,
D. A., Moran, A. E., Sacco, R. L., Anderson, L., Truelsen, T., et al. (2014). Global and
regional burden of stroke during 1990–2010: findings from the Global Burden of Disease
Study 2010. The Lancet, 383(9913):245–255.
Ferrarin, M., Palazzo, F., Riener, R., and Quintern, J. (2001). Model-based control of
FES-induced single joint movements. Transactions on Neural Systems and Rehabilitation
Engineering, 9(3):245–257.
Freeman, C. T. (2012). Constrained point-to-point iterative learning control with experi-

mental verification. Control Engineering Practice, 20(5):489–498.
Freeman, C. T. (2014). Electrode array-based electrical stimulation using ILC with restricted
input subspace. Control Engineering Practice, 23:32–43.
Freeman, C. T. (2015). Upper limb electrical stimulation using input-output lineariza-

tion and iterative learning control. IEEE Transactions on Control Systems Technology,
23(4):1546–1554.
Freeman, C. T. (2016). Control system design for electrical stimulation in upper limb re-
habilitation. Switzerland: Springer International Publishing.
Freeman, C. T., Exell, T., Meadmore, K. L., Hallewell, E., and Hughes, A.-M. (2015). Com-
putational models of upper-limb motion during functional reaching tasks for application
in FES-based stroke rehabilitation. Biomedical Engineering/Biomedizinische Technik,
60(3):179–191.
Freeman, C. T. and French, M. (2015). Estimation based multiple model iterative learning
control. In 54th IEEE Conference on Decision and Control (CDC), pages 6070–6075.
Freeman, C. T., Hughes, A. M., Burridge, J. H., Chappell, P. H., Lewin, P. L., and Rogers,
E. (2007). An experimental facility for the application of iterative learning control as an
intervention aid to stroke rehabilitation. Measurement and Control, 40(1):20–23.
Freeman, C. T., Rogers, E., Hughes, A.-M., Burridge, J., and Meadmore, K. (2012). Iterative
learning control in health care: Electrical stimulation and robotic-assisted upper-limb
stroke rehabilitation. IEEE Control Systems Magazine, 32(1):18 –43.
Freeman, C. T., Tong, D., Meadmore, K. L., Cai, Z., Rogers, E., Hughes, A. M., and
Burridge, J. H. (2011). Phase-lead iterative learning control algorithms for functional
electrical stimulation-based stroke rehabilitation. Part I: Journal of Systems and Control
Engineering, Proceedings of the Institution of Mechanical Engineers, 225(6):850–859.
Freeman, C. T., Yang, K., Tudor, J., and Kutlu, M. (2016). Feedback control of
electrical stimulation electrode arrays. Medical Engineering & Physics. In Press.
doi:10.1016/j.medengphy.2016.07.002.
BIBLIOGRAPHY 157
Frost, S. B., Barbay, S., Friel, K. M., Plautz, E. J., and Nudo, R. J. (2003). Reorganization
of remote cortical regions after ischemic brain injury: a potential substrate for stroke
recovery. Journal of Neurophysiology, 89(6):3205–14.
Fu, Q. and Santello, M. (2010). Tracking whole hand kinematics using extended kalman
filter. In IEEE Annual International Conference of the Engineering in Medicine and
Biology Society (EMBC), pages 4606–4609.
Gabr, U., Levine, P., and Page, S. J. (2005). Home-based electromyography-triggered stim-
ulation in chronic stroke. Clinical Rehabilitation, 19(7):737–745.
Getmyo (2013). MYO - The Gesture Control Armbandthalmic labs. [online] Retrieved from:
http://www.getmyo.com/. [Accessed: 2013-04-05].
Graham, J. (2013). The management and treatment of foot drop in stroke. British Journal
of Neuroscience Nursing, 9(1):1–5.
Graham, J. (2014). Technology for rehabilitating the upper limb after stroke: the hand.
British Journal of Neuroscience Nursing, 10(Sup2):19–24.
Guadagnoli, M. A. and Lee, T. D. (2004). Challenge point: a framework for conceptualizing

the effects of various practice conditions in motor learning. Journal of motor behavior,
36(2):212–224.
Hanneson, D. (2000). NeuroMove NM900 - fast rehab to help relearn mus-

cle movement following stroke or spinal cord injury. [online] Retrieved
from:http://www.biomation.com/physiotherapy/neuromove.html. [Accessed: 2013-
20-05].
Hara, Y., Ogawa, S., Tsujiuchi, K., and Muraoka, Y. (2008). A home-based rehabilita-
tion program for the hemiplegic upper extremity by power-assisted functional electrical
stimulation. Disability and Rehabilitation, 30(4):296–304.
Heetderks, W. J. (1988). RF powering of millimeter-and submillimeter-sized neural pros-

thetic implants. IEEE Transactions on Biomedical Engineering, 35(5):323–327.
Heller, B. W., Clarke, A. J., Good, T. R., Healey, T. J., Nair, S., Pratt, E. J., Reeves,
M. L., van der Meulen, J. M., and Barker, A. T. (2013). Automated setup of functional
electrical stimulation for drop foot using a novel 64 channel prototype stimulator and
electrode array: results from a gait-lab based study. Medical Engineering & Physics,
35(1):74–81.
Hof Rehabilitationszentrum Weißer (2007). EMG triggered rehabilitation of complex

movements–biofeedback/stimulation system STIWELL med4.
Houwink, A., Nijland, R. H., Geurts, A. C., and Kwakkel, G. (2013). Functional recovery
of the paretic upper limb after stroke: who regains hand capacity? Archives of Physical
Medicine and Rehabilitation, 94(5):839–844.
158 BIBLIOGRAPHY
Hughes, A. M., Freeman, C. T., Burridge, J. H., Chappell, P. H., Lewin, P. L., and Rogers,
E. (2009). Feasibility of iterative learning control mediated by functional electrical stim-
ulation for reaching after stroke. Neurorehabilitation and Neural Repair, 23(6):559–68.
Hughes, A.-M., Freeman, C. T., Meadmore, K. L., Burridge, J. H., and Rogers, E. (2011).
Research and development work relating to assistive technology 2010-11: Presented to
parliament pursuant to section 22 of the chronically sick and disabled persons act 1970.
Technical report, Department of Health, London, UK.
Ilić, M., Vasiljević, D., and Popović, D. B. (1994). A programmable electronic stimulator
for FES systems. IEEE Transactions on Rehabilitation Engineering, 2(4):234–239.
Ingram, J. N., Körding, K. P., Howard, I. S., and Wolpert, D. M. (2008). The statistics of
natural hand movements. Experimental Brain Research, 188(2):223–236.
Intel (2014). Realsense - develop for 3 amazing cameras — intel developer zone. [Accessed:
2014-22-11].
Intercollegiate Stroke Working Party (2012). National clinical guideline for stroke. Technical
report, London: Royal College of Physicians.
Johnson, M. J. (2006). Recent trends in robot-assisted therapy environments to improve real-

life functional performance after stroke. Journal of NeuroEngineering and Rehabilitation,
3:29.
Kenney, L. P., Heller, B. W., Barker, A. T., Reeves, M. L., Healey, J., Good, T. R., Cooper,
G., Sha, N., Prenton, S., Liu, A., et al. (2016). A review of the design and clinical
evaluation of the shefstim array-based functional electrical stimulation system. Medical
Engineering and Physics.
Kern, H., Carraro, U., Adami, N., Biral, D., Hofer, C., Forstner, C., Mödlin, M., Vogelauer,
M., Pond, A., Boncompagni, S., et al. (2010). Home-based functional electrical stimulation
rescues permanently denervated muscles in paraplegic patients with complete lower motor
neuron lesion. Neurorehabilitation and Neural Repair, 24(8):709–721.
Kim, D., Hilliges, O., Izadi, S., Butler, A. D., Chen, J., Oikonomidis, I., and Olivier, P.
(2012). Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In
Proceedings of the 25th annual ACM Symposium on User Interface Software and Tech-
nology, UIST ’12, pages 167–176.
Kim, S.-H., Park, J.-H., Jung, M.-Y., and Yoo, E.-Y. (2016). Effects of task-oriented training
as an added treatment to electromyogram-triggered neuromuscular stimulation on upper
extremity function in chronic stroke patients. Occupational therapy international.
Klauer, C., Schauer, T., Reichenfelser, W., Karner, J., Zwicker, S., Gandolla, M., Am-
brosini, E., Ferrante, S., Hack, M., Jedlitschka, A., et al. (2014). Feedback control of
BIBLIOGRAPHY 159
arm movements using neuro-muscular electrical stimulation (NMES) combined with a

lockable, passive exoskeleton for gravity compensation. Frontiers in Neuroscience, 8.
Kobayashi, H., Onishi, H., Ihashi, K., Yagi, R., and Handa, Y. (1999). Reduction in sub-
luxation and improved muscle function of the hemiplegic shoulder joint after therapeutic
electrical stimulation. Journal of Electromyography and Kinesiology, 9(5):327–336.
Kowalczewski, J. and Prochazka, A. (2011). Technology improves upper extremity rehabil-

itation. Progress in Brain Research, 192:147–59.
Krabben, T., Prange, G. B., Molier, B. I., Stienen, A. H., Jannink, M. J., Buurke, J. H., and
Rietman, J. S. (2012). Influence of gravity compensation training on synergistic movement
patterns of the upper extremity after stroke, a pilot study. Journal of NeuroEngineering
and Rehabilitation, 9(1):1.
Kuechle, D. K., Newman, S. R., Itoi, E., Morrey, B. F., and An, K.-N. (1997). Shoulder
muscle moment arms during horizontal flexion and elevation. Journal of Shoulder and
Elbow Surgery, 6(5):429–439.
Kurosawa, K., Futami, R., Watanabe, T., and Hoshimiya, N. (2005). Joint angle control
by FES using a feedback error learning controller. IEEE Transactions on Neural Systems
and Rehabilitation Engineering, 13(3):359–371.
Kwakkel, G., Kollen, B. J., van der Grond, J., and Prevo, A. J. (2003). Probability of
regaining dexterity in the flaccid upper limb impact of severity of paresis and time since
onset in acute stroke. Stroke, 34(9):2181–2186.
Kwakkel, G., Winters, C., Van Wegen, E. E., Nijland, R. H., Van Kuijk, A. A., Visser-Meily,
A., De Groot, J., De Vlugt, E., Arendzen, J. H., Geurts, A. C., et al. (2016). Effects of
unilateral upper limb training in two distinct prognostic groups early after stroke: the
explicit-stroke randomized clinical trial. Neurorehabilitation and neural repair, 30(9):804–
816.
Lange, B., Chang, C.-Y., Suma, E., Newman, B., Rizzo, A. S., and Bolas, M. (2011).
Development and evaluation of low cost game-based balance rehabilitation tool using the
Microsoft Kinect sensor. In IEEE International Conference of the Engineering in Medicine
and Biology Society (EMBC), pages 1831–1834.
Langhorne, P., Coupar, F., and Pollock, A. (2009). Motor recovery after stroke: a systematic
review. Lancet Neurology, 8(8):741–54.
Langhorne, P., Widen-Holmqvist, L., et al. (2007). Early supported discharge after stroke.
Journal of Rehabilitation Medicine, 39(2):103–108.
Le, F., Markovsky, I., Freeman, C. T., and Rogers, E. (2010). Identification of electrically
stimulated muscle models of stroke patients. Control Engineering Practice, 18(4):396–407.
160 BIBLIOGRAPHY
Leap Motion (2013). Leap Motion. [online] Retrieved from: https://www.leapmotion.com/.

[Accessed: 2013-21-05].
Leap Motion (2014). Leap motion VR. [online] Retrieved from:

https://www.leapmotion.com/vr. [Accessed: 2014-01-09].
Lee, G. (2013). Effects of training using video games on the muscle strength, muscle tone,
and activities of daily living of chronic stroke patients. Journal of Physical Therapy
Science, 25(5):595–597.
Leonardi, F., Covi, D., Petri, D., and Stoppa, D. (2009). Accuracy performance of a time-
of-flight cmos range image sensor system. Transactions on Instrumentation and Measure-
ment, 58(5):1563–1570.
Levin, M. F. (1996). Interjoint coordination during pointing movements is disrupted in

spastic hemiparesis. Brain, 119(1):281–293.
Lo, A. C., Guarino, P. D., Richards, L. G., Haselkorn, J. K., Wittenberg, G. F., Federman,
D. G., Ringer, R. J., Wagner, T. H., Krebs, H. I., Volpe, B. T., et al. (2010). Robot-
assisted therapy for long-term upper-limb impairment after stroke. New England Journal
of Medicine, 362(19):1772–1783.
Loeb, G. E., Richmond, F. J. R., and Baker, L. L. (2006). The BION devices: injectable
interfaces with peripheral nerves and muscles. Neurosurgical Focus, 20(5):E2.
Lynch, C. L. and Popovic, M. R. (2008). Functional electrical stimulation. IEEE Control

Systems Magazine, 28(2):40–50.
Mackay, J., Mensah, G. A., Mendis, S., and Greenlund, K. (2004). The atlas of heart disease
and stroke. World Health Organization.
Majidi, C. (2014). Soft robotics: a perspectivecurrent trends and prospects for the future.
Soft Robotics, 1(1):5–11.
Malešević, N. M., Maneski, L. Z., Ilić, V., Jorgovanović, N., Bijelić, G., Keller, T., Popović,
D. B., et al. (2012). A multi-pad electrode based functional electrical stimulation system
for restoration of grasp. Journal of NeuroEngineering and Rehabilitation, 9(1):66.
Mangold, S., Schuster, C., Keller, T., Zimmermann-Schlatter, A., and Ettlin, T. (2009).
Motor training of upper extremity with functional electrical stimulation in early stroke
rehabilitation. Neurorehabilitation and neural repair, 23(2):184–90.
Mann, G., Taylor, P., and Lane, R. (2011). Accelerometer-triggered electrical stimulation for
reach and grasp in chronic stroke patients a pilot study. Neurorehabilitation and Neural
Repair, 25(8):774–780.
BIBLIOGRAPHY 161
Mann, G. E., Burridge, J. H., Malone, L., and Strike, P. (2005). A pilot study to investigate
the effects of electrical stimulation on recovery of hand function and sensation in subacute
stroke patients. Neuromodulation: Technology at the Neural Interface, 8(3):193–202.
Martin, R., Lorbach, M., and Brock, O. (2014). Deterioration of depth measurements
due to interference of multiple rgb-d sensors. In IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS 2014), pages 4205–4212. IEEE.
Meadmore, K., Exell, T., Hallewell, E., Hughes, A., Kutlu, M., Benson, V., Burridge, J.,
Rogers, E., and Freeman, C. (2014). The application of precisely controlled functional
electrical stimulation to the shoulder, elbow and wrist for upper limb stroke rehabilitation:
A feasibility study. Journal of NeuroEngineering and Rehabilitation, 11(1):105.
Meadmore, K. L., Hughes, A.-M., Freeman, C. T., Cai, Z., Tong, D., Burridge, J. H., and
Rogers, E. (2012). Functional electrical stimulation mediated by iterative learning control
and 3D robotics reduces motor impairment in chronic stroke. Journal of NeuroEngineering
and Rehabilitation, 9:32.
Mehrholz, J., Pohl, M., Platz, T., Kugler, J., and Elsner, B. (2015). Electromechanical and
robot-assisted arm training for improving activities of daily living, arm function, and arm
muscle strength after stroke. The Cochrane Library.
Meida, A. (2000). Physical performance tests after stroke. Archives of Physical Medicine
and Rehabilitation, 79(December):519–525.
Meilink, A., Hemmen, B., Seelen, H. A. M., and Kwakkel, G. (2008). Impact of EMG-
triggered neuromuscular stimulation of the wrist and finger extensors of the paretic hand
after stroke: a systematic review of the literature. Clinical rehabilitation, 22(4):291–305.
Merrill, D. R., Davis, R., Turk, R., and Burridge, J. H. (2011). A personalized sensor-
controlled microstimulator system for arm rehabilitation poststroke. Part 1: System ar-
chitecture. Neuromodulation : Journal of the International Neuromodulation Society,
14(1):72–9; discussion 79.
Metcalf, C. D., Robinson, R., Malpass, A. J., Bogle, T. P., Dell, T. a., Harris, C., and
Demain, S. H. (2013). Markerless motion capture and measurement of hand kinematics:
validation and application to home-based upper limb rehabilitation. IEEE Transactions
on Bio-medical Engineering, 60(8):2184–92.
Micera, S., Keller, T., Lawrence, M., Morari, M., and Popovic, D. B. (2010). Wearable
neural prostheses. Engineering in Medicine and Biology Magazine, IEEE, 29(3):64–69.
Microsoft (2012a). Kinect for windows sensor. [online] Retrieved from:

https://msdn.microsoft.com/en-us/library/hh855355.aspx. [Last accessed: 2016-17-
08].
162 BIBLIOGRAPHY
Microsoft (2012b). Kinect for windows sensor components and specifications. [online] Re-
trieved from: https://msdn.microsoft.com/en-us/library/jj131033.aspx. [Last accessed:
2016-17-08].
Microsoft.com (2014). Kinect 2. [online]Retrieved from: http://www.microsoft.com/en-

us/kinectforwindows/meetkinect/features.aspx. [Accessed: 2014-22-11].
Mufti, T., Slovak, M., Barker, A. T., and Farrow, T. F. (2016). 24-channel transcutaneous
electrical sensory stimulation of the forearm: Effects on cognitive performance and auto-
nomic arousal compared with single-electrode stimulation. Cogent Medicine, 3(1):1149992.
Nathan, R. H. (1994). Device for generating hand function. US Patent 5,330,516.
Nathan, R. H. (2005). Closed-loop control of the upper limb. In Proceedings of 10th Annual
Conference of the IFESS, pages 1–3.
National Audit Office (2010). Progress in improving stroke care. TSO.
National Geographic (2014). Brain-pictures, Brain Photos. [online] Retrieved from:

http://science.nationalgeographic.com/science/photos/brain/. [Accessed: 2014-15-08].
Neuromove.com (2011). NeuroMove Stroke Rehabilitation — Zynex NeuroDiagnostics. [on-

line] Retrieved from: http://www.neuromove.com/files/clinicaltraining.pdf. [Accessed:
2013-22-05].
Nicolás Cuenca, J. and Lazar, E. (2008). Functional electrical simulation (FES) in stroke.
Odstockmedical.com (2016). Microstim 2v2 kit — odstock medical limited. [online] Re-
trieved from: http://www.odstockmedical.com/products/microstim-2v2-kit. [Accessed:
2016-05-1].
Og-giken.co.jp (2002). OG GIKEN Products : Physiotherapy. [online] Retrieved

from:http://www.og-giken.co.jp/english/products/physiotherapy/kr7.html. [Accessed:
2013-20-05].
OML (2017). Odstock medical ltd: Leading rehabilitation through technology. [online]
Retrieved from: http://odstockmedical.com/. [Accessed: 2017-01-12].
Palmer, S. T., Martin, D. J., Steedman, W. M., and Ravey, J. (1999). Alteration of interfer-
ential current and transcutaneous electrical nerve stimulation frequency: effects on nerve
excitation. Archives of Physical Medicine and Rehabilitation, 80(9):1065–1071.
Pelton, T., van Vliet, P., and Hollands, K. (2012). Interventions for improving coordination
of reach to grasp following stroke: a systematic review. International Journal of Evidence-
Based Healthcare, 10(2):89–102.
Pmdtec (2015). Leading 3D chip technology provider — pmdtec.com. [online] Retrieved

from: http://pmdtec.com/. [Accessed: 2014-20-11].
BIBLIOGRAPHY 163
Polygerinos, P., Galloway, K. C., Savage, E., Herman, M., O’Donnell, K., and Walsh, C. J.
(2015a). Soft robotic glove for hand rehabilitation and task specific training. In 2015
IEEE International Conference on Robotics and Automation (ICRA), pages 2913–2919.
IEEE.
Polygerinos, P., Wang, Z., Galloway, K. C., Wood, R. J., and Walsh, C. J. (2015b). Soft
robotic glove for combined assistance and at-home rehabilitation. Robotics and Au-
tonomous Systems, 73:135–143.
Pomeroy, V., Evans, E., and Richards, J. (2006). Agreement between an electrogoniometer
and motion analysis system measuring angular velocity of the knee during walking after
stroke. Physiotherapy, 92(3):159–165.
Popovic, D., Baker, L., and Loeb, G. E. (2007). Recruitment and comfort of BION implanted
electrical stimulation: implications for FES applications. IEEE Transactions on Neural
Systems and Rehabilitation Engineering, 15(4):577–586.
Popović-Bijelić, A., Bijelić, G., Jorgovanović, N., Bojanić, D., Popović, M. B., and Popović,
D. B. (2005). Multi-field surface electrode for selective electrical stimulation. Artificial
organs, 29(6):448–452.
Prange, G. B., Jannink, M. J., Groothuis-Oudshoorn, C. G., Hermens, H. J., and IJzer-
man, M. J. (2006). Systematic review of the effect of robot-aided therapy on recovery of
the hemiparetic arm after stroke. Journal of Rehabilitation Research and Development,
43(2):171.
Prenton, S., Kenney, L. P., Stapleton, C., Cooper, G., Reeves, M. L., Heller, B. W., Sobuh,
M., Barker, A. T., Healey, J., Good, T. R., et al. (2014). Feasibility study of a take-home
array-based functional electrical stimulation system with automated setup for current
functional electrical stimulation users with foot-drop. Archives of Physical Medicine and
Rehabilitation, 95(10):1870–1877.
PrimeSense (2012). Carmine 1.09 - PrimeSense. [online] Retrieved

from:http://www.primesense.com/get-your-sensor2/carmine109/. [Accessed: 2013-
21-07].
Ragnarsson, K. (2007). Functional electrical stimulation after spinal cord injury: current
use, therapeutic effects and future directions. Spinal Cord, 46(4):255–274.
Reeves, M., Heller, B. W., Good, T., Healey, T., van der Meulen, J., Clarke, A., Pratt, E.,
Nair, K., and Barker, A. (2010). Shefstim: A clinical trail of self-tuning array stimulation
for foot-drop. In 1st annual conference of the UK and Republic of Ireland Chapter of the
International Functional Electrical Stimulation Society, page 48.
REF (2017). Research excellence framework 2014: Innovative

technologies for stroke rehabilitation. [online] Retrieved from:
http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=43853. [Accessed: 2017-01-12].
164 BIBLIOGRAPHY
Roger, V. L., Go, A. S., Lloyd-Jones, D. M., Benjamin, E. J., Berry, J. D., Borden, W. B.,
Bravata, D. M., Dai, S., Ford, E. S., Fox, C. S., et al. (2012). Heart disease and stroke
statistics2012 update a report from the american heart association. Circulation, 125(1):e2–
e220.
Rus, D. and Tolley, M. T. (2015). Design, fabrication and control of soft robots. Nature,
521(7553):467–475.
Saebo.com (2016). Saebo MyoTrac Infiniti — Saebo. [online] Retrieved from:

http://www.saebo.com/saebo-myotrac-infiniti/. [Accessed: 2016-05-11].
Saka, Ö., McGuire, A., and Wolfe, C. (2009). Cost of stroke in the united kingdom. Age
and ageing, 38(1):27–32.
Schulman, J. (2008). The feasible FES system: battery powered BION stimulator. Proceed-
ings of the IEEE, 96(7):1226–1239.
Schuon, S., Theobalt, C., Davis, J., and Thrun, S. (2008). High-quality scanning using
time-of-flight depth superresolution. In IEEE Computer Society Conference on Computer
Vision and Pattern Recognition Workshops (CVPRW), pages 1–7.
Seki, K., Tanaka, Y., Fujii, T., and Ichie, M. (2002). Change of medio-lateral cop shift
induced by electrical stimulation in hemiparetic stroke patients. 7th Annual Conference
of the International Functional Electrical Stimulation Society, page 134.
Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C.,
Leichter, I., Vinnikov, A., Wei, Y., Freedman, D., Kohli, P., Krupka, E., Fitzgibbon, A.,
and Izadi, S. (2015). Accurate, robust, and flexible real-time hand tracking. In Proceedings
of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15,
pages 3633–3642.
Shiratsu, A. and Coury, H. (2003). Reliability and accuracy of different sensors of a flexible
electrogoniometer. Clinical Biomechanics, 18(7):682–684.
Shotton, J. D. J., Keskin, C., Taylor, J. J., Sharp, T., Izadi, S., Fitzgibbon, A. W., Kohli,
P., and Robertson, D. P. (2016). Tracking hand/body pose. US Patent 20,160,085,310.
Slovak, M., Chindo, J., Nair, K. P. S., Reeves, M. L., Heller, B., and Barker, A. T. (2016).
Sensory barrage stimulation in the treatment of elbow spasticity: A crossover double blind
randomized pilot trial. Neuromodulation: Technology at the Neural Interface, 19(2):220–
226.
Snoek, G., IJzerman, M., in ’t, F. G., Stoffers, T., and Zilvold, G. (2000). Use of the NESS
Handmaster to restore handfunction in tetraplegia: clinical experiences in ten patients.
Spinal Cord, 38(4):244–249.
BIBLIOGRAPHY 165
Softkinetic (2013). DepthSense Cameras. [online]Retrieved from:

http://www.softkinetic.com/en-us/products/depthsensecameras.aspx. [Accessed:
2013-23-05].
Stiwell (2016). Electrotherapy for neurorehabilitation — STIWELL. [online] Retrieved from:

http://stiwell.medel.com/en/products/stiwell-electrotherapy. [Accessed: 2016-05-11].
Stone, L., Low, W., and Grande, A. (2014). Stem cell therapies for ischemic stroke. Stem
Cells and Neurodegenerative Diseases, 142–154.
Sullivan, J. E. and Hedman, L. D. (2004). A home program of sensory and neuromuscular

electrical stimulation with upper-limb task practice in a patient 5 years after a stroke.
Physical Therapy, 84(11):1045–54.
Sullivan, J. E. and Hedman, L. D. (2007). Effects of home-based sensory and motor ampli-
tude electrical stimulation on arm dysfunction in chronic stroke. Clinical Rehabilitation,
21(2):142–150.
Taylor, L., Lewis, G. N., and Taylor, D. (2012). Short-term effects of electrical stimulation
and voluntary activity on corticomotor excitability in healthy individuals and people with
stroke. Journal of Clinical Neurophysiology, 29(3):237–243.
Taylor, P. N. (2004). Functional electrical stimulation based training orthosis for hand
function following stroke. PhD thesis, University of Southampton.
Taylor, T. (2012). InnerBody.com — Your Interactive Guide to Human Anatomy. [online]

Retrieved from: http://www.innerbody.com/. [Accessed: 2013-18-05].
Teasell, R., Foley, N., Bhogal, S. K., and Speechley, M. (2008). 7 . Outpatient Stroke
Rehabilitation. Technical report, Evidence-Based Review of Stroke Rehabilitation.
Teasell, R., Foley, N., Salter, K., Bhogal, S., Jutai, J., and Speechley, M. (2009). Evidence-
based review of stroke rehabilitation: executive summary. Topics in Stroke Rehabilitation,
16(6):463–488.
Teo, J. T. H. (2009). Motor learning and neuroplasticity in humans. PhD thesis, University
College London.
Timmermans, A. A., Seelen, H. A., Willmann, R. D., Bakx, W., De Ruyter, B., Lanfermann,
G., and Kingma, H. (2009). Arm and hand skills: training preferences after stroke.
Disability and rehabilitation, 31(16):1344–1352.
Tong, D. (2012). Development of an Upper Limb Rehabilitaion System using Functional

Electrical Stimulation mediated by Iterative Learning Control. PhD thesis, University of
Southampton.
166 BIBLIOGRAPHY
Tornero, D., Wattananit, S., Madsen, M. G., Koch, P., Wood, J., Tatarishvili, J., Mine,
Y., Ge, R., Monni, E., Devaraju, K., et al. (2013). Human induced pluripotent stem
cell-derived cortical neurons integrate in stroke-injured cortex and improve functional
recovery. Brain, 136(12):3561–3577.
Townsend, N., Wickramasinghe, K., Bhatnagar, P., Smolina, K., M, N., Leal, J., Luengo-
Fernandez, R., and Rayner, M. (2012). Coronary heart disease statistics. Technical report,
British Heart Foundation.
Turk, R., Burridge, J. H., Davis, R., Cosendai, G., Sparrow, O., Roberts, H. C., Hughes,
A.-M., and Schulman, J. (2008). Therapeutic effectiveness of electric stimulation of the
upper-limb poststroke using implanted microstimulators. Archives of Physical Medicine
and Rehabilitation, 89(10):1913–22.
Van der Loos, M. (2002). Robot-assisted movement training compared with conventional
therapy techniques for the rehabilitation of upper-limb motor function after stroke. Arch
Phys Med Rehabil, 83.
Veerbeek, J. M., van Wegen, E., van Peppen, R., van der Wees, P. J., Hendriks, E., Rietberg,
M., and Kwakkel, G. (2014). What is the evidence for physical therapy poststroke? A
systematic review and meta-analysis. PLoS ONE, 9(2):e87987.
Vicon.com (2010). Vicon motion capture. [online] Retrieved from: https://www.vicon.com/.

[Accessed: 2013-23-05].
Wang, R. Y. and Popović, J. (2009). Real-time hand-tracking with a color glove. In ACM
Transactions on Graphics (TOG), volume 28, page 63. ACM.
Ward, A. R. (2009). Electrical stimulation using kilohertz-frequency alternating current.

Physical Therapy, 89(2):181–190.
Ward, A. R. and Shkuratova, N. (2002). Russian electrical stimulation: the early experi-
ments. Physical Therapy, 82(10):1019–1030.
Winstein, C. J., Rose, D. K., Tan, S. M., Lewthwaite, R., Chui, H. C., and Azen, S. P.
(2004). A randomized controlled comparison of upper-extremity rehabilitation strategies
in acute stroke: a pilot study of immediate and long-term outcomes. Archives of Physical
Medicine and Rehabilitation, 85(4):620–628.
Winstein, C. J., Stein, J., Arena, R., Bates, B., Cherney, L. R., Cramer, S. C., Deruyter,
F., Eng, J. J., Fisher, B., Harvey, R. L., et al. (2016). Guidelines for adult stroke reha-
bilitation and recovery a guideline for healthcare professionals from the american heart
association/american stroke association. Stroke, 47(6):e98–e169.
Wolpert, D. M., Ghahramani, Z., and Flanagan, J. R. (2001). Perspectives and problems
in motor learning. Trends in Cognitive Sciences, 5(11):487–494.
BIBLIOGRAPHY 167
World Health Organization (2011). World report on disability 2011. Technical report, World
Health Organisation.
Wu, X., Zhu, B., Fu, L., Wang, H., Zhou, B., Zou, S., and Shi, J. (2013). Prevalence,
incidence, and mortality of stroke in the chinese island populations: a systematic review.
PloS ONE, 8(11):e78629.
Wysiw, W. and Jackson, R. (2007). Electrical stimulation. Elsevier, Philadelphia, USA.
Yang, K., Freeman, C. T., Torah, R., Beeby, S., and Tudor, J. (2014). Screen printed fabric
electrode array for wearable functional electrical stimulation. Sensors and Actuators A:
Physical, 213:108–115.
Yang, L., Zhang, L., Dong, H., Alelaiwi, A., and El Saddik, A. (2015). Evaluating and im-
proving the depth accuracy of kinect for windows v2. IEEE Sensors Journal, 15(8):4275–
4285.
Yang, Q., Yang, R., Davis, J., and Nistér, D. (2007). Spatial-depth super resolution for range
images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages
1–8.
Zhang, D., Guan, T. H., Widjaja, F., and Ang, W. T. (2007). Functional electrical stim-
ulation in rehabilitation engineering: a survey. In Proceedings of the 1st International
Convention on Rehabilitation Engineering & Assistive Technology: in Conjunction with
1st Tan Tock Seng Hospital Neurorehabilitation Meeting, pages 221–226. ACM.
Zhang, S., Hu, H., and Zhou, H. (2008). An interactive internet-based system for tracking
upper limb motion in home-based rehabilitation. Medical & Biological Engineering &
Computing, 46(3):241–9.

A Home-Based Functional Electrical Stimulation System For Upper-Limb Stroke Rehabilitation PDF

Uploaded by

Copyright:

Available Formats

A Home-Based Functional Electrical Stimulation System For Upper-Limb Stroke Rehabilitation PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Home-Based Functional Electrical Stimulation System For Upper-Limb Stroke Rehabilitation PDF

Uploaded by

Copyright:

Available Formats

What are some of the main causes and effects of stroke?

What are some of the main causes and effects of stroke?

What are some of the challenges of stroke rehabilitation?

What are some of the challenges of stroke rehabilitation?

UNIVERSITY OF SOUTHAMPTON

A Home-Based Functional Electrical

A thesis submitted in fulfillment for

List of Tables xiii

2 Upper Limb Stroke Rehabilitation 11

3 Home-Based FES Technology 21

3.4.3 Clinical and Non-Clinical Evaluation . . . . . . . . . . . . . . . . . . . 28

4 Goal-Oriented Stroke Rehabilitation Utilising Electrical Stimulation 37

5 Sensor Technologies for the Hand and Wrist 59

6 GO-SAIL with Improved Hand Functionality 79

7 GO-SAIL Compact System Implementation 105

8 Conclusions and Future Research 121

A Identification Protocols 127

B Graphical User Interface 131

C Non-Contact Sensors for Hand and Wrist Rehabilitation 135

D Previous Leap Motion Experiment 139

E Selection of feedback controller Cc,w and ILC operator Lw 147

F Multichannel Stimulator for GOSAIL Compact 151

2.1 Targeted muscles in upper limb (anterior view) . . . . . . . . . . . . . . . . . 12

3.1 BION taken from Loeb et al. (2006) . . . . . . . . . . . . . . . . . . . . . . . 22

4.1 The GO-SAIL system components . . . . . . . . . . . . . . . . . . . . . . . . 38

5.1 Outputs of Vicon system taken from Vicon.com (2010) . . . . . . . . . . . . . 60

5.8 Kinect 2 taken from Microsoft.com (2014) . . . . . . . . . . . . . . . . . . . . 67

6.1 System components: 1) motion tracking hardware components, 2) interactive

7.1 GO-SAIL Compact system overview . . . . . . . . . . . . . . . . . . . . . . . 107

A.1 Anterior deltoid identification with parameters γ, β. . . . . . . . . . . . . . . 128

B.1 Start screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

C.1 Duo3D taken from Duo3d (2013) . . . . . . . . . . . . . . . . . . . . . . . . . 135

F.1 Electronics design of Ilić et al. (1994) . . . . . . . . . . . . . . . . . . . . . . . 152

4.1 Participant demographics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.1 Leading examples of contact sensors . . . . . . . . . . . . . . . . . . . . . . . 64

6.1 Stroke participant demographic characteristics . . . . . . . . . . . . . . . . . 92

7.1 GO-SAIL+ and GO-SAIL Compact comparisons . . . . . . . . . . . . . . . . 119

I, Mustafa C. Kutlu, declare that the thesis entitled

A Home-Based Functional Electrical Stimulation System for Upper-Limb Stroke Rehabil-

• I have acknowledged all main sources of help;

i:j Sequence of indices i,...,j

Figure 1.1: Stroke mechanisms

1.2 Problem Statement

A carefully orchestrated sequence of stimulation using FES to different muscle groups is

1.3 Research Contribution

Specific contributions are summarised as follows:

1.4 Thesis Structure

Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 8

Upper Limb Stroke Home-Based FES Goal-Oriented Stroke

Part 2: GO-SAIL+ Conclusions

Part 3: GO-SAIL Compact

Compact System Implementation

Figure 1.2: Organisation of thesis and steps in development of three systems

Upper Limb Stroke Rehabilitation

Figure 2.1: Targeted muscles in upper limb (anterior view)

Figure 2.2: Morphologic features of skeletal muscle (anterior view).

The locomotive systems comprise two components that are excitable:

A variety of therapeutic approaches exist to assist patients’ movement, with a comprehensive

2.1 Conventional and Robotic Therapy