PROGRESS REPORT

www.advintellsyst.com

Machine Learning for Soft Robotic Sensing and Control

Keene Chin, Tess Hellebrekers, and Carmel Majidi*

closed form. In addition, their dynamics

Herein, the progress of machine learning methods in the field of soft robotics, are often highly nonlinear, resulting in sys-
specifically in the applications of sensing and control, is outlined. Data-driven tems that are functionally stochastic with
methods such as machine learning are especially suited to systems with gov- respect to a finite state representation.
These properties of soft robots, which sim-
erning functions that are unknown, impractical or impossible to represent
plify low-level control and sensing issues,
analytically, or computationally intractable to integrate into real-world solutions. complicate the high-level planning, control,
Function approximation with careful formulation of the machine learning and state estimation of these systems.
architecture enables the encoding of dynamic behavior and nonlinearities, with Methods based on rigid-body dynamics
the added potential to address hysteresis and nonstationary behavior. Supervised serve as the backbone of many traditional
robotics solutions to control problems
learning and reinforcement learning in simulation and on a wide variety of
requiring heavy modification and approxi-
physical robotic systems have shown promising results for the use of empirical mation to work for soft systems.[13]
data-driven methods as a solution to contemporary soft robotics problems. Among other factors, the difficulty of rep-
resenting the high-dimensional soft system
dynamics means that the gap between the
performance of these methods in theory
1. Introduction and in practice is usually especially large. High-fidelity models
of soft systems have been successfully used in the control of soft
Soft robots belong to a diverse category of machines, the borders of systems, but this requires accurate system identification and
which are quite porous depending on the application and field of expensive run-time computation, limiting the potential applica-
interest.[1–4] Soft materials are those with a stiffness below some tions.[14] For soft sensing, this high dimensionality also poses a
application-specific threshold—typically 1 MPa in the case of sys- fundamental challenge as many physical states can produce the
tems that interact with biological tissue or mimic soft biological same sensor reading as the higher dimensional data is trans-
organisms.[5] The mechanical compliance of these materials can duced. In addition, these sensors are typically subject to a greater
be exploited to obviate or simplify many low-level sensing and con- variation in manufacturing outcomes and shapes, have notice-
trol problems, e.g., by stabilizing footfalls in legged locomotion[6] able degrees of hysteresis, nonstationarity, and other mechanical
or deforming around object geometry in grasping. This ability to nonlinearities that may be negligible in their rigid counterparts.
encode a desired capability into the mechanical design of the robot Approaches to these sensing, actuation, and control problems
itself is one appealing attribute of soft materials.[7–10] Another dis- based on data-driven machine learning provide a parallel path
tinguishing feature is the increased safety of using them in mobile to the continued analytical modeling work and have produced
systems due to their lower hardness, increased damping, and promising results in recent years.
lower mass as compared to metals. Because of these attributes, Machine learning methods involve the empirical approxima-
compliant materials are increasingly attractive options for use tion of an unknown model of some system, e.g., sensor transduc-
in modern robotics applications. tion behavior or robot manipulator dynamics.[15] Although this
As these materials are continuously deformable, their full state process commonly involves the use of neural networks for func-
is governed by continuum mechanics rather than rigid-body tion approximation, the term is usefully applied to many optimi-
dynamics.[11,12] This means that they are infinite-degree-of- zation strategies and other data-driven approaches. As applied to
freedom (DOF) systems, which are difficult to represent in its use in engineering robotic systems, a critical axis to consider
is the degree to which final system performance is dominated by
the training data versus an a priori model.[16]
K. Chin, T. Hellebrekers, Prof. C. Majidi
Department of Mechanical Engineering We consider soft sensors to be devices that transduce from
Carnegie Mellon University some physical phenomenon, e.g., force, to a signal that encodes
Pittsburgh, PA 15213, USA that information, and which use deformable materials either for
E-mail: [email protected] the method of sensing or in the mechanical structure in which the
The ORCID identification number(s) for the author(s) of this article sensing element is embedded. Machine learning is used in decod-
can be found under https://doi.org/10.1002/aisy.201900171. ing the signal to reconstruct an intelligible representation of the
© 2020 The Authors. Published by WILEY-VCH Verlag GmbH & Co. KGaA, original phenomenon. For control of soft robotic systems, we look
Weinheim. This is an open access article under the terms of the Creative at systems where the physical structure and/or actuators of the
Commons Attribution License, which permits use, distribution and robot itself are significantly soft. In the literature, this is over-
reproduction in any medium, provided the original work is properly cited. whelmingly represented by soft continuum manipulators and
DOI: 10.1002/aisy.201900171 end effectors, with a few significant standouts. Machine learning

Adv. Intell. Syst. 2020, 2, 1900171 1900171 (1 of 8) © 2020 The Authors. Published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
 26404567, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.201900171 by Readcube (Labtiva Inc.), Wiley Online Library on [24/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advintellsyst.com

applied to soft robot control involves either an approximation of Keene Chin received his B.S. degree in
the kinematic/dynamic model of the system from data or the mechanical engineering from the
direct learning of controllers for a given behavior. In this work, University of Texas at Dallas in 2018.
we outline the major trends over the last five years for applying He is currently a Ph.D. student in the
machine learning to applications in soft robotics, including com- Robotics Institute at Carnegie Mellon
mon approaches and pitfalls, and potential extensions. University working with the Soft
Machines Lab under the supervision
of Professor Carmel Majidi. His
2. Sensing research interests are on applications
of data-driven techniques for soft
There are two common threads found in the literature for appli-
robot state estimation, planning, and control.
cations of machine learning to soft sensors (Figure 1). Although
supervised learning techniques are applied in both, they differ in Tess Hellebrekers received her B.S.
the method of data collection. One approach utilizes sensor data degree in biomedical engineering from
collected in a designed test environment and the other builds upon the University of Texas at Austin in
sensor data collected in a fully integrated robotic system. One rea- 2016. She is currently a Ph.D. student
son for these two methods is that soft sensor characterization is in the Robotics Institute at Carnegie
largely dependent on the mounting structure and shape. Due to Mellon University working with the
the variety of robotic testbeds and hardware limitations, it is not Soft Machines Lab under
always possible to accurately measure real-world values in each the supervision of Professor Carmel
system implementation. However, not establishing a mapping Majidi. Her research focus is on
between real-world values and raw sensor output misses an oppor- developing a broad range of soft
tunity for model comparison and results that users can interpret. sensing technologies for soft robotic grasping and
multimodal monitoring.
2.1. Sensor Characterization
Carmel Majidi is a professor of
The first technique emphasizes the mapping between raw sensor mechanical engineering at Carnegie
output and real-world values, such as force in newtons, strain in Mellon University. He leads the Soft
percentage, or pressure in pascals. This is useful to support novel Machines Lab, which develops novel
soft sensor implementations and compare trends against known materials architectures, modeling
sensors with similar transduction modes. Setting up controlled techniques, and fabrication methods
experiments with additional verification hardware (e.g., triaxis for soft multifunctional composites
force sensors) also provides ground truth references to deter- and microfluidics. Professor Majidi’s
mine the sensor resolution, range, and sensitivity. All these met- research has applications to the fields
rics are crucial to engineers and roboticists who are looking to of soft bioinspired robotics, wearable
select a sensor to match the desired design requirements for computing, and mechanics of soft matter. Prior to joining
their systems. Controlled dynamic experiments along with a Carnegie Mellon University, he had postdoctoral
recurrent neural network[17,18] and controlled static experiments fellowships at Princeton and Harvard. He received his B.S.
paired with feed-forward neural networks have been shown to degree from Cornell University and M.S. and Ph.D.
estimate magnitude (in N) and location of touch (in mm)[19] degrees from UC Berkeley.
and object orientation[20] (Figure 2A,B).
(long short-term memories),[22] as well as classifying gestures
2.2. Systems Characterization with capacitive stretch sensors (convolutional neural network),[23]
and object gesture/shape/size/and weight recognition with
The second, more common, technique avoids direct controlled multimodal optical/pressure inputs.[24] Deep learning is also being
characterization and instead prioritizes higher-level labels, such used for identifying caging grasps on unknown objects with point
as successful grasp recognition, slip detection, or object classifica- cloud data,[25] texture classification with piezoresitive taxels,[21]
tion. We attribute this largely to the discrepancy in individual contact estimation and localization with fiber optic embedded soft
sensor characterization and practical use, specifically, the inherent pad,[26] object recognition with soft tactile skins and joint forces/
limitation that the soft material sensor response will change with torques,[27] and full-body motion estimation.[28] In summary, both
the properties of neighboring structures and mounting equip- approaches are necessary to continue developing both novel
ment. In addition, the real-world values provided by the first sensors with supporting models and practical implementation
method are not necessarily understood by the robotic system. knowledge.
In this case, raw data and real-world calibrated values are equally
useful as input to the learning models, and it is desirable to avoid
an unnecessary middle step. A handful of systems that use this 3. Control
approach are shown in Figure 2. Temporal data and, therefore,
temporal learning methods, have been shown to be effective for Recent efforts to enable control of soft robots through machine
identifying grasp success and stability with microfluidic sensors learning have addressed different levels of the control pipeline.

Adv. Intell. Syst. 2020, 2, 1900171 1900171 (2 of 8) © 2020 The Authors. Published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
 26404567, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.201900171 by Readcube (Labtiva Inc.), Wiley Online Library on [24/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advintellsyst.com

Figure 1. Overview of machine learning strategies common in the field. A) Sensor characterization involves collecting raw sensor data aligned with
ground truth in a controlled and monitored environment (e.g., lab setting). This data is then typically provided as input to a neural network to predict
further values. B) In contrast, systems characterization collects sensor data in a less controlled environment that mimics their use in the field. In this case,
ground truth measurements such as force are more difficult to obtain. Therefore, users often circumvent this by mapping to higher-level classifications,
such as grasp success and texture recognition. C) Learning inverse kinematics requires training data composed of matched pairs of robot position and
actuator configuration. A function approximator (neural network, etc.) encodes the relationship between sampled data to generalize to arbitrary desired
positions. D) Learning forward dynamics collects sets of two sequential positions, as well as the actuator action that caused the transition between them.
The time-dependent dynamic behavior is encoded into a neural net, allowing for open-loop following of arbitrary trajectories. E) Learning controllers
function by iteratively evaluating the performance of the resultant trajectory for a controller with respect to some cost function and updating parameters to
increase that performance via optimization.

Figure 2. Examples of soft robotic systems by A) Hellebrekers et al.[19] and B) Han et al. (Reproduced with permission.[17] Copyright 2018, IEEE.) where
individual sensors were independently characterized in carefully monitored environments prior to their application. The bottom row shows systems from
C) Baishya and Bäuml (Reproduced with permission.[21] Copyright 2016, IEEE.), D) Zimmer et al.,[22] and E) Larson et al. (Reproduced with permission.[23]
Copyright 2019, Mary Ann Liebert.) in which the sensors are characterized for their specific tasks and setups.

Adv. Intell. Syst. 2020, 2, 1900171 1900171 (3 of 8) © 2020 The Authors. Published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
 26404567, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.201900171 by Readcube (Labtiva Inc.), Wiley Online Library on [24/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advintellsyst.com

Table 1. Overview of different parameters for learning control. NN, neural network; GMM, Gaussian mixture model.

Paper Dynamic Model representation Hardware training Simulation training Adaptive Feedback

Lee et al. [32]

Ensemble of NNs ✓ ✓ ✓ ✓
Ansari et al.[33] Linear function approximator ✓
Thuruthel et al.[34] NN ✓ ✓
Trumic et al. [35]
✓ Parameterized analytical model ✓ ✓ ✓
Chen and Lau[36] GMM ✓
Thuruthel et al.[37] NN ✓
Jiang et al. [38]
Hierarchical ensemble of NNs ✓ ✓
Rolf et al.[29] Constrained extreme learning machine ✓ ✓ ✓
Zhang et al.[39] NN Q-function ✓ ✓ ✓
Thuruthel et al.[30] ✓ Recurrent NN ✓ ✓
Gillespie et al. [40]
✓ Deep NN ✓
Bern et al.[41] Spring network energy simulation ✓
Zhang et al.[42] Deep NN ✓ ✓ ✓
Schlagenhauf et al.[31] Deep NN ✓ ✓
Della Santina et al.[43] ✓ Parameterized analytic model ✓ ✓ ✓

The two most common tactics are learning the inverse kinematics/ learning tends to not scale to the dimensionality and dynamic
statics of soft manipulators[29] and learning the forward dynamics sensitivity of mobile systems.
model to enable predictive control.[30] A less common strategy, The forward kinematics of soft robots are usually highly non-
which has produced interesting results, is the direct learning of linear. For example, in the case of a pneumatic continuum arm,
controllers for specific behaviors.[31] An overview of these efforts the vector of pressures in segment chambers, i.e., the configu-
is provided in Table 1. The type of function approximation used to ration space, maps to the pose of the manipulator tip, i.e., the
encode the kinematic model, dynamic model, or controller is task space, through an equilibrium function that depends on
shown, as well as the origin of the training data. Dynamic models, local material properties and design geometry. This mapping
as defined here, do not rely on the quasistatic assumption and have between actuator configuration space and task space can also
time dependence. Adaptive models can learn from additional data be contingent on the environment. For example, the curvature
after the initial training, and feedback models produce closed loop response of a segment to a given pressure can vary significantly
controllers. in the presence of external forces such as gravity or surface trac-
tions. A global representation of the Jacobian is difficult to con-
3.1. Learning Inverse Kinematics vey analytically, even without concerns of nonstationarity and
stochasticity. Inverting any found Jacobian is even more diffi-
Supervised learning is the basis of much of the machine learn- cult. Because of this, a common technique in the field is approx-
ing work in learning the kinematics of soft robots. If the system imating the global Jacobian with a function approximator,
is treated as quasistatic, i.e., it reaches static equilibrium which is most commonly a neural network[32,34,37,38] but can
between control steps, a mapping between actuator configura- be of other forms, like linear function approximators,[33] con-
tion and task-space position is achievable in open loop. For strained extreme learning machines,[29] and Gaussian mixture
redundant systems, an injective mapping can be enforced models (GMMs).[36] It should be noted that finding this map-
through local optimization, which results in a bijective mapping ping can be done for a given position without learning any
within the workspace. This is ideal for supervised learning from model by performing gradient descent over the actuator config-
labeled pairs of training data. To collect training data from uration with the desired position as the target.[46] The downside
which to learn the kinematics function, a random walk through of this method is that it induces undefined motion iteration in
actuation space on the physical hardware (motor babbling) is a the robot between each commanded position, which is poten-
straightforward method and has been used for cable-driven con- tially unsafe and results in much slower execution. The global
tinuum manipulators,[34] pneumatic continuum manipulators inverse Jacobian can also be approximated by aggregating an
such as the Bionic Handling Assistant (Figure 3A)[29] and hon- ensemble of learned local Jacobians sampled from across the
eycomb pneumatic network manipulator (Figure 3B),[38] and full state space.[32,36] Explicit global representations are more
simulated manipulators.[32,37] Data can also be collected from concise, while ensemble methods allow prioritization of perfor-
human demonstration, by manually replicating important con- mance in the regions of most interest.
figurations and recording the associated actuator parameters.[31] By virtue of their mechanical compliance, the equilibrium
Data-driven methods have been used to achieve locomotion state of soft systems is dependent on external forces and trac-
with soft systems as well,[41] but straightforward supervised tions. Replacing the purely kinematic mapping between the

Adv. Intell. Syst. 2020, 2, 1900171 1900171 (4 of 8) © 2020 The Authors. Published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
 26404567, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.201900171 by Readcube (Labtiva Inc.), Wiley Online Library on [24/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advintellsyst.com

Figure 3. Examples of soft robotic systems that implement machine learning for A,B) learning inverse kinematics, C,D) learning forward dynamics, and
E–G) directly learning controllers. The systems are: (A) the Bionic Handling Assistant pneumatic continuum manipulator from Rolf et al, Reinhart and
Steil. (Reproduced with permission.[44] Credit Procedia Technology.); (B) Honeycomb pneumatic network continuum manipulator from Jiang et al.
(Reproduced with permission.[38] Copyright 2017, IEEE.); (C) 1 DOF pneumatic finger from Gillespie et al (Reproduced with permission.[40]
Copyright 2018, IEEE.); (D) one segment pneumatic continuum manipulator for dynamic motion from Thuruthel et al (Reproduced with permission.[45]
Copyright 2018, IEEE.); (E) humanoid tendon-driven foam hand end effector from Schlagenhauf et al.[31]; (F) SUPERball mobile tensegrity robot from
Zhang et al (Reproduced with permission.[42] Copyright 2017, IEEE.); (G) stuffed tendon-driven mobile hexapod from Bern et al.[41]

robot configuration and task spaces with a function that incorpo- Rolf et al.[29] were more mechanically sophisticated than the sys-
rates the force equilibrium has been shown to be effective for tem used here, but the full mechanical complexity was not lev-
position tracking of continuum manipulators.[29,37] Even without eraged due to the lack of a useful dynamics model. Thuruthel
explicitly learning the force balance, the combination of end et al. demonstrate that an approximate dynamics model can be
effector feedback with this compliance can be used for distur- learned and is sufficient for achieving performance beyond what
bance rejection in simulation.[34] Due to this decoupling of kine- is possible with quasistatic methods.
matic shape and external force balance, joint position–stiffness
controllers can be learned for soft manipulators.[33,35] 3.3. Direct Controller Learning

3.2. Learning Forward Dynamics Learning kinematic or dynamic models for a soft robot means
that, while part of the control pipeline relies on empirically learnt
Quasistatic approximations of robotic systems are useful for sys- models, the controller itself is still engineered. Reinforcement
tems with slow time-scales or tasks where timing is not critical. learning is a machine learning strategy that allows the controller
However, for many applications, dynamic control is more appro- itself to be created through learning from sequential environ-
priate. The forward dynamics of soft systems are even more mental interactions, rather than from previously collected exog-
analytically difficult than the kinematics, so machine learning enous data. Policy gradient based reinforcement learning
techniques have been applied to a variety of these problems. converges to a locally optimal controller without an analytical
Incorporating dynamics into position control has been shown model of the robot dynamics, but requires much more time
to enable time-sensitive trajectory tracking in a hybrid soft–rigid and data for training than supervised learning. This is due to
arm,[35] and a 1 DOF pneumatic finger (Figure 3C).[40] Dynamics the need to evaluate the full trajectory produced by following
models can be learned that are adapted to varying dynamic a controller from a specific state before making updates to the
parameters of the system, e.g., with stiffness-tuning soft manip- model at a given optimization step. A common robotics solution
ulators.[43] Exploiting system dynamics to achieve performance is learning in simulation for many trials, as for the complex
beyond quasistatic predictions is a benefit of learning the tendon-driven humanoid hand in Figure 3E.[31] The controller
dynamic system behavior and has been shown to increase learned in simulation can be further refined on the physical
manipulator workspace for the soft system of a continuum robot.[39] Trajectory optimization is a common numerical optimi-
arm.[30] The work of Thuruthel et al.[45] on using learned dynamic zation method for generating local controllers, which can be
models exemplifies a significant step in the evolution of soft implemented in a machine learning context. Machine learning–
manipulators enabled by machine learning methods. Earlier based trajectory optimization has been used to generate a library
examples of soft continuum manipulators such as the work of of trajectories for a mobile tensegrity robot (Figure 3F)[42]

Adv. Intell. Syst. 2020, 2, 1900171 1900171 (5 of 8) © 2020 The Authors. Published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
 26404567, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.201900171 by Readcube (Labtiva Inc.), Wiley Online Library on [24/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advintellsyst.com

and a forward walking trajectory for a cable-driven soft quadru- vegetables,[49] which can be considered a dual problem to control-
ped (Figure 3G).[41] Zhang et. al[42] show that a global controller ling a deformable robotic system in a rigid environment.
can be learned by using the outputs of local trajectory stabilizing
controllers as labels for supervised learning. The success of local 4.1. Hysteresis and Nonstationarity
methods lies in the ability to simplify the space needed to explore
for data-driven methods. The humanoid soft hand used by There are unique considerations for soft systems that hint at the
Schlagenhauf et al. is much more mechanically expressive than potential for unique strategies that come primarily from the syn-
the previous works using continuum soft manipulators. A global ergy between soft systems and data-driven methods. Hysteresis
dynamic or kinematic model for this system would require an and nonstationarity are inherent side effects of the use of elas-
unfeasible amount of data for physical collection. By creating tomers and other soft materials in the construction of soft robot
a basis out of a small number of finite poses, the manipulator systems. Current methods often ignore these effects or treat
is capable of performing dexterous motion without a full model them as unmodeled noise.[29,50] Hysteresis can be addressed
of system behavior. Leveraging the ability to create useful behav- by conditioning learned models on system state history. This
ior without global models will enable an evolution in the com- explicit time dependence can be encoded in structures such as
plexity of potential soft robotic systems and the performance recurrent neural networks[30] or by simply concatenating multi-
of existing systems. ple time-steps of state data as the input to the model. These types
of solutions are not wholly incompatible with quasistatic
3.4. Hybrid Approaches approaches but lend themselves to more dynamic conceptualiza-
tions of the system behavior. Similarly, nonstationary behavior,
Systems do not need to be purely data driven for machine learn- e.g., baseline drift of raw sensor data due to polymer degradation,
ing to be helpful. The dichotomy between empirical, data-driven induces a need for models that can adapt; otherwise retraining/
knowledge and understanding drawn from first principles is recalibration is periodically required.[29]
not an exclusive one. Rather, the strengths of each strategy
can reinforce the other. Hybrid approaches allow the leveraging 4.2. Model Bias and Overfitting
of existing knowledge so only the most intractable system
components need to be learned. Learning the parameters of Using machine learning to enable the sensing and control of
an analytical dynamics model, similarly to traditional adaptive soft robots can lead to unpredictable effects if the limitations
control methods, has been shown to be fast and effective if such of machine learning strategies are not explicitly addressed.
a model can be constructed with enough fidelity.[35] It is also pos- Model bias and overfitting to the training set are fundamental
sible to decompose control of multiactuator systems into analytic considerations in the machine learning literature. Model bias
kinematic targets, where each actuator achieves the final shape results from attempting to fit the training data to an a priori
through a system-level controller[29] or individual actuator-level model that does not adequately represent the true behaviour
controllers.[38] Model predictive control strategies can be used of the system dynamics, and is one of the motivations for using
on systems with uncertain models by using neural networks model-free machine learning methods. Given an architecture
to encode approximate models of system dynamics.[40] capable of encoding the dataset information, learned models
can match the distribution of training data closely, as machine
learning excels at uncovering unknown structure. However,
4. Upcoming Challenges these data-driven methods can encode artifacts of the training
data rather than the underlying general features, leading to
The use of machine learning to enable soft robotic functionality overfitting to the training data, i.e., loss of generality. Even if
has produced many promising results, but there remain several the underlying features are learned appropriately over the space
challenges to overcome and unknowns to explore, both from of the training set distribution, extrapolations beyond the range
properly training general machine learning models and handling of that distribution can produce unreliable results.
the spatial and temporal nonlinearities of soft systems. Many
machine learning techniques that have produced good results 4.3. Validation and Reproducibility
in rigid-body robot systems have yet to be fully explored in
the soft robot domain. Domain randomization involves augment- Due to the approximate nature of machine learning models,
ing simulated training data by perturbing the simulator’s esti- a priori analytical verification of model performance is not gen-
mates of the physical parameters of the real task. This strategy erally possible. Specific model validation techniques for machine
combined with a deep neural network controller trained with learning have been developed and should be implemented
reinforcement learning has allowed a humanoid robotic hand when using these methods. Tools such as dropout, k-fold cross-
to solve Rubik’s cubes without specialized hardware or encoded validation, and input perturbation can prevent models from over-
human knowledge.[47] The stochastic and nonlinear dynamics fitting to the training data and allow greater generalization.
of soft systems is a major difficulty in their control. Similarly dif- Metrics for evaluating the training efficacy of learned models
ficult dynamics have been learned using machine learning, can be used to allow better comparison between systems. Rather
including a low-cost manipulator with many of the same compli- than reporting the performance metrics of the final learned
ance and nonstationarity concerns of soft systems,[48] as well as model in isolation, the relationship between training set perfor-
an industrial manipulator using a knife to cut deformable mance and testing set performance should be tracked as the

Adv. Intell. Syst. 2020, 2, 1900171 1900171 (6 of 8) © 2020 The Authors. Published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
 26404567, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.201900171 by Readcube (Labtiva Inc.), Wiley Online Library on [24/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advintellsyst.com

model is trained. A high training error after the learned model Keywords
converges (underfitting) indicates the presence of model bias.
control, machine learning, neural networks, sensing, soft robotics
A low training error and a high testing error means the model
has overfitted to the training data. In these cases, either training Received: December 10, 2019
for less time or using a model that matches the complexity of the Revised: January 10, 2020
underlying governing equations can be a solution. Cross-study Published online: March 3, 2020
reproducibility is another important component in the scientific
study of these robotic systems. When learned models are used as
part of the system, conveying a study’s experimental parameters [1] D. Trivedi, C. D. Rahn, W. M. Kier, I. D. Walker, Appl. Bionics Biomech.
does not only consist of describing the physical system and the 2008, 5, 99.
type of machine learning technique used. The training data itself [2] G. M. Whitesides, Angew. Chem. Int. Ed. 2018, 57, 4258.
is an integral part of the system’s eventual behavior, and collabo- [3] C. Majidi, Adv. Mater. Technol. 2019, 4, 1800477.
[4] D. Rus, M. T. Tolley, Nature 2015, 521, 467.
ration and validation would be greatly enhanced by the sharing of
[5] C. Majidi, Soft Robot. 2014, 1, 5.
such data between researchers.
[6] G. Buondonno, J. Carpentier, G. Saurel, N. Mansard, A. De Luca,
J.-P. Laumond, in IEEE/RSJ Int. Conf. on Intelligent Robots and
4.4. Increasing System Complexity Systems, IEEE, Piscataway, NJ 2017, p. 705.
[7] K. Man, A. Damasio, Nat. Mach. Intell. 2019, 1, 446.
Continuum robots and other systems that rely on the constant [8] D. Howard, A. E. Eiben, D. F. Kennedy, J.-B. Mouret, P. Valencia,
curvature assumption still dominate the literature[51] due to D. Winkler, Nat. Mach. Intell. 2019, 1, 12.
their simplicity of analysis. However, when the kinematic and [9] K. Nakajima, H. Hauser, T. Li, R. Pfeifer, Soft Robot. 2018, 5, 339.
dynamic behavior is being learned empirically, an analytically [10] S. Haddadin, K. Krieger, A. Albu-Schaffer, T. Lilge, IEEE Trans. Robot.
2018, 34, 91.
simple system and an analytically intractable system do not nec-
[11] K. M. de Payrebrune, O. M. O’Reilly, Extreme Mech. Lett. 2016, 8, 38.
essarily require different approaches to learn. Complex functions
[12] E. Coevoet, T. Morales-Bieze, F. Largilliere, Z. Zhang, M. Thieffry,
that are not easily represented analytically can be encoded by
M. Sanz-Lopez, B. Carrez, D. Marchal, O. Goury, J. Dequidt,
training function approximators with sampled data from physical C. Duriez, Adv. Robot. 2017, 31, 1208.
systems. Given the ability of machine learning models to dis- [13] D. Ross, M. P. Nemitz, A. A. Stokes, Soft Robot. 2016, 3, 170.
cover the underlying structure in the data without a full a priori [14] C. Duriez, in Proc. IEEE Int. Conf. on Robotics and Automation, IEEE,
model, the physical system that produces the data need not be Piscataway, NJ 2013, p. 3982.
analytically feasible to model. Because of this, riskier and more [15] T. M. Mitchell, Machine Learning, McGraw-Hill Education, New York
esoteric robot morphologies can be explored, leveraging the 1997.
rich diversity of design that has been generated within the soft [16] C. Bishop, Pattern Recognition and Machine Learning, 1st ed.,
robotics literature. An expansion in capabilities of control and Springer, New York 2006.
sensing could also enable open-source, user-fabricated plat- [17] S. Han, T. Kim, D. Kim, Y. L. Park, S. Jo, IEEE Robot. Automat. Lett.
forms with limited standardization such as those proposed by 2018, 3, 873.
Della Santina et al[52] to proliferate. More complex systems that [18] T. G. Thuruthel, B. Shih, C. Laschi, M. T. Tolley, Sci. Robot. 2019, 4,
feature multifunctional materials, computationally optimized eaav1488.
[19] T. Hellebrekers, O. Kroemer, C. Majidi, Adv. Intell. Syst. 2019, 1,
geometry,[53] or an intrinsic interplay between structure and
1900025.
control[54] (morphological computation) are all potentially quite
[20] M. H. Rosle, R. Kojima, K. Or, Z. Wang, S. Hirai, IEEE Robot. Automat.
difficult to model analytically but have properties that are nonethe-
Lett. 2019, 5, 159.
less highly enabling and advance the field of soft robotics. Using [21] S. S. Baishya, B. Bäuml, in IEEE Int. Conf. on Intelligent Robots and
machine learning to aid in sensing and control of these systems is Systems, IEEE, Piscataway, NJ 2016, p. 8.
a promising synergy that can allow the novel capabilities of soft [22] J. Zimmer, T. Hellebrekers, T. Asfour, C. Majidi, O. Kroemer,
materials and novel soft robotics hardware to impact the space Predicting Grasp Success with a Soft Sensing Skin and Shape-Memory
of what is possible with intelligent robotic systems. Actuated Gripper, IEEE, Piscataway, NJ 2019, pp. 7120–7127.
[23] C. Larson, J. Spjut, R. Knepper, R. Shepherd, Soft Robot. 2019, 6, 611.
[24] H. Huang, J. Lin, L. Wu, B. Fang, Z. Wen, F. Sun, Machine Learning-
Acknowledgements Based Multi-Modal Information Perception for Soft Robotic Hands,
Science Robotics, Washington, DC.
This work was in part supported by the National Science Foundation [25] C. Choi, W. Schwarting, J. Delpreto, D. Rus, IEEE Robot. Automat.
(NSF) Graduate Research Fellowship Program (GRFP) under Grant No.
Lett. 2018, 3, 2370.
DGE 1252522 and the National Oceanographic Partnership Program
[26] J. I. Kim, D. Kim, M. Krebs, Y. S. Park, Y.-L. Park, IEEE Robot. Automat.
(NOPP) under Grant No. N000141812843 (PM: Dr. Reginald Beach).
Any opinions, findings, and conclusions or recommendations expressed Lett. 2019, 4, 3481.
in this material are those of the author(s) and do not necessarily reflect the [27] A. Schmitz, Y. Bansho, K. Noda, H. Iwata, T. Ogata, S. Sugano,
views of the NSF or NOPP. in IEEE-RAS Int. Conf. on Humanoid Robots, IEEE, Piscataway, NJ
2015, p. 1044.
[28] D. Kim, J. Kwon, S. Han, Y. L. Park, S. Jo, IEEE/ASME Trans.
Mechatron. 2019, 24, 56.
Conflict of Interest [29] M. Rolf, K. Neumann, J. F. Queißer, R. F. Reinhart, A. Nordmann,
The authors declare no conflict of interest. J. J. Steil, Adv. Robot. 2015, 29, 847.

Adv. Intell. Syst. 2020, 2, 1900171 1900171 (7 of 8) © 2020 The Authors. Published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
 26404567, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.201900171 by Readcube (Labtiva Inc.), Wiley Online Library on [24/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.advintellsyst.com

[30] T. G. Thuruthel, E. Falotico, F. Renda, C. Laschi, Bioinspiration [41] J. M. Bern, P. Banzet, R. Poranne, S. Coros, in Proc. Robotics: Science
Biomimetics 2017, 12, 066003. and Systems, Robotics: Science and Systems 2019.
[31] C. Schlagenhauf, D. Bauer, K.-H. Chang, J. P. King, D. Moro, S. Coros, [42] M. Zhang, X. Geng, J. Bruce, K. Caluwaerts, M. Vespignani,
N. Pollard, Control of Tendon-Driven Soft Foam Robot Hands, Tech. V. Sunspiral, P. Abbeel, S. Levine, in Proc. IEEE Int. Conf. on
Rep., IEEE, Piscataway, NJ. Robotics and Automation, IEEE, Piscataway, NJ 2017, p. 634.
[32] K. H. Lee, D. K. Fu, M. C. Leong, M. Chow, H. C. Fu, K. Althoefer, [43] C. Della Santina, M. Bianchi, G. Grioli, F. Angelini, M. Catalano,
K. Y. Sze, C. K. Yeung, K. W. Kwok, Soft Robot. 2017, 4, 324. M. Garabini, A. Bicchi, IEEE Robot. Automat. Mag. 2017, 24, 75.
[33] Y. Ansari, M. Manti, E. Falotico, M. Cianchetti, C. Laschi, IEEE Robot. [44] R. F. Reinhart, J. J. Steil, Proc. Technol. 2016, 26, 12.
Automat. Lett. 2018, 3, 108. [45] T. G. Thuruthel, E. Falotico, M. Manti, C. Laschi, IEEE Robot. Automat.
[34] T. G. Thuruthel, E. Falotico, M. Manti, A. Pratesi, M. Cianchetti, Lett. 2018, 3, 1292.
C. Laschi, Soft Robot. 2017, 4, 285. [46] R. K. Katzschmann, A. D. Marchese, D. Rus, Soft Robot. 2015, 2, 155.
[35] M. Trumic, K. Jovanovic, A. Fagiolini, Kernel-Based Nonlinear Adaptive [47] OpenAI, I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin,
Control of Stiffness and Position for Soft Robots Actuators, Tech. Rep., B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas,
IEEE, Piscataway, NJ. J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan,
[36] J. Chen, H. Y. Lau, in Proc. of the 2nd Int. Conf. on Control, Automation W. Zaremba, L. Zhang, Solving Rubik’s Cube with a Robot Hand, 2019.
and Robotics, IEEE, Piscataway, NJ 2016, p. 103. [48] M. P. Deisenroth, C. E. Rasmussen, D. Fox, Robot. Sci. Syst. 2011,
[37] T. G. Thuruthel, E. Falotico, M. Cianchetti, F. Renda, C. Laschi, in 13th V, 57.
Int. Conf. on Informatics in Control, Automation and Robotics, Doctoral [49] I. Lenz, R. A. Knepper, A. Saxena, Robot. Sci. Syst. 2015, 11.
Consortium, Vol. 2, Science and Technology Publications, Setubal, [50] C. Lee, M. Kim, Y. J. Kim, N. Hong, S. Ryu, H. J. Kim, S. Kim, Int. J.
Portugal 2016, p. 303. Control Automat. Syst. 2017, 15, 3.
[38] H. Jiang, Z. Wang, X. Liu, X. Chen, Y. Jin, X. You, X. Chen, in Proc. IEEE [51] T. George Thuruthel, Y. Ansari, E. Falotico, C. Laschi, Soft Robot.
Int. Conf. on Robotics and Automation. IEEE, Piscataway, NJ 2017, 2018, 5, 149.
p. 6127. [52] C. Della Santina, C. Piazza, G. M. Gasparri, M. Bonilla, M. G. Catalano,
[39] H. Zhang, R. Cao, S. Zilberstein, F. Wu, X. Chen, in Proc. 10th G. Grioli, M. Garabini, A. Bicchi, IEEE Robot. Automat. Mag. 2017,
International Conference on Intelligent Robotics and Applications 24, 48.
(ICIRA), Vol. 10462, Springer, Cham, Switzerland 2017, pp. 173–184. [53] N. Cheney, R. MacCurdy, J. Clune, H. Lipson, ACM SIGEVOlution
[40] M. T. Gillespie, C. M. Best, E. C. Townsend, D. Wingate, 2014, 7, 11.
M. D. Killpack, in IEEE Int. Conf. on Soft Robotics, RoboSoft 2018, [54] N. Cheney, J. Bongard, V. SunSpiral, H. Lipson, J. R. Soc. Interface
2018, p. 39. 2018, 15, 20170937.

Adv. Intell. Syst. 2020, 2, 1900171 1900171 (8 of 8) © 2020 The Authors. Published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim