Guardians of Trust

Guardians
of trust
Who is responsible for trusted
analytics in the digital age?
KPMG International
Data & Analytics
kpmg.com/guardiansoftrust
Contents
© 2018 KPMG International Cooperative (“KPMG International”). KPMG International provides no client services and is a Swiss entity with which the independent member firms of the KPMG network are affiliated.
Foreword 2
Summary 4
Trust in an age of digital transformation 6
The four anchors of trust 10
The trust gap: A lack of executive confidence in analytics 12
Insight — Machines misbehaving: New risks 17
Who is to blame? 18
Governance: Taking responsibility for trusted analytics 22
Interview — Microsoft: Designing a principles-based approach 27
Building the governance of AI into the core business 28
Interview — Amsterdam ArenA: Building an ecosystem of trust 31
Eight areas of essential controls for trusted data and analytics 32
In summary 34
Methodology 36
About Intelligent Automation and Data & Analytics at KPMG 38
Contributors 39
Country contacts 40
Acknowledgements 40
Additional references 41
Foreword
Data breaches. Machine-based decisions. The rise of the robots.
Amid fears and uncertainties in the digital age, the value of trust in a business cannot be overstated.
Today, that trust relates not only to a company’s brand, products, services and people — but also to the
data and analytics (D&A) that are powering its technology.
However, KPMG International research shows that companies are struggling to build this trust.
In the recent Guardians of trust study, KPMG International commissioned Forrester Consulting to survey
almost 2,200 global information technology (IT) and business decision makers involved in strategy for
data initiatives. The survey found that just 35 percent of them have a high level of trust in their own
organization’s analytics.
At a time when machines are working in parallel with people, this study points to a clear need for
proactive governance of analytics in order to build trust.
But who should be responsible for trusted analytics? And what does good governance look like? As
organizations undergo digital transformation, with artificial intelligence (AI) sweeping through almost every
industry, is someone taking responsibility for the quality, effectiveness, integrity and resilience of D&A?
In the following report, we identify some emerging principles as well as some worrying opinions. The
report summarizes the research findings, discussions with industry leaders who are making strides in
building trust, and the recommendations and observations on the governance of analytics.
On behalf of KPMG International, I thank the senior executives who participated in the research and
interviews. Your candor, transparency and ideas have been invaluable in helping lead the way towards
greater trust between people and machines.
To discuss the issues raised in this report or to join us in the effort to bridge the trust gap, please contact
your local KPMG office or any of the authors listed at the back of this report.
Dr. Thomas Erwin

Global Head of KPMG Lighthouse
Center of Excellence for Data
& Analytics and Intelligent
Automation
Partner, KPMG in Germany
2 | Guardians of trust
At a time when machines are
working in parallel with people,
this study points to a clear need
for proactive governance of
analytics in order to build trust.
Guardians of trust | 3
Summary
A trusted organization has traditionally been anchored by the behaviors and decisions of trusted people. As people give
way to machines, a trusted organization (and a trusted platform) also requires trusted data and analytics.
KPMG International’s Guardians of trust report looks closely at the intimate relationship between trust and digital transfor-
mation within an organization — who is responsible for ensuring trusted analytics and what good governance can look like
in a digital world.
Trust in analytics is lacking* Trust in an age of digital transformation*

Trust is becoming a defining factor of an organization’s
success or failure. Underpinning a company’s license to
operate effectively, trust reduces uncertainty and builds
Only 35% of
respondents say they
resilience as well as:
have a high level of

trust in their own
organization’s use of
different types of
analytics
drives customer
influences reputation satisfaction and loyalty
and 25%
admit that they
either have limited
trust or active distrust.
enables global
inspires employees markets to function
Executives and Trust in a

customers are wary digital world
of technology
The need for trust is expanding
Rapid, uncertain tech from trust in brands, organizations
disruption can lead to unstable and their employees to also
levels of internal and public include trust in machines,
confidence. algorithms and analytics.
The trust gap grows: C-suite executives

question the trustworthiness of data,
analytics and intelligent automation*
Few decision makers trust the way their organization
uses different types of analytics. But the trust gap is
92% are
worried about the
not reducing with experience or time. impact on reputation
Understanding that trust in analytics is Levels of trust vary by geography*
founded on four key anchors The trust gap is not the same in every country and
decision makers may need to adjust their approach
depending on the market they are in.
Quality Effectiveness
65% 21%
high levels of trust
Trust in India vs US
Spreading the blame*

Everyone should share some level of responsibility and
Resilience Integrity accountability for faulty or untrustworthy analytics.
62% say that the blame for an

autonomous vehicle accident lies
Trusted analytics is not a vague concept or with the organization that
theory. At its core are rigorous strategies developed the software.
and processes that aim to maximize trust.
Like human, like machine Who holds organizational responsibility?*
The governance of machines should not be fundamentally It is not clear who within
?
different from the governance of humans. the organization has primary
? responsibility for ensuring
the trustworthiness and
?
accuracy of advanced
analytics and models.
A larger percentage says it
rests with the technology
function.
Key takeaways
If you can’t Prioritize Create
measure it, risks trust-impact
Creating the foundation you can’t personas
There are eight areas that form the basis for emerging manage it
standards, enablers and controls for trusted analytics.
Governance Processes Create a Stay legal Checklist

buddy manifesto
system for data and
analytics
Regulation Data
People & Don’t let Be flexible Create a

Technology
culture the board with horses mesh
off the for courses governance
hook framework
Strategic Alliances and
alignment supplier networks
* Source: A commissioned study conducted by Forrester Consulting on behalf of KPMG International, July 2017
Trust in an age of
digital transformation
Trust is a defining factor in an organization’s success or failure. Indeed, trust
underpins reputation, customer satisfaction, loyalty and other intangible assets,
which now represent nearly 85 percent of the total value of companies in the
S&P 5001, compared with tangible assets such as bricks and mortar. Trust also
can inspire employees, enables global markets to function, reduces uncertainty
and builds resilience.
It is no wonder that in the KPMG International Disrupt and and travel. Disruptors are leveraging data, sophisticated
grow 2017 Global CEO Outlook survey, 61 percent of analytics, robotics and, increasingly, artificial intelligence (AI)
nearly 1,300 chief executives said that building trust among to create new value propositions and business models.
customers and other external stakeholders is a ‘top three’
The age of AI also offers new ways of protecting public trust
priority for their organization. Almost three-quarters said their
as we shift from humans towards machines. In audit, for
organization is now placing a greater importance on trust,
example, cognitive systems can analyze millions of records
values and culture in order to sustain their long-term future.
and identify patterns to create more insights on a company’s
Organizations are also making unprecedented investments processes, controls and reporting. Algorithms, meanwhile,
in new technologies. KPMG professionals are seeing mass can be designed to reduce human biases in decision making,
experimentation and uptake in industries from banking and and blockchain can offer greater data security and new
insurance to telecommunications, healthcare, manufacturing distributed trust models.
Figure 1
Trust influences corporate reputations
and drives customer satisfaction
Building greater trust among external stakeholders
and customers is among the top three priorities of
The widespread use of AI will
my organization today. make it imperative — and
more difficult — to ensure
trusted analytics.
61%
overall
agree
Source: 2017 Global CEO Outlook, KPMG International
Trust inspires employees, enables
global markets to function, reduces
uncertainty and builds resilience.
From hype to humble reality Not surprisingly, some of those who manage analytics and
automation are concerned about the risks. The Guardians of
In a high-speed, highly digitized world, trust that has taken trust survey questioned 2,190 global senior business decision
years to build can be destroyed almost instantly. Amid the makers involved in setting direction for data and analytics
rise of machines, organizations require trust not only in their from nine countries. Ninety-two percent admitted they are
brands and people but also in their data and analytics. Getting worried about the reputational damage that inappropriate use
this right is not something that any enterprise can claim to of analytics could cause for their organization.
have mastered.
Figure 2
The impact of analytics on corporate reputation
How concerned are you about the following consequence if your organization's data and analytics
models do not work as intended or are inappropriately used?
92%
are moderately or highly
concerned about the
impact on corporate
reputation
Base: 2,190 global IT and business decision makers with involvement in setting strategy for data initiatives at their organizations
Source: A commissioned study conducted by Forrester Consulting on behalf of KPMG International, July 2017
Against this backdrop, C-suite executives are starting to

ask difficult questions about the trustworthiness of data,
analytics and intelligent automation.
How can decision

What types of
makers trust the
governance
insights they are
frameworks and
receiving when
controls are required
they don’t own the
when the risk-takers
source data?
are machines, not
humans?
How do I know that
How do we my models and
redesign processes algorithms are doing
How would we
when we don’t the right thing?
know if something
fully understand
has gone wrong,
the algorithms that
and how should we
power them?
manage that?
A double-edged sword The widespread use of AI will make it imperative — and
more difficult — to ensure trusted analytics. Indeed, as
While the digital age creates opportunities, it also creates organizations adopt more sophisticated analytics, machine
new concerns that can undermine trust across industries and learning models and automated decisions, many regulators
our society as a whole. are exploring new controls on all organizations that collect,
For example, constant news of data breaches, data misuse analyze and use customer and business data. It’s time to ask
and inaccuracies is eroding public trust. What’s more, how complex algorithms will be governed to help ensure fair
technology-driven disruptions can fuel increased nationalism treatment and accurate outcomes.
and protectionism as the media predicts job losses and And herein lies the challenge of the double-edged sword.
redundancies due to automation. There also is concern Organizations must embrace new technology, while
that the benefits of digital transformation will not be evenly ensuring high, stable levels of trust in an uncertain, fast-
distributed, therefore worsening disparities between the changing digital age.
haves and the have-nots.
It’s time to ask how

complex algorithms will be
governed internally to help
ensure fair treatment and
accurate outcomes.
Figure 4
concerned about misuse
of data and analytics on
corporate
are reputation
aremoderately orof highly
92%
The four anchors of trust How strong
How well does
your anchors
concerned about misuse
your organization
trust? A vie
align with bes
Figure
ase: 2,190 global IT and business decision 4 withof
makers
data and analytics on
involvement in setting strategy for data in
ource: A commissioned study conducted How strong corporate
by Forrester are your on
Consulting
reputation
anchors
behalf ofof trust?
KPMG, A 2017
July view
How well does your organization align with best
Quality of data, tools and
10%
se: 2,190 global IT and business decision makers with involvement in setting strategy for data in
ource: A commissioned study conducted by ForresterQuality
Consulting on behalf of KPMG, July 2017
D&A capabilities
Quality of data, tools and m
10%
re 4 Breaking the cycle of mistrust in D&A, it was proposed that Quality
In KPMG International’s 2016 report Building trust in analytics: As noted in that 2016 report, trusted analytics is not a vague
D&A Accuracy
concept or theory. At its core of models
are rigorous strategies
capabilities
and and p
w strong are your anchors of trust? A view
trust in analytics is founded on four key anchors: quality,
effectiveness, integrity and resilience. And over the past
from 2016.
processes that aim to maximize trust. Some are well known
but challenging, such as improving data quality. Others are
wellyear,
doesKPMGyour organization
professionals align with
have used this framework to help best
Effectiveness practice
relatively in each
new and undefined in thetrust
analyticsanchor?
sphere, such as
organizations assess their key trust gaps. ethics and integrity.
So how can the four anchorsUtility
improve of trustmodels
in analytics? and proc
re 4 Accuracy of models and pro
strong are your anchors of trust? A view from 2016.
well does your organization Quality of data, alignEffectiveness
withand
tools bestmethodologies
practice in each trust anchor?
10%
D&A regulatory complian
Utility of models and proce
Quality
Anchor 1: Quality
D&A capabilities 10% Integrity
Anchor 3: Integrity 22%
Organizations need to ensure that both inputs and analytics D&Aintegrity
In the context of trusted analytics, privacy
refers and ethical u
to ethical
Quality of data, tools and and
models are appropriate for the context in which the insights methodologies D&A
acceptable use, from regulatory
compliance compliance
with data privacy
will be used. In many cases, this starts with questions about
the quality of the underlying data. And as analytics become
Quality
more sophisticated and machines start to do their own
Accuracy of models and processes
learning, quality also extends to the models and algorithms.
16% 22%
Integrity
laws to less clear areas such as the ethics of profiling and
predicting behaviors. This anchor is a growing concern
of consumers, and it is rapidly becoming a key focus for
regulators and policy-makers, as they strive to assess the
1
D&A capabilities ‘fairness’ of analytical approaches. D&Aprivacy
D&A security
and ethical us
ctiveness
Utility of models and processes 16%
Resilience 20%
D&A governance
Accuracy of models and processes D&A security
ctiveness D&A regulatory compliance

Anchor 2: Effectiveness
Utility of models and
Base: 2,165 data and analytics decision-makers
Resilience
Source: a Anchor
commissioned
processes
4: Resiliencestudy conducted by Forrester
D&A governance
20%
13%
Note:
Effectiveness is about the extent to which models achieve
The chart shows the percentage of respondents w
Resilience is about optimizing data sources and analytics
Integrity
desired results, providing value to decision makers who D&A rely trust anchor.
models for the long term. Cyber security is a well known
Base: 2,165 data but
on the generated insights. When analytics are thought to be example, and analytics
executives shoulddecision-makers
also think about the changing
ineffective, or are usedD&A privacy and ethical use
in an inappropriate context, trust can use of their data sources and digital infrastructure. This kind
quickly erode.
D&A regulatory compliance Source: a commissioned study
of resilience is particularly conducted
important as analyticsby Forrester Co
become
self-learning and reliant on one another, using integrated
Integrity
D&A privacy
Note: The chart
D&A trust anchor.
securityand ethical use
shows
algorithms the
to acquire percentage
input data. of respondents wh
13% 20%
D&A
Measuring the four anchors of trust
In the 2016 survey, organizations were asked how they of respondents struggled to achieve excellence across each of
measured each of the trust anchors. We found that almost all the anchors. Just one in 10 respondents said they excelled in
organizations needed to close several capability gaps. developing and managing analytics, while only 13 percent said
they excelled in privacy and ethical use. Fewer than one-fifth
In fact, with the exception of regulatory compliance (where thought they performed well in ensuring accuracy.
organizations tended to perform strongest), the vast majority
Figure 3
How strong are your anchors of trust? A view from 2016
How well does your organization align with best practice in each trust anchor?
10%
Quality of data, tools and methodologies
Quality
D&A capabilities
22%
Accuracy of models and processes

16%
Effectiveness
Utility of models and processes
20%
44%
D&A regulatory compliance
Integrity 13%
D&A privacy and ethical use
D&A security
20%
Resilience
D&A governance
18%
Base: 2,165 data and analytics decision makers
Source: a commissioned study conducted by Forrester Consulting on behalf of KPMG International, July 2016
Note: The chart shows the percentage of respondents who selected 'describes our approach exactly' for all of the capabilities explored under the
D&A trust anchor.
The trust gap:
A lack of executive
confidence in
analytics
Given the power that it holds, trust in D&A should be a non-negotiable business
priority. Yet the Guardians of trust survey suggests there is a growing trust gap.
Businesses want the benefits that digital and automation can deliver, but they don’t
always trust the underlying analytics that power those machines.
In the survey, just 35 percent of respondents said they have This lack of trust does not seem to have shifted in the past
a high level of trust in their own organization’s use of different year. In the 2016 KPMG Building trust in analytics survey,
types of analytics. A quarter admitted that they either had 34 percent of respondents reported a high level of trust in
limited trust or active distrust. their operational analytics, and 38 percent professed high
trust in the analytics that drive their customer insights.
Figure 4
The trust gap
To what extent do you trust the way your organization uses different types of analytics?
High level of trust

35%
Usually trusted,
but some
reservations 40%
Limited trust, with
regular challenges,
signs of uncertainty 19%
Active distrust or
reluctance to use 6%
Businesses want the benefits
that digital and automation can
deliver, but they don’t always
trust the underlying analytics that
power those machines.
Reasons for mistrust This uncertainty may increase as businesses explore more
sophisticated D&A approaches. For example, AI systems may
Most people already have an idea of what ‘trusted data and be seen as a ‘black box,’ making important decisions when
analytics’ should mean in both their professional and their few people outside of analytics teams, data science labs and
personal lives. They want to know that the data and analytics technology centers can fully understand how.
are correct, and they want to know when something is wrong.
They also want data to be used in a way they understand, by “We often see organizations run dual processes — one
people they trust, and for a purpose they believe is valuable. managed by humans and one managed by machines — to
determine whether the machine-generated insights align
Some trust issues are straightforward. If management to those delivered by their tried-and-true, human-generated
has experienced unreliable data or poor insights, they are processes,” says Brad Fisher, National Leader D&A, KPMG in
likely to lose trust in the system they are using. But as the US. “That’s simply because many executives don’t have
technologies become more complex, the trust issues also confidence that the insights are reliable and accurate.”
become more complex.
In fact, according to the KPMG International Disrupt and
“Executives and managers are being asked to make major grow 2017 Global CEO Outlook survey, more than half of
decisions based on the output of an algorithm that they didn’t respondents are concerned about their ability to integrate AI
create and don’t always fully understand,” noted Dr. Thomas into their existing automation processes, and almost a third
Erwin, Head of Global Lighthouse, KPMG International. “As admitted they are not ready to adopt AI into the business.
a decision maker, you really need to have confidence that the
insights you are getting are reliable and accurate, but many “Many organizations are getting some comfort by testing
of these executives can’t even be sure if their models are of the potential for advanced analytics and then validating the
sufficient quality to be trusted. It’s an uncomfortable situation models against historical results and decisions,” Professor
for any decision maker to be in.” Sander Klous, D&A Leader, KPMG in the Netherlands, says.
“But when it comes to predictions based on new sources of
data, some organizations don’t have sufficient confidence to
hand these processes over to machines.”
“People will often forgive a small glitch in the data but they won’t be
as forgiving if the algorithms they rely on are patently wrong. It’s not
just about having trustworthy data, but it’s also about making sure
the data are being managed in a trustworthy way.”
— Colin Jones, Operations Manager at Ambulance Victoria in Australia
“ In 20 years’ time, we may see
a very different approach to
pharmaceutical product approval
where data is simply loaded
into a machine that then decides
whether a new drug will be
approved or not. While you can
validate a system, the system is
likely to make mistakes as well.
That worries me a bit because the
question is whether the public
will accept these mistakes as they
will be different from the ones
made by human beings.”
— Dr. Tilo Netzer, CEO of Pharmalex
Trust goes beyond technology performance depending on their market,” notes Julie Caredda, Partner
Data & Analytics Lead, KPMG in France. “In France, we see
There clearly is an opportunity for data and analytics to be companies that are very keen to deploy data-driven programs
proven accurate, effective and secure. But the research to increase sales. French decision makers usually trust their
suggests that the trust gap has as much to do with people’s analytics and they believe that data and analytics can unlock
expectations and perceptions as it does with the actual value and create new opportunities. But at the same time,
performance of the technology or the risks associated with these same companies are very concerned by the EU’s
it. Indeed, humans often prioritize their emotional response General Data Protection Regulation. This regulation includes
over their logical response, and more information does not severe penalties of up to 4 percent of worldwide turnover in
necessarily help people build trust. cases where companies are judged to be non-compliant.”
Local debate about regulation may have a negative affect on
For example, the level of trust that decision makers place in
levels of trust.
their analytics seems to vary significantly by geography. The
Guardians of trust survey shows that UK respondents are the In some cases, people build trust only with repeated use over
least likely to trust their analytics, with 43 percent reporting many years, as we have seen with consumer technologies
either limited trust or active distrust. The US is close behind with such as GPS, online shopping, ride-hailing services and
42 percent saying the same. But only 8 percent of respondents chatbots. Education can help bridge this trust gap earlier,
from India, 15 percent from Brazil and 19 percent from France but the real trust comes only after a user has had repeated,
admitted a lack of trust in their analytics. On the other hand, the successful experiences.
survey does not show any significant variation in trust across
Finally, the ends and means are also critical for trust. For
different roles in the organization (technical or non-technical) or
example, is the machine achieving the ‘right thing’ ethically,
between the industries surveyed. This seems counterintuitive,
as well as financially, for all those affected by it? Is it overseen
and suggests that the variations could be due to cultural
by people who exercise effective control and can manage
or social factors, or may relate to respondents’ personal
changes, risks and uncertainties? As machines take on more
experiences with analytics and AI.
day-to-day decision making, who is judging whether these
“The point is that the trust gap is not the same in every controls are proper and effective?
country, so decision makers may need to adjust their approach
Figure 5
The trust gap varies across countries
To what extent do you trust the way your organization uses different types of analytics?
High level of trust Usually trusted, but some reservations

Limited trust, with regular challenges, signs of uncertainty Active distrust or reluctance to use
Total 35% 40% 19% 6%

India 65% 28% 5% 3%
Brazil 47% 38% 13% 2%
South Africa 39% 42% 16% 3%
France 34% 46% 13% 6%
Australia 31% 34% 23% 11%
Germany 31% 51% 15% 3%
China* 28% 50% 20% 2%

United States 24% 34% 35% 7%
United Kingdom 21% 36% 28% 15%
*Inclusive of HKSAR
Base: 2,190 global IT and business decision makers with involvement in setting strategy for data initiatives at their
organizations.
INSIGHT
Machines misbehaving: New risks

As the use of machine learning and AI increases, we may be surprised by the new ways
that machines may fail us.
One likely factor in declining interactions — which can lead to The ‘bad-human’ behavior
trust is the sobering awareness high-speed ‘arms races’ — are problem
of high-profile or high-impact also a growing risk outside the Machines can be designed to
failures driven by algorithms, financial sector, in areas such as perform more fairly and ethically
which creates new challenges retail pricing optimization systems. than humans, but algorithms
for organizations, regulators and that use machine learning can
end users. In 2017, many public The ‘subhuman’ behavior also pick up bad habits or biases
companies were in the headlines problem from the human behavior they
with crises relating to data Just as algorithms can seem seek to emulate.6 For instance,
breaches or analytics failures. In incredibly smart, they can also act an algorithm used by courts to
some cases, these failures were in ways that seem surprisingly identify individuals with a high
not strictly technical but resulted dumb. For example, people have likelihood of reoffending was
from an inability to predict how been ‘injured by GPS’ when shown to have developed a bias
advanced digital technologies following directions that are against black offenders.7 In another
might perform, ‘misbehave’ or be outdated and wrong. Some cases example, Facebook algorithms
perceived in the real world. have occurred when people drove allowed advertisers to specifically
into Death Valley and went missing target users who expressed
The ‘superhuman’ behavior due to following wrong GPS data interest in anti-Semitic subjects.
problem or simply arrived in another country Facebook has since apologized,
Many machines are of course due to the same city names.4 removed bigoted categories and
superhuman by design. But increased human oversight of the
sometimes their performance Visual recognition is still in its automated ad-buying process.8
is almost ‘too good’ and we find infancy, despite impressive
ourselves unable to predict the advances in some specialized There have also been cases in
consequences. Many believe that areas such as medical image which algorithms have been
the first so-called ‘Flash Crash’2 diagnostics. According to a recent ‘taught’ by malicious users to
of 2010 was exacerbated when news report, Google believes that spread messages of hate and
hundreds of high-frequency its AI can identify and remove discrimination.9 Elsewhere, some
algorithms unexpectedly reacted around 83 percent of extremist state actors have allegedly used
to a single, albeit large, futures videos from YouTube, but that still algorithms to influence elections
trade. This reaction caused the leaves 17 percent that require and sow social discord.10 In
major market indexes to drop by human intervention.5 There are medicine, ethical issues may
more than 9 percent (including also cases where researchers increase as predictive algorithms
a decline of roughly 7 percent have ‘tricked’ deep-learning AI into open up avenues for treating
in a 15-minute interval) before misidentifying objects, making a conditions or behaviors that
a partial rebound, resulting in computer think, for example, that a people have not, and may never,
the temporary disappearance turtle is actually a rifle. This clearly actually express.11
of US$1 trillion in market value.3 is not a mistake that humans
Today, these kinds of bot-to-bot would easily make.
Who is to blame?
When an algorithm acts unexpectedly or leads to a negative outcome, who
should be held responsible? While we may like to blame our machines, they are
simply machines and, as such, cannot be held accountable for the decisions or
insights they produce.
Should responsibility rest with the people who programmed Furthermore, when asked who should hold responsibility for
the code? Or should it rest with the entities that sold, own or poor analytics-led business decisions, respondents said even
manage the machine? What about the users? more of the blame should move over from the business to
technology roles.
Respondents were asked which role in their organization
would be held primarily responsible for a decision resulting This allocation of blame seems to mirror respondents’ instincts
in significant financial or customer loss due to poor analytics. from their lives as consumers. For example, they were asked
Fifty-five percent said the technology function, including the which parties should be held primarily responsible if a fully
chief data officer (CDO) and data scientists, would carry the autonomous vehicle causes an accident. The most popular
brunt of the responsibility. response was the organization that developed the software,
ahead of the manufacturer, the passenger and regulators.
Just one in three respondents said responsibility would fall on
the core business, such as the CEO and functional leaders.
Figure 6
Where does responsibility lie?
If a poor business decision (one resulting in significant financial loss/loss of customers, etc.) is made based on insight from
advanced analytics, who currently bears primary responsibility for this decision today? Who should have responsibility?
Core business 33%

functions
25%
Technology 55%
functions and
service providers 62%
Regulatory and 11%

control Has
functions 13% Should have
“ I think the problem and the failure
are when different people from
different industries use analytics
when they are not really aware
of how the analytics work. I think
it’s a case of educating them to
help them understand where the
information is coming from and
how best to use it.”
— Global insurance executive
Figure 7
Spreading the blame
If a non-fatal accident was caused by an autonomous vehicle while the human driver was not in control, who do you believe
should be primarily responsible for the accident? (Select all that apply.)
The organization that developed

the software (that failed to
prevent the accident) 62%
The manufacturer who embedded
the software components into
the car 54%
The passenger who could
override the vehicle controls
54%
The manufacturer of the vehicle

50%
Regulatory bodies who enforce

safety 42%
The government who set the

regulations 21%
In both scenarios (corporate analytics and autonomous The C-suite is going to have one level of responsibility. The
vehicles), the responses also suggest that risk and regulatory business leader below them who maybe is making the
functions should shoulder a certain level of responsibility. decision has a different level, and the technical people have a
Inside the organization, 13 percent of respondents suggested different level of responsibility. But it’s hard to make this work
the blame should rest primarily with internal risk and audit well on the ground.”
functions — or with external risk functions such as regulators
Herein lies a key challenge: According to the research,
and third-party auditors. From a consumer perspective,
leaders’ impulse is to absolve the core business for decisions
42 percent of respondents said blame for an autonomous
made with machines. This impulse is understandable given
vehicle accident should rest with regulatory bodies.
technology’s legacy as a support service, the difficulty of
“It’s too easy for people to think ’It’s black and white. It’s assessing the trustworthiness of analytics, and the tendency
science, and you told me this is the answer’, when in fact it to give all things technical ‘to the experts’. However, many IT
is not so straightforward at all,” says Brad Fisher, National professionals simply do not have the domain knowledge —
Leader D&A, KPMG in the US. “The next step is to think about nor the overall capacity — required to ensure trust in D&A.
different levels and types of responsibility. Instead, as we explore in the coming pages, the responsibility
should rest with the core business.
Leaders’ impulse is to absolve the core

business for decisions made with machines.
“ It’s too easy for people to think ’It’s black and white. It’s
science, and you told me this is the answer’, when in
fact it is not so straightforward at all. The next step is to
think about different levels and types of responsibility.
The C-suite is going to have one level of responsibility.
The business leader below them who maybe is making
the decision has a different level, and the technical
people have a different level of responsibility. But it’s
hard to make this work well on the ground.”
— Brad Fisher, National Leader D&A, KPMG in the US
Governance: Taking
responsibility for
trusted analytics
While it is easy to point fingers when things go wrong, the harder task is to
proactively govern analytics in ways that build trust, resilience, integrity, quality and
effectiveness. While there is a growing sense that organizations should be more
accountable for their use of D&A, best practice in governance is yet to be defined.
Figure 8
Uncertainty about governance
Who in your organization has primary responsibility for ensuring the trustworthiness/accuracy of advanced analytics and
models?
Chief information officer (CIO) 19%

Chief data officer (CDO) 13%
Data scientists 11%
Data developers/Information architects 9%
Risk management leader 9%
C-level executive decision makers
7%
Internal auditors 6%
Third-party auditors
6%
Data technology and service providers
5%
All users of data and analytics within
the organization 4%
Regulatory bodies who enforce policy 4%
Functional business leaders (marketing,
sales, etc.) 3%
The government who set the regulations 3%
Other, please specify 0%
No one should have responsibility 0%
No one
0%
Don’t know/no opinion
0%
Who has primary responsibility for governing the Somewhat surprisingly, 18 percent of respondents suggested
trustworthiness and accuracy of advanced analytics? The that primary responsibility for ensuring trustworthiness
survey suggests significant uncertainty. The CIO received should sit with roles outside the organization altogether —
the most votes (although only 19 percent), and respondents with external suppliers, third-party auditors, regulators or
also nominated a wide range of other roles. Twelve percent government.
named the CDO, 13 percent named data scientists and
9 percent named data developers.
“ The CEO is ultimately held responsible but that means

companies need to have a system in place to ensure that things
have been developed appropriately and that the proper quality
checks are in place. If everything has been done according to
expectations, there may be nobody to blame. There is always a
risk when working with new technologies.”
— Dr. Tilo Netzer, CEO of Pharmalex
Who is holding the anchors in place? Misplaced accountability
To better grasp the governance challenge, respondents were These results are troublesome for the future. They again
asked who should hold responsibility for D&A across the suggest that, at best, there is a lack of clarity around the core
four trust anchors (identified on page 10). Across the board, business’ ownership of the impact of analytics. At worst, the
respondents maintained that the primary responsibility core business is absolved of responsibility altogether.
should rest within the technology function.
The purpose of risk functions and regulators is not to be
Only 27 percent said the business should take responsibility responsible for core business systems but rather to ensure
for effectiveness, ensuring that the outputs of models work effective controls and processes. Moreover, blaming
as intended and deliver value to the organization in practice. regulators for failing to anticipate misuse or bad outcomes
For integrity and resilience, including ethics, even fewer provides no shelter from potential reputational damage.
respondents assigned responsibility to the business. For
External providers, meanwhile, do play critical roles in digital
resilience, more than a quarter gave primary responsibility
transformation but that does give businesses an excuse for
to risk management leaders, internal auditors, third-party
avoiding responsibility.
auditors and regulatory authorities.
Figure 9
Different hands on different anchors
Who in your organization should have primary responsibility for the following areas of data and analytics?
Quality Effectiveness
Core business Core business
functions 32% functions 27%
Technology Technology
functions and 59% functions and 62%
service providers service providers
Regulatory and Regulatory and

control functions 8% control functions 9%
Integrity Resilience
Core business Core business

functions 18% functions 19%
Technology Technology
functions and 60% functions and 53%
service providers service providers
Regulatory and
control functions
20% Regulatory and
control functions
27%
“ It’s often the CDO, or the Chief Analytics
Officer, who is seen as a protector. But
that’s easy to say and much harder
to do. The reality now is that data
is percolating all over companies,
originating at many points, and it’s a
huge feat to try to control that.”
— Anthony Coops, D&A Leader, KPMG Australia
As for the technology function, most respondents in the needs. Few seem to have the resources or the inclination to
survey expect technical people to ensure the trustworthiness take on greater responsibility for the overall governance of AI
of analytics and take on wider responsibilities. This function and analytics across the core business.
includes what we will call the ‘data suite’ (D-suite) and the
“The responsibly often falls on me as the lead data architect,”
‘analytics suite’ (A-suite) — in other words, the CDO, data
says a director at a European financial services organization.
scientists, data developers and IT managers. But both the
“But I think the executives I provide the reports to are
D-suite and the A-suite are under increasing pressure from
also responsible. They need to use this as part of their
the business to manage multiple emerging objectives.
judgment — they need to take ownership”.
“It’s often the CDO, or the Chief Analytics Officer, who is
Traditional frameworks for IT governance have been seen
seen as a protector,” says Anthony Coops, D&A Leader,
as subsets of corporate governance. In practice they have
KPMG Australia. “But that’s easy to say and much harder
focused on managing performance and rely heavily on
to do. The reality now is that data is percolating all over
processes operated by the IT function. Ultimately, there
companies, originating at many points, and it’s a huge feat to
are huge areas of risk and responsibility that may not be
try to control that”.
effectively managed by traditional governance models. For
For example, conversations with CDOs suggest they are organizations undergoing rapid digital transformation there
expected not only to ensure the quality and integrity of data, is a clear need to find new ways of working that balance the
but also to create and manage external relationships, create desire for expert technical oversight with the reality that the
opportunities for data monetization, improve the reliability core business is increasingly digital.
of internal reporting and forecast the organization’s analytics
“ The responsibility often falls on me as

the lead data architect. But I think the
executives I provide the reports to are
also responsible. They need to use this
as part of their judgment — they need to
take ownership”.
— director, European financial services organization
INTERVIEW
Microsoft: Designing a principles-based approach

An interview with Emma Williams, General Manager for Microsoft’s Bing.com search
engine user experience and a recognized leader in the field of AI ethics.
What is your vision for AI? In such a rapidly evolving field, Who should be responsible for
how do you ensure you remain creating trust standards and
At Microsoft, our mission has
at the forefront of these issues? expectations for AI?
always been to empower every
person and every organization The reality is that we are inventing I’m not sure any one party is
to achieve more. And we see AI the future as we go along and responsible. That is why, last year,
as a fundamental way to amplify there are no existing design we founded the Partnership on AI
human ingenuity. Right now, our templates or user experience along with IBM, Google, Amazon
focus is on trying to blend what we patterns to model. My team and others. Today, the Partnership
call EQ, or ‘emotional intelligence,’ includes the classic UX designers includes more than 40 of the top
with traditional IQ, so we can that you would expect to see, but tech companies and about 30
create computers that can truly we also have people with PhDs in nonprofit organizations. As the
understand us in a much warmer human psychology and cognitive Partnership defines and debates
and more empathetic way. behavior, as well as ethicists, the ethical nuances, I believe we
anthropologists and psychologists. will start to see standards emerge
How are you addressing the that will hold for the broader
ethical implications of AI from a How do ethical and trust industry.
strategic perspective? considerations influence the
product development and What advice would you offer
We are thinking about a broader
user experience strategy at executives as they look to the
context that includes fairness,
Microsoft? future?
accountability, transparency and
ethics: an approach we call AI We’re very committed to a I think the best advice is to really
FATE. Ethics is one part of the principles-based approach when think deeply about who you are
equation, which is really all about designing our products and user building AI for and what they want
making sure humans remain at the experiences. And for the past year, to achieve. It’s really going back
center. But we are also thinking we’ve been working on developing to first principles about knowing
about how we ensure fairness a set of AI design principles your customer and focusing the
and remove bias. We’re thinking that will be embedded into our business on the value that you
about how we make the training products. Some are core to our give them. If you continue to do
of AI more accountable. And we’re vision of putting the human at that and really focus on making
talking about how we help people the center and understanding the the human the hero, it starts
understand the mechanism behind context within which the AI will be to become easier to wrap your
the AI and the impact it has on used. Others are more focused on head around some of the ethical
their lives and activities. creating a warmer experience — implications and decisions that
integrating that EQ aspect — need to be made.
when interacting with the AI.
Building the
governance of AI
into the core
business
KPMG professionals believe that as advanced analytics become integral to
business operations, the management of machines is as important as the
management of people. Therefore, the governance of machines must become a
core part of the governance of the whole organization. The goal should be to match
the potential power and risk of data and analytics with the wisdom to optimize
value, in the context of each organization and the society in which it sits.
The experience of early adopters suggests that AI is not “ The governance of machines must become
fundamentally about technology outputs, nor is it about replacing a core part of governance for the whole
people to reduce costs. Rather, it is about a blended human and
organization.”
digital workforce that shares decisions as well as tasks.
As organizations think about the behavior of machines as — Nadia Zahawi, Director, D&A, KPMG in the UK
parallel to the behavior of people, they also are considering
new models of governance to support the leaps in trust Governance: Changing frameworks of
that this new workforce requires. At a fundamental level, enablers and controls
ownership of machines must be held firmly by the CEO and
functional leaders. The ‘guardians of trust’ are of course not just people with
new responsibilities. We are also seeing the expansion of
Human oversight by a range of different domain experts is
standards and controls from the purely technical into softer
a critical part of governing advanced analytics. For example,
strategic, cultural and ethical domains. These standards and
systems at risk of high-impact ‘misbehavior’ require constant
controls are also expanding from early stages of the software
attention, which will be especially critical in contexts such
lifecycle to a whole-life, whole-system view.
as driving on busy roads or interacting with customers and
the public. In addition to technical and industry expertise, While there are many technical standards to support the
these kinds of scenarios require participation from ethicists, development of IT systems and tools, these standards need
anthropologists, psychologists, lawyers and ‘ordinary’ to be aligned, improved and adapted. There are recognized
members of the public. This oversight will help build trust in AI gaps and inconsistencies in areas of emerging risks such as AI
and optimize performance at the boundary between human testing, data privacy and ethical standards. For example, under
and machine. the European Union’s new General Data Protection Regulation
(GDPR), organizations that carry out large-scale monitoring
Figure 10
Top five recommendations for building public and customer trust in data and analytics (as
suggested by respondents)
1. Develop standards to provide guardrails for all organizations.
2. Modernize regulations to build confidence in D&A.
3. Increase transparency of algorithms and methodologies.
4. Create professional codes for data scientists.
5. Strengthen internal and external assurance mechanisms.
of individuals must appoint a data protection officer (DPO). In some cases, existing governance frameworks could
However, where DPOs do exist, their mandated role is often be adjusted to fit the analytical enterprise of the future.
limited, focusing on legal compliance with the GDPR. For example, a common and widely accepted governance
principle is the ‘three lines of defense’ framework, which
There are particular governance challenges where multiple
helps organizations clearly identify roles and responsibilities in
parties are involved. Shared data and analytics platforms are
the business and practice ongoing risk management.
becoming more common, for example in healthcare, scientific
research and ‘smart city’ initiatives. “We see clients asking 1. The business. As the first line of defense, the business
for help with the management of platforms that involve units must be aware of the analytics they are using and
multiple competitors and other players in a value chain, which be able to manage the four anchors of trust: the quality,
is not overseen by any clear guardian of trust, unlike Uber effectiveness, integrity and resilience of their algorithms.
or Facebook,” says Professor Sander Klous, D&A Leader,
2. Risk oversight. As the second line of defense, the risk-
KPMG in the Netherlands. “They need to create a very broad
oversight functions will need to establish policies and
ecosystem that delivers a range of services and adds value.
procedures that serve as guardrails for the organization.
We expect to see new frameworks put forward by regulators
or other authorities with certain rules and regulations, and 3. Assurance and audit. As the third line of defense,
obligations to hire an independent third party to ensure that independent assurance providers and internal auditors should
they comply with these rules and regulations. This role may validate the controls and identify potential areas of weakness.
become a type of data and analytics accountant”.
“ We see clients asking for help with the management of

platforms that involve multiple competitors and other
players in a value chain, which is not overseen by any clear
guardian of trust, unlike Uber or Facebook.”
— Professor Sander Klous, D&A Leader, KPMG in the Netherlands
Interestingly, in analytical organizations, these three lines conclusions about the areas of audit focus. In the future,
of defense are starting to evolve. In banking and high tech AI can increasingly allow auditors to obtain and analyze
sectors, for example, some of the policies and procedure information from non-traditional sources, such as all forms
checks created by the second line of defense may now be of media — print, digital and social — and, combined with
automated within the processes of the first line of defense, other information, draw a deeper, more robust understanding
for example by bots that check compliance or flag risks. of potential business risks. Predictive analytics capabilities
Meanwhile, internal auditors as part of the third line of will likely be developed further, enabling auditors to take
defense may now include data scientists who traditionally information from multiple sources and apply it to the outlook
only sat in operational roles within the first line of defense. and risks facing businesses.
A growing number of organizations recognize that audit As such, organizations also need to look beyond the human
evidence and business insight are two sides of the same oversight of machines in core operations. Organizations also
data. And by enabling auditors to analyze unstructured data, need to ensure trust in the human-machine partnerships,
AI can help decision makers to obtain evidence and insights which increasingly underpin corporate governance and
that allow them to make more confident decisions and ensure public trust in business as a whole.
KPMG Clara
KPMG Clara client collaboration is an online tool that facilitates
secure collaboration between KPMG and clients. It is used by
both group and component teams and promotes seamless
communication, giving continuous visibility into key audit
tasks in real time.
Through advanced tracking and monitoring capabilities, KPMG
teams can access the information they need when they need
it, whether that is the latest updates on audit discussion topics/
issues, the status of the prepared by client (PBC) process or the
progress of group reporting by component auditors.
“ KPMG Clara client collaboration is also the way we will

deliver outputs from the data and analytics capabilities,
which provide greater transparency to the audit and access
to deeper insights, which can ultimately enhance trust and
confidence in financial reporting.”
— Murray Raisbeck, D&A Leader, KPMG in the UK
INTERVIEW
Amsterdam ArenA: Building an ecosystem of trust

An interview with Henk van Raan, Chief Innovation Officer and Director of Facility
Management for Amsterdam ArenA, one of Europe’s leading stadiums and concert venues.
What is Amsterdam ArenA’s innovators — small entrepreneurs value — clearly demonstrating

vision for data and analytics? and big enterprises — who the benefits for consumers — it
are looking for opportunities becomes much easier to build
Data and analytics have allowed
to combine our data and their public trust. We also have a
us to develop a much richer
knowledge to create new services rigorous process in place to ensure
understanding of how we can
and a safer city. that our partners abide by the rules
better serve our visitors, but also
and regulations when dealing
keep them safe - how we can
Are you concerned about the with personal information. I
optimize their experience and
quality of the data or analytics think that the key to trust is clear
the way they move around and
that are being shared by your communication and transparency
inside the stadium. We use data
partners? about your actions. Our customers
from cameras, ticket sales and
We recognize that, for innovation understand that we use data
traffic systems to help steer the
to thrive, all parties must have to improve security, safety and
crowds and avoid congestion
a high level of trust in the enjoyment, so they are usually
around the stadium. Yet this is
data environment. This is why happy to be involved.
just the beginning. In the next
phase, we hope to leverage more we developed a robust data
governance process that complies Do you see a need for regulation
sophisticated AI technologies
with Dutch regulatory laws. or standards around the use of
to help automate some of
Analytics and data availability are data and analytics?
those decisions on a real-time
basis and, ultimately, become a always linked to a clear goal, which I suspect we will see more rigor
testing ground for future `smart provides a framework for both around the assessment of data and
city’ applications for the city of quality assessment and control. In analytics models, possibly including
Amsterdam. the future we expect to be able to a form of audit or alignment to
manage those processes better global standards. For now, I think
How are analytics approaches by leveraging blockchain or other the focus for companies and
developed and implemented distributed ledger systems. innovators is to be as transparent
within the arena? as possible with their customers
How are you developing public about how their data is being used,
The core Amsterdam ArenA team
trust in the way you collect and along with the potential benefits
is very small, which is why we
use customer data? or risks. In parallel, data analysts
focus on partnerships to recruit
I think everyone starts out should also actively keep asking
and develop certain capabilities
being cautious about how their themselves “why are we doing
around data, analytics, innovation,
information is being used. If you this?” and “is this still in line with
business modeling and so on. We
can make a very solid case for our common goal?”
see ourselves as an `innovation
arena’ or digital playground for
Eight areas of essential
controls for trusted
data and analytics
For an analytical enterprise, an effective framework of enablers and controls
is a board priority. KPMG professionals are seeing a range of new roles and
ways of working across the organization to strengthen the four anchors of
trust, effectively creating a distributed system of trust for the digital age. These
eight areas may form the basis of a more strategic, integrated and distributed
framework for governance of trusted analytics.
“Building trust requires customized actions,” says Dr. Thomas lot of people are struggling. It’s important to make your own
Erwin. “One size doesn’t fit all. People realize there needs assessment against the four anchors of trust. We are seeing a lot
to be controls around data and analytics. But there’s a line to of experimentation with new enablers and controls that are likely
walk between agility and control. That’s the trade-off where a to drive future standards and new governance frameworks.”
A wide range of ideas are emerging across eight essential enabling areas as the basis for ensuring trusted analytics:
1) Structure and roles Example
Many organizations are evolving from a centralized, IT- Partnering and ‘parenting’ algorithms
driven model — or from a ‘wild west’ model of multiple Banks are beginning to identify a nominated human partner
teams — toward a more organized, scalable, distributed in the core business, responsible for each critical algorithm.
approach. Many clients are creating powerful centers The concept of ‘parenting’ AI throughout its lifecycle is also
of excellence which, in addition to providing technical gaining traction. ‘Parenting’ recognizes that machines can be
expertise, also oversee a system of trust, including considered as demonstrating behaviors that change over time
risk management, innovation, standards, support and and in different contexts, and it will not be enough to teach or
education. This may be part of a governance model at test AI in a narrow technical context.
multiple scales: data ownership, algorithm ownership,
portfolio ownership and ecosystem ownership.
2) Regulation and standards Example
Beyond compliance with current legal and regulatory ‘Explainable AI’

requirements, organizations are also considering Even if it can be technically explained (which is not always the
proactive moves in areas of standards and ‘regulatory case), AI’s decision making process is usually too difficult for
lag’ (where there is a gap between what society expects most people to understand, including regulators. The purpose
and what is considered a minimum legal requirement). of ‘explainable AI’ is to ensure machine-learning systems have
This gap often creates trust issues and can lead to post- the ability to explain their rationale to humans and provide
hoc exposure to large fines or reputation damage. The some understanding of how they will behave in future. This
gap may need to be fixed manually. For example, some is particularly critical for high risk AI, for autonomous vehicles
social media sites are greatly increasing the number of and robots in defense, for example.
human content moderators.
3) People and culture Example
Many clients are particularly active in this area, including Human-machine job evaluations
initiatives in communication, education, training, job design A telco organization is changing its job evaluation policies to
and user experience design — within and beyond the walls take into account a mixed workforce of bots and humans.
of the organization. Traditional job evaluation assumed a lower job grading
for jobs with greater automation or decision support, but
the telco recognized that these staff were responsible for
increased output and higher-value functions. The company
changed its job evaluation approach to account for human-
machine workforces.
4) Strategic alignment Example
It is increasingly critical to ensure that the use of machines Ethics boards

aligns with cultural and social values as well as internal Silicon Valley giants Google, DeepMind, Facebook,
business strategy and objectives. Concepts of fairness, for Amazon, IBM, Apple and Microsoft joined forces to create
example, vary significantly between geographies and may the Partnership on Artificial Intelligence to Benefit People
need new mechanisms to ensure ongoing oversight and and Society.12 This organization aims to advance public
debate. understanding of the sector, as well as develop standards for
future researchers, which align with perceived cultural and
social values.
5) Processes Example
Beyond standard IT development and quality assurance Human-centered machine learning

processes, organizations are creating new processes Before developing an algorithm, Google focuses on how
to improve the design, performance and auditability people might solve a problem manually.13 This approach
of algorithms. Design thinking and customer journey connects the development team to the users and highlights
mapping are becoming standard capabilities to bring bot- the outputs that have the largest impact.
building out of the sole control of the technology function.
6) Data management Example
The explosion of data sources and uses is driving a Independently curated data
wide range of new approaches to data management. Amid the growing number of data sources, organizations are
The collection and use of data is generally shifting from using independently ‘curated’ data14 — verified by a trusted
a central IT function to a wide range of partners and source — before using it under agreed conditions. Some
stakeholders within an ecosystem that needs more careful companies are creating data depositories, enabling individuals
management and controls. or organizations to define and control the use of their data,
and put a wall between the data they are willing to share and
the data they are not.
7) Technology Example
As a control mechanism, organizations are increasingly RegTech

using machines to oversee other machines, while ensuring High-speed algorithmic financial trading relies on code to
that humans can intervene if needed. This approach can examine and regulate the vast volumes of trades, which
include built-in master off switches, intentionally reduced cannot be encompassed by humans.
complexity, ‘go-slow’ modes on processing speed, and
real-time ‘watchdogs’ of algorithm behavior.
8) Alliances and ecosystems Example
As organizations create common platforms for improved Blockchain

analytics, they need new trust frameworks to ensure all As Henk van Raan notes in his interview on page 31,
parties are managing data properly. Organizations can also Amsterdam ArenA is building common processes to ensure
achieve great value by partnering with others to create that partners abide by certain rules and procedures for the use
governance ecosystems. of data within the Amsterdam community. The organization
expects to leverage blockchain or other distributed ledger
systems to support the alliance.
In summary
In the digital age, trusted analytics will be a critical source of competitive
advantage. And the road to trust is paved with a thoughtful, strategic approach to
governance — one that stretches well beyond the traditional focus on technology
and risk. Indeed, amid the unprecedented use of D&A to drive decisions and the
growing human-machine workforce, the governance of machines must become
more strategic and integrated with governance of the entire enterprise.
Gone are the days when IT can be the catchall for anything related to technology. Instead, it’s time for the core business to take
responsibility for its analytics and AI — and ensure quality, effectiveness, integrity and resilience.
To that end, chief executives and functional leaders will need to manage machines as rigorously as they manage their people.
Such an approach requires standards and controls that go beyond the operational to also focus on the cultural, ethical and other
emerging considerations for managing advanced technology across the enterprise.
1 If you can’t measure it, you can’t manage it

Start by assessing the level of trust that users place in existing analytics and the related
controls. The Four Anchors of Trust are a useful framework for this exercise.
2
Prioritize risks
Don’t try to do everything at once. As the trust framework is being developed, focus on
those that pose the greatest risk for the organization and its stakeholders.
3 Create trust-impact personas

Ensure that your organization is thinking about all of the stakeholders impacted —
managers, customers, regulators, markets and media, for example.
4
Create a buddy system
Make sure all business-critical and high-risk algorithms have a human partner who is
accountable for their performance and impact.
5
Stay legal
Track the changing landscape of regulations and create alignment between not only
current regulations and requirements, but also those that can be predicted to arise in the
future.
6 Create a manifesto for data and analytics

Provide employees and process owners with appropriate guidelines, particularly in fast-
changing areas such as ethics, transparency, accountability and auditability.
7
Don’t let the board off the hook
Ultimately, executives and boards will remain responsible for the actions and inactions
of an organization. Educate them in technology risks and controls.
8
Be flexible with horses for courses
Don’t focus on just one standard approach or framework to define and improve
your controls.
9
Review your governance framework
No one party can be wholly responsible for algorithms, which means that governance and
controls must also be distributed among multiple internal and external parties.
10
Point fingers carefully
Create clear responsibilities and accountabilities for specific systems or algorithms to
spot potential challenges that may occur over time and prevent unexpected errors.
Methodology
Survey demographics: Country and employees
Using your best estimate, how many employees
In which country do you work? work for your firm/organization worldwide?
8% 13% 2%
8% 14%
26%
8% 13%
27%
11% 13%
13% 32%
13%
United States Germany South Africa 250 to 499 5,000 to 19,999

employees (medium) employees (large)
United Kingdom France Brazil
India China* Australia 500 to 999 employees 20,000 or more
(medium to large) employees
*Inclusive of HKSAR 1,000 to 4,999
Base: 2,190 global IT and business decision makers employees (large)
with involvement in setting strategy for data initiatives
at their organizations Note: percentages do not total 100 percent due to rounding.
Source: A commissioned study conducted by Forrester Base: 2,190 global IT and business decision makers with
Consulting on behalf of KPMG International, July 2017 involvement in setting strategy for data initiatives at
their organizations
Source: A commissioned study conducted by Forrester
Consulting on behalf of KPMG International, July 2017
Leaders from KPMG International, KPMG member

firms, clients and alliance partners also contributed
analysis and commentary to this study.
Note: percentages do not total 100 percent due to rounding.
Survey demographics: Industry and title
Which of the following best describes the Which title best describes your position at
industry to which your company belongs? your organization?
Directo
11% 14%
18% VP
18% CEO
18% 60% 26%
18%
18%
Telecommunications Healthcare/life sciences C-level executive Director (manage a

services (e.g., CEO, CMO) team of managers
Financial services/
Retail and high-level
Banking Vice president (in charge
contributors)
of one/several large
Insurance Retail departments)
Note: percentages do not total 100 percent due to rounding. Base: 2,190 global IT and business decision makers with
Base: 2,190 global IT and business decision makers with involvement in setting strategy for data initiatives at
involvement in setting strategy for data initiatives at their organizations
their organizations Source: A commissioned study conducted by Forrester
Consulting on behalf of KPMG International, July 2017
Sources
1
2015 Intangible Asset Market Value Study, Ocean Tomo
2
LSE Business Review, 2017
3
Findings Regarding The Market Events of May 6, 2010, 30 September 2010, America’s Securities and Exchange Commission and its
Commodity Futures Trading Commission
4
ARS Technica, 2016
5
Four ways Google will help to tackle extremism, 18 June 2017, Financial Times
6
Sheryl Sandberg, Facebook post, September 2017
7
The next Web, 2017
8
Facebook enabled advertisers to reach ‘Jew Haters’
9
Reuters, 2017
10
Social and Economic Sciences, 2014
11
Quartz, 2017 (Quartz, 2017)
12
Partnership on AI to Benefit People and Society
13
Google
14
http://www.dataversity.net/evolution-data-preparation-data-analytics/
About Intelligent
Automation and Data
& Analytics at KPMG
In a global environment defined by constant disruption,
business leaders need technology they can trust to
have worked across industries to help clients address their
long-term, strategic objectives. And as an internationally
inform their most important decisions. KPMG’s Intelligent regulated accounting and professional services network,
Automation and Data & Analytics (D&A) team has earned that member firms have an unwavering commitment to precision
trust with an evidence-based, business-first approach that’s and quality in everything they do. For more information, visit
at our core. For more than 100 years, KPMG professionals kpmg.com/trust.
Contributors
Dr. Thomas Erwin Maurice op het Veld
Global Head of KPMG Lighthouse Data & Analytics Leader, the Netherlands,
Center of Excellence for Data & Analytics and Intelligent KPMG International
Automation Partner, KPMG in the Netherlands
Partner, KPMG in Germany E: [email protected]
Thomas drives the growth of the global Data & Analytics Maurice is the D&A Leader for KPMG in the Netherlands and
practice. Since 2002, Thomas has worked with numerous has 18 years’ experience providing IT advisory and audit
KPMG global clients, helping them find the right Data & services to multinationals across industries. As well as
Analytics (D&A) approaches and solutions, including the managing the D&A team in the Netherlands, he is also
implementation and worldwide roll-out of respective tools responsible for a number of international clients, delivering
and technologies. advisory services. He is also a guest lecturer at the Tilburg
University — TIAS School for Business and Society.
Brad Fisher
Data & Analytics Leader, US, KPMG International Murray Raisbeck
National Leader D&A, KPMG in the US Data & Analytics Leader, UK, KPMG International
E: [email protected] Partner, KPMG in the UK
E: [email protected]
As the D&A Leader for KPMG in the US, Brad has presided
over the firm’s rapid growth in the scale and range of Murray is a partner within KPMG in the UK’s Financial
advanced capabilities of the firm’s Data & Analytics services. Services practice and the D&A lead in the UK. Murray also
Under his leadership, the KPMG Lighthouse was created as leads KPMG’s Global FinTech network. He has worked with
the firm’s D&A Center of Excellence, housing close to 300 clients across the financial sector and the FinTech network
advanced data scientists and Big Data software engineers in that spans 31 countries in key FinTech hubs. An auditor by
the US. Brad works with clients to help them unlock the value background, Murray works with global financial services
of AI by providing insight on what tools to innovate with and clients in the insurance, asset management and wealth
how to implement sustainable processes built on AI insights. management industries. Murray’s 18+ years of advisory
experience includes finance change and regulatory programs,
Anthony Coops Sarbanes Oxley implementation projects, accounting advice
Data & Analytics Leader, ASPAC, KPMG International on group reconstructions, Part VII transfers, and a variety of
Partner, KPMG Australia transactions.
Nadia Zahawi
Anthony is the Leader of D&A for the Asia-Pacific region and
Director, Global Data & Analytics
head of D&A for the Australian member firm. A partner with
KPMG in the UK
KPMG Australia, Anthony has global experience working
across multiple countries, where he helps clients address
specific needs in areas such as risk, optimization, workforce Nadia leads thought leadership for Global Data & Analytics
and customer analytics. based in the UK. Nadia works with both client-facing and
internal KPMG teams to explore the value of emerging D&A
Professor Sander Klous solutions and capabilities. She has 20 years of experience
Data & Analytics Leader, the Netherlands, working with leaders across multiple sectors to drive large-
KPMG International scale change, particularly with the UK government.
Partner, KPMG in the Netherlands
Sander is D&A Leader for KPMG in the Netherlands and
professor in Big Data ecosystems for business and society
at the University of Amsterdam. He has a PhD in high energy
physics and worked for over a decade on a number of projects
for CERN, the world’s largest physics institute in Geneva.
His best-selling book We are Big Data was runner-up for the
management book of the year award in 2015. His new book
Trust in a Smart Society is a top selling management book in
the Netherlands.
Country contacts
Julie Caredda Sai Chin Li
Partner, Data & Analytics, KPMG in France Data & Analytics Lead, KPMG China
[email protected] [email protected]
Anthony Coops Marlene Pappas

Partner, KPMG in Australia Data & Analytics Lead, KPMG in South Africa
Ricardo Santana Diniz Murray Raisbeck

Data & Analytics Lead, KPMG in Brazil Partner, KPMG in the UK
Dr. Thomas Erwin Abhijit Varma
Partner, KPMG in Germany Partner, Data & Analytics, KPMG in India
Brad Fisher Maurice op het Veld
National Leader D&A, KPMG in the US Partner, KPMG in the Netherlands
Professor Sander Klous

Partner, KPMG in the Netherlands
[email protected]
Acknowledgements
We greatly appreciate the participation of the executives who Thanks to the KPMG D&A team, Anthony Coops, Professor
completed the survey, particularly the following executives Sander Klous, Brad Fisher, Dr. Thomas Erwin, Bill Nowacki,
whom we interviewed: Emma Williams at Microsoft, Wilds Ross, Traci Gusher, Nadia Zahawi, Murray Raisbeck,
prof. dr. Cees de Laat at University of Amsterdam, Henk van Julie Caredda and Carina Schöllman for their insights and
Raan at Amsterdam ArenA, Dr. Tilo Netzer at Pharmalex, Colin guidance. Special thanks to the Forrester analyst, Mike
Jones at Ambulance Victoria. Gualtieri for his insight and support as well as Varun Sedov,
the Forrester consultant who helped from beginning to end,
survey to analysis. The team at Forrester has been integral to
the KPMG Trusted Analytics framework.
Additional references
kpmg.com/trust
kpmg.com/data
kpmg.com/socialmedia
The information contained herein is of a general nature and is not intended to address the circumstances of any particular individual or entity. Although we endeavor to
provide accurate and timely information, there can be no guarantee that such information is accurate as of the date it is received or that it will continue to be accurate in
the future. No one should act on such information without appropriate professional advice after a thorough examination of the particular situation.
© 2018 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG
International. KPMG International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-
vis third parties, nor does KPMG International have any such authority to obligate or bind any member firm. All rights reserved.
The KPMG name and logo are registered trademarks or trademarks of KPMG International.
The views and opinions expressed herein are those of the interviewees/survey respondents and do not necessarily represent the views and opinions of KPMG
International or any KPMG member firm.
Publication name: Building trust in analytics

Publication number: 133783-G
Publication date: February 2018

Guardians of Trust

Uploaded by

Copyright:

Available Formats

Guardians of Trust

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Guardians of Trust

Uploaded by

Copyright:

Available Formats

Guardians

Dr. Thomas Erwin

Trust in analytics is lacking* Trust in an age of digital transformation*

have a high level of

Executives and Trust in a

The trust gap grows: C-suite executives

Spreading the blame*

62% say that the blame for an

Like human, like machine Who holds organizational responsibility?*

Governance Processes Create a Stay legal Checklist

People & Don’t let Be flexible Create a

Source: 2017 Global CEO Outlook, KPMG International

Against this backdrop, C-suite executives are starting to

How can decision

It’s time to ask how

ctiveness D&A regulatory compliance

Accuracy of models and processes

High level of trust

High level of trust Usually trusted, but some reservations

Total 35% 40% 19% 6%

China* 28% 50% 20% 2%

United Kingdom 21% 36% 28% 15%

Machines misbehaving: New risks

Core business 33%

Regulatory and 11%

The organization that developed

The manufacturer of the vehicle

Regulatory bodies who enforce

The government who set the

Leaders’ impulse is to absolve the core

Chief information officer (CIO) 19%

“ The CEO is ultimately held responsible but that means

Regulatory and Regulatory and

Core business Core business

“ The responsibility often falls on me as

Microsoft: Designing a principles-based approach

1. Develop standards to provide guardrails for all organizations.

2. Modernize regulations to build confidence in D&A.

3. Increase transparency of algorithms and methodologies.

4. Create professional codes for data scientists.

5. Strengthen internal and external assurance mechanisms.

“ We see clients asking for help with the management of

“ KPMG Clara client collaboration is also the way we will

Amsterdam ArenA: Building an ecosystem of trust

What is Amsterdam ArenA’s innovators — small entrepreneurs value — clearly demonstrating

1) Structure and roles Example

2) Regulation and standards Example

Beyond compliance with current legal and regulatory ‘Explainable AI’

4) Strategic alignment Example

It is increasingly critical to ensure that the use of machines Ethics boards

Beyond standard IT development and quality assurance Human-centered machine learning

6) Data management Example

As a control mechanism, organizations are increasingly RegTech

8) Alliances and ecosystems Example

As organizations create common platforms for improved Blockchain

1 If you can’t measure it, you can’t manage it

3 Create trust-impact personas

6 Create a manifesto for data and analytics

United States Germany South Africa 250 to 499 5,000 to 19,999

Leaders from KPMG International, KPMG member