How To Calculate Measurement Uncertainty
How To Calculate Measurement Uncertainty
How To Calculate Measurement Uncertainty
Calculating measurement uncertainty is not easy. In fact, I speak with people every day who
are having problems with estimating uncertainty. Therefore, I decided to put together this
guide disclosing my exclusive seven step process to calculating measurement uncertainty.
In this guide, you will learn how to calculate measurement uncertainty in seven easy steps.
Also, you will learn what information you need to calculate uncertainty, how to identify
contributors to uncertainty, and how to evaluate your calculations to prevent overestimating
or underestimating uncertainty. Furthermore, I will share with you some of my exclusive tips
to help you calculate uncertainty like a pro.
Now, this guide is not a complete “how to” manual. Nor, will it answer all of your questions.
Instead, it should be used as a quick reference guide to simplify the uncertainty estimation
process into seven steps and learn some of my personnel secrets used when I calculate
uncertainty.
So, read this guide and use my advice to help you calculate uncertainty. If you have
questions, make sure to contact me. Additionally, feel free to use this guide to help you write
an uncertainty procedure for your laboratory.
Click here to download the simple uncertainty calculator for free!
Therefore, I developed a seven-step process that you can use every time you estimate
measurement uncertainty. Just follow the steps below to when you need to create an
uncertainty budget.
Step 1. Specify the Measurement Process and Equation
Before you dive in and begin calculating uncertainty, it is best to have a plan. The first part of
your plan should be to identify the measurement process or system that you wish to evaluate.
This will help you frame your uncertainty analysis and focus your attention on what matters
most.
Where applicable, identify the mathematical equation that characterizes the measurement
function.
If you are having trouble with this process, try answering the following questions:
1. What am I measuring?
2. How will I measure it?
3. What method will I use?
4. What equipment will I need?
5. What is the range (e.g. min and max) of my measurement capability?
6. What are my target test-points?
After answering the questions above, use your answers to identify what measurement process
you are evaluating. Then, add that information to your uncertainty budget. Take a look at the
image below.
Once you have outlined what you will be evaluating, you can move on to the next step.
If you are performing indirect measurements that require you to calculate your measurement
results, then you should evaluate the equation used to determine your measurement result.
Each variable in the equation will have its own uncertainty that will directly affect the
uncertainty associated with the calculated measurement result.
To help you out, think of using dead weight testers or calibrating torque transducers and
standard resistors. Each one of these measurement processes require you to use an equation to
calculate a result for comparison purposes. To estimate uncertainty, you will want to break
down the equation and evaluate the uncertainty of each variable in the equation.
If you want to learn more about specifying the measurement function and process for your
uncertainty analysis, check out this guide:
This process is not typically easy and can be very frustrating. So, stay calm, be patient, and
keep researching. You may be surprised by how many influences can affect your
measurement results.
Before you begin, I recommend that you find a book or guide on the measurement process
you are evaluating. Physics, Chemistry, and Engineering textbooks can come in handy for
understanding background and detailed information about your measurement process. If new
textbooks are too expensive, you should be able to buy reasonably priced used books on
websites like eBay, Amazon, or Chegg.
Other resources that you may want to consider are ASTM and ISO methods. However, if you
like free resources (like I do), you may want to search National Metrology Institute websites,
such as NIST, NPL, and BIPM. They may have downloadable guides related to your specific
measurement processes.
Finding sources of uncertainty can be difficult. It requires a lot of time and effort to conduct
research. It is the most time-consuming process when evaluating measurement uncertainty.
Based on my experience, finding factors that influence uncertainty typically requires 50% of
the time you spend estimating uncertainty. Take a look at the graph below to see how you
will typically spend your time estimating uncertainty.
However, if you spend time evaluating your process and conducting research, you should be
able to identify several sources of uncertainty for your analysis. Afterward, make a list of
these items. You will attempt to quantify them later.
Pro-Tip: Save and archive your notes and the resources you spent so much time searching
for. It will save you time in the future.
To find sources of uncertainty for your analysis, follow steps listed below:
When you need to find sources of uncertainty, it helps to have a list of available resources.
Below is a list of places, including links, that you can use to help you find sources of
uncertainty.
1. Manufacturer Manuals
2. Manufacturer Datasheets
3. White Papers
4. Technical Notes & Guides
5. Conference Papers
6. Textbooks
7. NIST Special Publications 250 Series
8. NIST Internal Reports
9. NIST Journal of Research
10. NPL Good Practice Guides
11. BIPM Publications
12. MSL Technical Guides
13. EURAMET Calibration & Technical Guides
14. Metrologia
15. CalLab Magazine
16. NCLSI Measure Magazine
For example, if you are evaluating the calibration of a torque transducer, you will first write
out the equation.
When you evaluate the equation further, you begin to account for other factors that influence
the equation. In this example, we begin to consider the radius of the torque arm and cable, the
mass of the weights and pan, and the local gravity. If needed, we can evaluate the equation
even further to account for more influences and increase the complexity of your uncertainty
analysis.
Now that you have identified the equation and the variables, you can start to research what
factors may cause changes or variations to each variable. Using the example above, think
about how temperature variations can cause thermal linear expansion or contraction of the
arm radius, and how it can affect air density which affects air buoyancy correction that can
vary the magnitude of applied force.
As you can see, evaluating equations can help you find sources of uncertainty. While this
process may seem easy, it can become quite difficult depending on the complexity of the
equation. Knowing the rules for the propagation of uncertainty can come in handy in step 5.
Most of the measurement functions that you evaluate will not have equations. So, you will
need to evaluate the measurement process to find the factors that influence measurement
uncertainty.
Start by evaluating the core elements of the measurement process, including the:
1. Method,
2. Equipment,
3. Personnel,
4. Environment,
5. Unit Under Test, and
6. Results
By evaluating these categories, you will find sources of uncertainty that influence
measurement results.
Take a look at the table below. Start breaking down each category to see what you find.
You may get lucky and find a paper or guide with a cause and effect (a.k.a. fishbone) diagram
or an uncertainty budget already prepared with information applicable to your uncertainty
analysis.
The main thing that you must do to find sources of uncertainty is to investigate and research.
To learn more about finding sources of measurement uncertainty, check out this guide:
To get started, you need to collect information and data related to your uncertainty analysis.
You should have found a lot most of this information in Step 2.
Take a look at the list below and gather the following items. You will need them to quantify
sources of uncertainty.
Using the items in the list above, you should be able determine how much uncertainty is
contributed from each source. If you need help, you can contact me for additional guidance or
hire me to analyze the data for you.
Next, you need evaluate the information that you have and find data that you will use for
estimating uncertainty. You need to find data related to your uncertainty analysis and
eliminate everything else from consideration.
a. measurement function,
b. measurement range, and
c. test-point.
Then, analyze the data that you have using appropriate methods of analysis to find the
magnitude of each uncertainty component. You can analyze data in many ways, so choose
methods that are appropriate for the data you are analyzing.
If you need help, get a quality statistics textbook or check the free NIST SEMATECH
Engineering Statistics Handbook. You can also check out some of my guides on quantifying
sources of uncertainty.
Finally, use your results to quantify each uncertainty component and add the values to your
uncertainty budget or uncertainty calculator.
You can add the uncertainty and unit of measurement directly into your uncertainty budget.
Or, you can add uncertainties, their unit of measurement, and a sensitivity coefficient to your
uncertainty budgets. The option is yours.
People use different techniques, and that is okay. Just make sure that you are able to explain
where your data came from and the way it is evaluated. I recommend adding detailed notes to
your uncertainty budgets. It will help you remember how you did it and why.
Sources of Uncertainty
Below, you will see a list of uncertainty components that you should include in every
uncertainty budget. Many of these factors are required by section 6 of the A2LA R205
requirements document. While this is not a requirement for everyone, I like their list of
minimum required uncertainty contributors and have decided to use them in each one of my
uncertainty analyses.
Additionally, I prefer to include more sources of uncertainty in my budgets since I find them
to typically be significant. The additional sources I like to consider are long-term stability,
bias, and drift.
1. Repeatability
2. Reproducibility
3. Stability
4. Bias
5. Drift
6. Resolution
7. Reference Standard Uncertainty
8. Reference Standard Stability
9. Other Significant Contributors
Repeatability
Reproducibility
Stability
Stability is an evaluation of the variability in your measurement process over time.
Bias
Drift
Drift is an evaluation of the systematic change in your measurement process or system over
time.
Resolution
How to Calculate Reference Standard Uncertainty
To learn more about sources of uncertainty and how to quantify them, check out these guides:
Uncertainty Types
The first step to characterizing your uncertainty components is to categorize them as either
Type A or Type B. Read the sections below to learn the difference between Type A and Type
B uncertainty.
Type A Uncertainty
Type B Uncertainty
1. Did you collect the data yourself via testing and experimentation?
a. If YES, go to question 2
b. If No, choose Type B
Probability Distributions
This is an important step because the probability distribution that you select will determine
how your source of uncertainty is converted to a standard deviation in the next step.
Although there are many different types of probability distributions that you can choose from,
the Normal (i.e. Gaussian) and Rectangular (i.e. Uniform) distributions are the most
commonly used.
Some of the most common probability distributions used to estimate uncertainty are;
Use the chart below to help you select the appropriate probability distribution.
To assign the appropriate distribution, consider how to characterize the data set for each
source of uncertainty.
If you evaluated resolution, environmental or physical influences, you may want to use a
rectangular distribution.
If you are not sure what distribution you should use, then it is typically less risk to assign a
rectangular distribution.
When selecting a probability distribution, you have two options to help you find the right
one.
This option is best for evaluating Type A data, but it more difficult and time-consuming if
you do not have statistical software. Most likely, you will not use this method. However, if
you do, you will find the instructions below.
Creating histograms is not for everyone and you can only do it if you have the data. In most
cases, you will not have the data needed to create a histogram because many of your
uncertainty components will be quantified by information that you find published in manuals,
papers, guides, etc.
Therefore, you will need to make some assumptions to select the right probability
distribution. To help you out, I created the probability distribution decision tree. It is the best
option for Type B data.
If you do not want to or cannot create a histogram of your data set, then try using the
probability distribution decision tree. All you need to do is answer the questions below:
1. Did you collect the data yourself via testing and experimentation?
a. If YES, choose Normal.
b. If NO, go to question 2.
2. Did others (e.g. manufacture, other lab, etc.) collect the data via testing and
experimentation?
If you want to learn more about probability distributions, check out the following guide:
Make sure to perform this task for each uncertainty contributor that you quantified in step 3.
To convert uncertainty components to standard deviations, follow the steps listed below:
Refer to the chart below to find the divisor associated with the probability distribution that
you selected in step 4.
Next, divide your uncertainty components by the appropriate divisor to convert them to a
standard uncertainty. Afterward, all of your contributors should be on the same confidence
level (i.e. 1-sigma or 68.27%) and equivalent to a standard deviation.
Normal Distribution
If you select a Normal distribution, then you will divide your uncertainty by it’s associated
coverage factor, k.
Use the Table from the JCGM 100:2008, Appendix G.
Take a closer look at the sources of uncertainty that you are evaluating to determine what
coverage factor you should use. Typically, your contributors will have a confidence level of
68%, 95%, or 99%. Respectively, this means that you will use a divisor of 1, 2, or 2.576.
Rectangular Distribution
If you select a Rectangular distribution, then you will divide your uncertainty component by
the square root of 3 or 1.7321.
U-Shaped Distribution
If you select a U-shaped distribution, then you will divide your uncertainty component by the
square root of 2 or 1.4142.
Triangular Distribution
If you select a Triangular distribution, then you will divide your uncertainty component by
the square root of 6 or 2.4495.
Quadratic Distribution
If you select a Quadratic distribution, then you will divide your uncertainty component by the
square root of 5 or 2.2361.
Log-Normal Distribution
If you select a Log-Normal distribution, then you will divide your uncertainty component by
2.3750.
Rayleigh Distribution
If you select a Rayleigh distribution, then you will divide your uncertainty component by
2.4477.
The most important thing to remember when converting uncertainty to standard deviation
equivalents is to make sure all the standard deviations are in the same units of measurement.
This is critical before calculating the combined uncertainty. Otherwise, your estimated
uncertainty will not be correct.
You cannot combine uncertainties with different units of measurement (without the use
of sensitivity coefficients).
If you have contributors with different units of measurement, you will need to use sensitivity
coefficients to convert them to units of measurement that match the measurement result or a
term relative to the measurement result (e.g. percent).
This is a mistake many people make when estimating uncertainty of measurement. So, make
sure to check for this before you calculate the combined standard uncertainty.
Sensitivity Coefficients
If you want to learn more about sensitivity coefficients, just click the link below to check out
my guide on sensitivity coefficients.
This will mathematically combine your uncertainty sources in quadrature. So, keep reading to
learn how to combine uncertainty.
To summarize the instructions above, simply square the value of each uncertainty source.
Next, add them all together to calculate the sum (i.e. the sum of squares). Then, calculate the
square-root of the summed value (i.e. the root sum of squares). The result will be your
combined standard uncertainty.
After you complete this process, you will have the combined standard uncertainty at a 1-
sigma level (i.e. 68.27% confidence) characterized by a Normal distribution per the Central
Limit Theorem.
When you combine uncertainty sources, you are also combining their probability
distributions.
According to the Central Limit Theorem, the sum of the set of independent random variables
(i.e. uncertainty sources) will approach a normal distribution regardless of the individual
variable’s distribution.
Therefore, the probability distribution associated with your combined uncertainty will now be
normal. Look at the image above for a visual representation.
If you are a more visual learner, like me, take a look at the process below to see if it makes
more sense.
Below, you will see the equation for calculating the combined uncertainty.
Where,
ci = sensitivity coefficient
ui(xi) = uncertainty of x
uc(y) = uncertainty of y
If the equation above looks confusing, you can try the simplified version below.
Where,
ui = uncertainty of x
uc(y) = uncertainty of y
This is the equation that I typically use since I usually include sensitivity coefficients (i.e. ci)
earlier in the process before I convert uncertainty components to standard deviations.
If you evaluate measurement uncertainty the same way, you should be able to use the
simplified equation. If you do not use sensitivity coefficients at all, you can use the simplified
equation as well.
Both equations give you the same result. So, use the equation that works best for you. If you
are using an excel spreadsheet calculator, you may find the function in the next section
beneficial.
If you use Microsoft Excel to estimate uncertainty, you can easily combine uncertainty using
the formula below. It is a combination of the square root and sum of squares function.
If you want to learn more about calculating the combined uncertainty, click the link to read
the guide:
Look at the image below to see the normal probability distribution when you expand your
uncertainty to 2-sigma or 95.45% confidence.
The result will be the expanded uncertainty and if you use a coverage factor of 2 or 1.96, you
will expand uncertainty to a 95% confidence level.
Check out the simplified equation below for calculating the expanded uncertainty.
Where,
EU – Expanded Uncertainty
k – Coverage Factor
CU – Combined Uncertainty
In the next section, you will learn some options for selecting a coverage factor.
The coverage factor is the multiplier that you will use to expand uncertainty to a 95%
confidence interval. However, you have some options. You can use:
Click the link below to see the Student’s T table.
Coverage Factors and Expanded Uncertainty
Note: To use the Student’s T table, you will need to calculate the effective degrees of
freedom using the Welch Satterthwaite equation.
To meet ISO/IEC 17025:2017 requirements, you must expand uncertainty to approximately
95%. Most people use an expansion factor (k) of 2 to achieve a confidence interval of
95.45%. However, you can also use an expansion factor of 1.96 for a confidence interval of
exactly 95.00%.
Additionally, you can find your coverage factor using the Student’s T table. This is not
common, but it is an option if you need it. Just calculate the effective degrees of freedom
using the Welch Satterthwaite equation and use the table to find the right coverage factor to
achieve a 95% confidence interval.
The choice is yours. Just make sure to select an expansion factor that you will consistently
use in each of your uncertainty analyses. Furthermore, it helps to know why you chose your
expansion factor so you can justify it to assessors (if they ask).
TIP: If you struggle with which option you should use, consider this:
a. Use a standard k-factor (e.g. 2 or 1.96) when your uncertainty budget contains plenty
of sources of uncertainty (Type A and Type B) each with their own value,
b. Use the Student’s T table when your uncertainty analysis is limited to mostly Type A
data and it is difficult to find or quantify other sources of uncertainty.
After finding your coverage factor (k), calculate the expanded uncertainty by multiplying the
coverage factor and the combined standard uncertainty. Use the formula below for guidance.
The result is the expanded uncertainty (i.e. U). This is your uncertainty in measurement
estimated to a 95% confidence interval.
Look at the image below to see the equation used in one of my uncertainty calculators.
That’s it! You have just learned how to calculate expanded uncertainty in 7 steps and
completed the process for the estimation of uncertainty in measurement.
However, you are not done yet. I recommend that you verify your calculations and evaluate
your results. In the next section, I will tell you how to evaluate uncertainty calculations for
appropriateness.
To learn more about coverage factors and expanded uncertainty or making CMC Uncertainty
equations for your scope of accreditation, click the links below:
To evaluate your expanded measurement uncertainty estimates, use one or more of the
following methods listed below. Then, determine if your expanded uncertainty is reasonable
and appropriate.
For this evaluation, review your expanded uncertainty and verify that it is larger than your
Reference Standard Uncertainty. If not, you have a problem and need to double-check the
value entered in your uncertainty budget and formulas used to calculate uncertainty.
For this evaluation, check the BIPM Key Comparison Database and make sure your
expanded uncertainty is larger than the value reported by your national metrology institute
(NMI). Sometimes this is not available, but you should at least check.
For this evaluation, review the Certificate of Analysis for the standard reference material that
your reference standard is traceable to and verify your uncertainty is larger than the
uncertainty of the SRM.
In this evaluation, compare your estimated uncertainty with other laboratories. Search your
accreditation body’s database and look at 3 to 5 other laboratory scopes of accreditation to
make sure that your expanded uncertainty is reasonably comparable. If not, you may have
overestimated or underestimated uncertainty.
This option works best for calibration laboratories since their uncertainty published in their
scopes of accreditation. This option is more difficult if you are a test lab. Most test labs do
not report their test uncertainties in their scope of accreditation which makes it difficult to
compare your capabilities with other laboratories.
For this evaluation, participate in a proficiency testing scheme and compare your uncertainty
with other laboratories. Then, determine whether or not your results are reasonable and
appropriate. Make sure that your expanded uncertainty is not significantly larger or smaller
than other participating laboratories.
When evaluating your proficiency test results, you really want to look at your z-score more
than the normalized error (En) value. Both can be beneficial, but the z-score compares your
laboratory’s performance with other participating labs.
If your En score is large or close to the value of one, then you may have reported an
understated value of uncertainty or you may have a problem with your measurement process.
If your z-score is large or close to the value of two, then you may have reported an
understated value of uncertainty. As a result, you need to evaluate your uncertainty budgets.
For this evaluation, perform a Repeatability and Reproducibility study in your laboratory.
Verify that your results are not larger than your uncertainty estimation. If so, you may have
understated your expanded uncertainty.
Evaluating your uncertainty budgets are critical. Although it is not a fool-proof process, it is
better than doing nothing. You do not want to go through all the work of calculating
measurement uncertainty only find mistakes during an assessment. It is better to perform the
hard work upfront than deal with all the paperwork and headaches that result from being cited
a deficiency. Furthermore, evaluating your uncertainty analyses gives you objective evidence
to support your results should an assessor question your expanded uncertainty.
I hope that the evaluations given in the section help you validate your results.
Conclusion
Estimating measurement uncertainty is not easy. It requires a lot of time and effort. However,
with the right processes, information sources, and tools, uncertainty analysis does not have to
be difficult.
In this guide, I have laid out seven steps to help you calculate measurement uncertainty.
While this is not a complete how-to guide, I have given you plenty of information to help you
perform uncertainty estimation yourself. So, start estimating uncertainty and tell me what
works for you and what you struggle with.
If you have additional questions or suggestions that will help improve this guide, contact me
and share your comments. I will glad to get your feedback.
If you need more help, please check out some of my uncertainty calculators and measurement
uncertainty training classes.
References
Castrup, H. (2007). Distributions for Uncertainty Analysis. Bakersfield, CA: Integrated
Sciences Group.
Miller, V. (2002). NISTIR 6919: Recommended Guide for Determining and Reporting
Uncertainties for Balances and Scales. Gaithersburg: National Institute of Standards and
Technology.
United Kingdom Accreditation Service. (2012). M3003: The Expression of Uncertainty and
Confidence in Measurement. Middlesex: United Kingdom Accreditation Service.