Precision

Precision (Number of significant decimals, Точность) is specification of an approximate evaluation of some quantity (experimental or numerical), characterizing its accuracy, precision.

Precision of some quantity indicates, how well it is evaluated.

Definition

Let \(M\) is range of allowed variation of some quality and
let \( s \) be maximal deviation of the evaluated quantity from some "true", "exact" value.
Then, precision

\(\displaystyle N = \log_{10} \left(\frac M s \right)\)

The precision is usually rounded down to integer. Then, it happens to be

\(\displaystyle P=\mathrm{round\_down} (N) \)

Examples

Temperature of water

In usual conditions (at the atmospheric pressure), temperature of water varies, roughly, from 273 Kelvin to 373 Kelvin.

Let one says, that water temperature is 299 Kelvins. If well measured, this may refer to the maximal deviation 0.5 , that is half of the last digit. Then, the precision

\( N = \log_{10}(100/0.5) = \log_{10}(200) \approx 2.301029995664 \approx 2 \)

The correct estimate of the precision at the evaluation of temperature of liquid is very important for the correct treatment of specifications of liquids, for example, the statement «The tea is absolutely cold!»
The Researcher is supposed to estimate the temperature and quality the precision of this estimate. The advanced guide how to treat specification «absolutely cold» is suggested in article «Female logic».

Scratched jeans

One researcher, being in a foreign country and having few cash, looks for some used clothes. He sees the jeans that looks a little bit scratched. The researcher reads the price price tag, recalculates the price to dollars, and tries to understand, why the used jeans cost two hundred dollars, higher than a new jeans. The relative error of the price (due to uncertainty at the money exchange) could be of order of 10 percent, while the researcher knows, that means usually cost from 8 to 20 dollars. The formula above would indicate, that the precision

\( N = \log_{10}(10\% * 200/ 12) \approx 0 \)

The researchers realizes, that the price is estimated with 0 significant decimals, and tries to express his opinion with the aboriginal dialect: why the used jeans cost higher than a new clothes. (However, the store owner explains that this is a special fashion style, extremely popular among bon vivants.)

In such a way, estimate of the Number of significant decimals helps the researcher to make the wise decision.

Exercise

Elsa has a two-story cottage in the countryside. She wants to update the stair. She calls the construction company. The agent asks to measure the altitude of the second floor.

Elsa puts the GPS on the floor and answers: «2834 meters above sea level».

GPS measures height with an error of half a meter.

In principle, the level of the floor can vary, roughly, from -432 meters to 9000 meters.

Did Elsa measure the height of the second floor with precision of five significant figures?

Will this accuracy be enough to choose the right stair for her cottage?

Rounding

Precision may refer to the problem of rounding. The solution is suggested to avoid these problems:

One friend of mine found:
Let all constants be round!
Let both, \( \pi \) and \(\mathrm e \)
Be equal to three,
And kilogram be 2 lb

Confusions

Some authors use confusing notation.
Especially this refers to terms related to Precision, number of significant decimals, uncertainty, Error.

The use of the confusing notations indicates, that the author does not understand
the difference between uncertainty, dispersion, error and mistake,
in the same way as some coauthors do not know the difference between Probability and math expectation;
or cannot distinguish an integral from a logarithm;
or even think, that \(\pi\) is an approximate number.

References

Keywords

Measurement, Precision

Точность