проверяется на уровне значимости 5 %

20:40 Dec 24, 2011 

If I claim that a beam will hold 100 tons of load +/-10 tons with statistical significance of 5% it means that, if I make 1000 such beams and load each of them with 110 tons, no more than 50 will crash. Theoretically, all may crash, but this is very unlikely. "Unlikely" means that the probability of such an outcome is very low.

Certainly, when I construct a building, and I use beams, which hold 100 tons +/-10% with significance of 5%, I will make sure that normally the beams will not be loaded by more than 50 tons (or whatever is prescribed by the standard, but anyway this load is lower than in the test).

18:48 Dec 24, 2011 

They are inter-related. The higher is the significance of an evaluation (the lower is the percentage of the significance), the more reliable is the evaluation, but the less accurate it is. Significance of 1% is much more reliable than of 5%. However, at 1% significance the accuracy (the relative and/or absolute error) is worse than it is at 5% significance. You have to have a leeway in accuracy for higher significance (lower percentage of significance) and higher reliability.

If I produce mechanical watches, I can announce that my watches can be ahead or behind the time within +/-60 seconds or +/-30 seconds per 24 hours. The implications are: if I announce 60 seconds, one customer in 1000 will ask me to replace the watch as less accurate than announced, if I announce 30 seconds, 20 customers in 1000 will claim my watches are not accurate enough.

However, if I announce 30 seconds rather than 60, I can charge higher price for my watches as for accurate devices.

So there is always a trade-off between the accuracy of a statistical evaluation and the significance thereof.