What does the residual standard deviation tell me?

How far the calibration points scatter about the fitted line, expressed on n − 2 degrees of freedom. It represents everything the straight line does not explain: instrument repeatability, short-term drift during the calibration, and any small mis-specification of the model. It is a Type A uncertainty contribution and belongs in the budget.

Why does the tool tell me to ignore R²?

Because R² measures how much of the total spread the fit explains, which is dominated by the range of the data rather than the quality of the fit. A genuinely curved response measured over a wide range still scores 0.999. The residual plot answers the question R² cannot: whether the residuals scatter randomly or form a pattern.

Why is the corrected value's uncertainty larger than the standard error of the line?

Because a corrected value carries two things: the uncertainty of where the line sits, and the scatter of the single new observation about it. That is the 1 in √(1 + 1/n + …). It also grows with distance from the centre of the calibration points, which is why points belong at the values you actually measure at.

How many calibration points should I use?

At least three, or there are no degrees of freedom and no residual standard deviation exists. Two points fit a straight line perfectly and prove nothing. Five or six spread across the working range is a common minimum, placed to cover the range rather than clustered where testing is convenient.

Calibration curve calculator

Paste your reference and indication pairs and get the least-squares fit, the residual plot that decides whether to believe it, and the residual standard deviation your uncertainty budget needs.

Calibration data

Reference and indication, one pair per lineUnit

Paste straight from a spreadsheet. Commas, tabs, semicolons or spaces all work, and a header row is ignored. The figures above are an example, not your data.

Runs entirely in your browser · nothing you enter is sent to a server

The fit

indication = 0.035714 + 1.0024 × reference

Residual standard deviation

0.013310 °C

On 4 degrees of freedom. This is a Type A contribution and belongs in the uncertainty budget.

R²

1.000

Reported because it is expected, not because it decides anything. Read the residual plot instead.

Standard error of slope

0.00015908

Standard error of intercept

0.0096327 °C

Residuals

Residual for each point against its reference value, in °C. Random scatter about the line means the model fits. An arc, a widening funnel or one point far from the rest means it does not, however high R² is.

The residuals scatter without an obvious pattern, which is what a straight line should produce.

Use the curve: an indication, corrected

Observed indication (°C)°C

This uncertainty is the contribution from the curve only. The uncertainty of the reference standard, the resolution of the instrument and its drift since calibration are all separate, and all belong in the budget with it. Never use the curve outside the range of the points it was fitted from.

1 line was skipped because it did not contain two numbers. A header row is the usual reason and is nothing to worry about.

How the calculation works

01Ordinary least squares with the reference as the known quantity and the indication as the observed one, which is the direction the calibration is performed in and the direction the least-squares assumptions hold for.
02Residual standard deviation on n − 2 degrees of freedom: s = √(Σrᵢ² / (n − 2)). This is a Type A uncertainty contribution in its own right.
03Standard errors of the slope and intercept from that residual spread and the spread of the reference values, Sxx = Σ(xᵢ − x̄)².
04Inverse prediction, which is how a calibration curve is actually used: u(x₀) = (s / |b|) · √(1 + 1/n + (x₀ − x̄)² / Sxx). Larger than the standard error of the line, and larger still towards the ends of the range.
05Outliers are flagged against a median absolute deviation rather than the standard deviation, because one large outlier inflates the very spread that would otherwise be used to find it.

Limitations

Straight-line fits only. If the residuals show an arc, the answer is a higher order or a split range, and this tool will say so rather than fit it for you.
The uncertainty shown is the contribution from the curve alone. The reference standard, the instrument's resolution and its drift since calibration are separate contributions and belong in the budget alongside it.
It assumes the error is in the indication and that the reference values are known. Where both carry comparable uncertainty, ordinary least squares understates the slope uncertainty.
Never use a fitted curve outside the range of the points it came from; the tool cannot stop you, and a polynomial in particular departs from reality quickly beyond its last point.

Frequently asked questions

What does the residual standard deviation tell me?: How far the calibration points scatter about the fitted line, expressed on n − 2 degrees of freedom. It represents everything the straight line does not explain: instrument repeatability, short-term drift during the calibration, and any small mis-specification of the model. It is a Type A uncertainty contribution and belongs in the budget.
Why does the tool tell me to ignore R²?: Because R² measures how much of the total spread the fit explains, which is dominated by the range of the data rather than the quality of the fit. A genuinely curved response measured over a wide range still scores 0.999. The residual plot answers the question R² cannot: whether the residuals scatter randomly or form a pattern.
Why is the corrected value's uncertainty larger than the standard error of the line?: Because a corrected value carries two things: the uncertainty of where the line sits, and the scatter of the single new observation about it. That is the 1 in √(1 + 1/n + …). It also grows with distance from the centre of the calibration points, which is why points belong at the values you actually measure at.
How many calibration points should I use?: At least three, or there are no degrees of freedom and no residual standard deviation exists. Two points fit a straight line perfectly and prove nothing. Five or six spread across the working range is a common minimum, placed to cover the range rather than clustered where testing is convenient.

Need the reasoning, not just the number?

Ask ValiTrac AI and see the standards evidence and the engine calculation behind the answer.

Ask ValiTrac AI

Calibration

Calibration curves: fitting, residuals and the uncertainty they add

Fitting a line to calibration data is the easy part. Judging whether the fit is honest, and carrying its uncertainty into the budget, is where most calibration curves go wrong.

2026-09-13 · 5 min read

Calibration

Choosing calibration points

Calibrate where you measure. Points should bracket the use range and include any temperature where a decision is made.

2026-09-13 · 3 min read

Uncertainty

Building an uncertainty budget step by step

Write the measurement model, list every input, evaluate each as a standard uncertainty, apply sensitivity coefficients, combine in quadrature, expand with k. Eight steps, one table.

2026-09-13 · 3 min read

Calibration