geoml.metrics

geoml.metrics.rmse(y_true, y_pred)

Root mean squared error.

Parameters:

• y_true (ArrayLike) – True values, one per location.

• y_pred (ArrayLike) – Predicted values, one per location.

Returns:

error (float) – The error, in the variable’s units.

Return type:

float

geoml.metrics.mae(y_true, y_pred)

Mean absolute error.

Parameters:

• y_true (array-like of shape (n_samples,)) – True values.

• y_pred (array-like of shape (n_samples,)) – Predicted values.

Returns:

error (float) – The error, in the variable’s units.

Return type:

float

geoml.metrics.bias(y_true, y_pred)

Mean error, prediction minus truth. Positive means overestimation.

Parameters:

• y_true (array-like of shape (n_samples,)) – True values.

• y_pred (array-like of shape (n_samples,)) – Predicted values.

Returns:

error (float) – The signed error, in the variable’s units.

Return type:

float

geoml.metrics.crps(y_true, y_pred)

Continuous ranked probability score, from samples. Lower is better.

The proper score for a probabilistic prediction against a measured value: it rewards putting probability near the truth and nothing else, so neither hedging with wide intervals nor feigning precision can improve it. With a single sample per location it reduces to the absolute error, which is the scale to read it on. Estimated by the energy form E|X - y| - E|X - X'| / 2, the pairwise term taken from the sorted samples in one pass.

Parameters:

• y_true (ArrayLike) – True values, one per location.

• y_pred (ArrayLike) – Candidate predictions per location, of shape (n_data, n_predictions) – measurement samples, normally.

Returns:

score (float) – The average score, in the variable’s units.

Return type:

float

geoml.metrics.variogram_score(y_true, y_pred, p=0.5, max_pairs=50000, coordinates=None, decluster=True)

How well the ensemble reproduces the differences between locations.

Scheuerer and Hamill’s (2015) proper score over pairs: for every pair of locations, the truth’s absolute difference to the power p against the ensemble’s mean one, squared and averaged. crps judges each location’s marginal and cannot see dependence; this is the score that punishes an ensemble whose realizations have the right histograms and the wrong spatial structure. Past the budget the pairs are strided down deterministically.

Given coordinates, each pair is weighted by w_i * w_j from geoml.math.geometry.declustering_weights(), so that a crowded patch of drilling counts once rather than once per hole. Without them the pairs are raw and the score describes the sampling as much as the field.

This is an estimate, and a biased one. The truth carries the likelihood noise while the realizations are of the ground with that noise integrated out, so a measured difference is systematically the wider of the two and the score never reaches zero however good the model is. The variogram figure corrects the same bias by raising its fan, which works there because a semivariogram is a second moment and independent noise adds a known variance to it. |difference| ** p is not a second moment and has no such constant, so putting the two sides on one footing here would mean drawing noise into the realizations, which is a seed inside a metric and a number that changes between calls. The bias is instead left in place: it is common to any two ensembles on the same locations, so read the score as a comparison between models on the same data rather than as an absolute quantity. geoml.plots.prepare.variogram() is the honest picture of the same question, and the one to reach for when the size of the disagreement matters rather than its ordering.

Parameters:

• y_true (ArrayLike) – True values, one per location.

• y_pred (ArrayLike) – Realizations at the same locations, of shape (n_data, n_realizations) – simulations, not measurement samples: dependence between locations is the thing under test.

• p (float) – The power. 0.5 is the authors’ recommendation.

• max_pairs (int) – The pair budget.

• coordinates (ArrayLike | None) – (n_data, n_dim) sample locations, needed to decluster.

• decluster (bool | float) – Weight pairs by cell-declustering weights. True chooses the cell size, a number fixes it, False leaves the pairs raw. Ignored when there are no coordinates to lay a lattice over.

Returns:

score (float) – Lower is better.

Return type:

float

References

Scheuerer, M., & Hamill, T. M. (2015). Variogram-based proper scoring rules for probabilistic forecasts of multivariate quantities. Monthly Weather Review, 143(4), 1321-1334.

geoml.metrics.interval_score(y_true, y_pred, alpha=0.05)

Interval score, based on confidence intervals estimated from y_pred.

Parameters:

• y_true (ArrayLike) – True values, one per location.

• y_pred (ArrayLike) – Candidate predictions per location, of shape (n_data, n_predictions).

• alpha (float) – One minus the interval’s nominal coverage.

Returns:

isc (float) – Interval score.

Return type:

float

geoml.metrics.bias_variance_decomposition(y_true, y_pred)

Compute bias and variance from predictions and true values.

Assumes multiple predictions for each true value (e.g., from bootstrapping or ensemble).

Parameters:

• y_true (ArrayLike) – True values, one per location.

• y_pred (ArrayLike) – Candidate predictions per location, of shape (n_data, n_predictions).

Returns:

• bias (float) – Mean squared bias.

• var (float) – Mean variance.

Return type:

tuple[float, float]

geoml.metrics.coverage(y_true, y_pred, probabilities=None)

How often the truth falls inside an interval of a given probability.

The numbers behind an accuracy plot. At every location the central interval holding a share p of the simulated values is built, and the fraction of true values inside it is counted. A model that knows what it does not know puts those fractions on the 1:1 line: below it the intervals are narrower than the errors they have to cover, above it the model is hedging.

Parameters:

• y_true (ArrayLike) – True values, one per location.

• y_pred (ArrayLike) – Simulated values at the same locations, of shape (n_data, n_realizations).

• probabilities (ArrayLike | None) – The nominal probabilities to check. Defaults to 0.05 to 0.95.

Returns:

• probabilities (array of shape (n_probabilities,)) – The nominal probabilities, as given.

• observed (array of shape (n_probabilities,)) – The share of true values actually inside each interval.

Return type:

tuple[NDArray[float64], NDArray[float64]]

geoml.metrics.goodness(probabilities, observed)

How close an accuracy plot sits to the 1:1 line. One is perfect.

Deutsch’s statistic. Intervals that are too wide are counted at half the weight of intervals that are too narrow: claiming a precision the model does not have is the worse mistake, since it is the one that leads someone to act on the number.

Parameters:

• probabilities (ArrayLike) – Nominal probabilities, as coverage() returns them.

• observed (ArrayLike) – Observed shares, as coverage() returns them.

Returns:

g (float) – One when the two agree everywhere, less as they part.

Return type:

float

geoml.metrics.aitchison_distance(comp_true, comp_pred)

Mean distance between two compositions, in the simplex’s own geometry.

The Euclidean distance between the centred log-ratios, which is what “close” means for parts of a whole: it ignores the closure and answers in ratios rather than in percentage points.

Parameters:

• comp_true (ArrayLike) – True compositions, of shape (n_data, n_parts), strictly positive.

• comp_pred (ArrayLike) – Predicted compositions, of the same shape.

Returns:

float – The distance, averaged over the locations.

Return type:

float