derivkit.forecasting.dali_bias module#

DALI systematic-bias forecasts.

The three main entry points return systematic MAP shifts, analytic posterior-mean shifts, and posterior-mean response shifts. Optional metadata explains where each estimator ends up and which terms contribute to its response. All shifts compare the same inference model with and without an injected data-model mismatch.

The implementation follows https://arxiv.org/abs/2610.02104.

derivkit.forecasting.dali_bias.build_dali_bias_tensor(function: Callable[[Sequence[float] | NDArray[floating]], floating | NDArray[floating]], theta0: Sequence[float] | NDArray[floating], cov: Sequence[Sequence[float]] | NDArray[floating], delta_nu: Sequence[float] | NDArray[floating] | Sequence[Sequence[float]], *, bias_order: int = 3, method: str | None = None, n_workers: int = 1, derivatives: dict[int, NDArray[float64]] | None = None, **dk_kwargs: Any) → dict[int, NDArray[float64]]#

Builds the systematic mismatch tensors entering the DALI bias expansion.

The tensors describe how the data-model mismatch delta_nu couples to successive derivatives of the model.

Parameters:

  • function – The scalar or vector-valued model function.

  • theta0 – Fiducial parameter values at which derivatives are evaluated.

  • cov – Covariance matrix of the observables.

  • delta_nu – Difference between two data vectors, which may represent a systematic mismatch or a difference between model predictions. Accepts a 1D array or a column vector of shape (n_observables, 1). The input is flattened internally.

  • bias_order – Highest mismatch tensor order to compute. Supported values are given in derivkit.forecasting.forecast_core.SUPPORTED_DERIVATIVE_ORDERS.

  • method – Numerical differentiation method or alias (e.g., "adaptive", "finite"). If None, uses derivkit.derivative_kit.DerivativeKit.default_method.

  • n_workers – Number of workers for per-parameter parallelization/threads. Default 1 (serial).

  • derivatives – Optional dictionary of model derivatives keyed by order. Existing derivatives are reused. Missing orders are computed and added to the supplied dictionary in place, modifying the input. If None, an internal dictionary is used.

  • **dk_kwargs – Additional keyword arguments passed to derivkit.derivative_kit.DerivativeKit.differentiate().

Returns:

A dictionary mapping derivative orders 1 up to and including bias_order to their corresponding mismatch tensors. Zeroth-order terms are omitted because they are independent of the parameter displacement.

Raises:

  • TypeError – If bias_order is not an integer.

  • ValueError – If bias_order is unsupported, theta0 is empty, or the data mismatch does not match the number of observables.

derivkit.forecasting.dali_bias.build_dali_map_shift(function: Callable[[Sequence[float] | NDArray[floating]], floating | NDArray[floating]], theta0: Sequence[float] | NDArray[floating], cov: Sequence[Sequence[float]] | NDArray[floating], delta_nu: Sequence[float] | NDArray[floating] | Sequence[Sequence[float]], *, expansion_order: int = 3, return_metadata: bool = False, method: str | None = None, n_workers: int = 1, derivatives: dict[int, NDArray[float64]] | None = None, **dk_kwargs: Any)#

Predict how the posterior maximum moves when a systematic is introduced.

Parameters:

  • function – Model predicting the observables from the parameters.

  • theta0 – Fiducial parameter values. Assumed to be the MAP without the systematic mismatch.

  • cov – Covariance matrix of the observables.

  • delta_nu – Actual difference between the systematically affected data vector and the fiducial model prediction. Its size is already included; no separate amplitude parameter is needed.

  • expansion_order – Number of perturbative MAP response orders (1 to 3).

  • return_metadata – If set to True, return the shift and a dictionary of explanatory results; otherwise return only the shift.

  • method – Numerical differentiation method or alias (e.g., "adaptive", "finite"). If None, uses derivkit.derivative_kit.DerivativeKit.default_method.

  • n_workers – Number of workers for per-parameter parallelization/threads. Default 1 (serial).

  • derivatives – Optional dictionary of model derivatives keyed by order. Existing derivatives are reused. Missing orders are computed and added to the supplied dictionary in place, modifying the input. If None, an internal dictionary is used.

  • **dk_kwargs – Additional keyword arguments passed to derivkit.derivative_kit.DerivativeKit.differentiate().

Returns:

If return_metadata=False, an array giving the MAP systematic shift: MAP with the mismatch minus MAP without the mismatch. Since the baseline MAP is assumed to be theta0, this is also the MAP displacement from the fiducial parameters.

If return_metadata=True, returns (shift, metadata). The metadata dictionary contains:

estimator: Predicted MAP parameter values with the mismatch.

This is theta0 + shift.

baseline: MAP without the mismatch, assumed to be theta0.

displacement: MAP with the mismatch minus theta0.

This equals shift because the baseline MAP is assumed to be theta0.

terms: Dictionary keyed by response order. Entry 1 is the

leading Fisher bias; entries 2 and 3 are the successive corrections. Their sum is the returned shift.

gaussian_terms: Terms obtained if the unbiased posterior is

approximated by its quadratic (Gaussian) Fisher form, while retaining the mismatch potential up to and including the requested order.

non_gaussian_terms: Differences between the DALI and Gaussian

terms at each order. They isolate corrections caused by the non-quadratic reference posterior within this expansion.

gaussian_contribution: Sum of gaussian_terms.

non_gaussian_contribution: Sum of non_gaussian_terms.

Adding the two contributions reproduces the MAP shift.

These Gaussian/non-Gaussian labels refer to the reference posterior, not to whether a term is linear or nonlinear in systematic amplitude.

derivkit.forecasting.dali_bias.build_dali_mean_response_shift(function: Callable[[Sequence[float] | NDArray[floating]], floating | NDArray[floating]], theta0: Sequence[float] | NDArray[floating], cov: Sequence[Sequence[float]] | NDArray[floating], delta_nu: Sequence[float] | NDArray[floating] | Sequence[Sequence[float]], *, theta: NDArray[float64], weights: NDArray[float64] | None = None, forecast_order: int = 3, response_order: int = 4, return_metadata: bool = False, method: str | None = None, n_workers: int = 1, derivatives: dict[int, NDArray[float64]] | None = None, **dk_kwargs: Any)#

Predict the posterior mean shift from its systematic response.

Expands the mean response around an unbiased reference posterior represented by samples or integration nodes.

Parameters:

  • function – Model predicting observables from parameters.

  • theta0 – Fiducial parameter values.

  • cov – Covariance matrix of the observables.

  • delta_nu – Full systematic difference in the data vector.

  • theta – Samples or integration nodes representing the unbiased reference posterior, with shape (n_samples, n_parameters).

  • weights – Weights representing the reference posterior at the supplied points. If None, equal weights are used, appropriate for equally weighted posterior samples. For integration nodes, the weights must include the reference posterior density and integration measure. Weights are normalized internally.

  • forecast_order – Highest model derivative order included in the systematic mismatch potential. Supported values are given in derivkit.forecasting.forecast_core.SUPPORTED_FORECAST_ORDERS.

  • response_order – Highest order retained in the systematic amplitude expansion of the posterior mean. Must be 1, 2, 3, or 4.

  • return_metadata – If True, return the shift and explanatory metadata.

  • method – Numerical differentiation method or alias (e.g., "adaptive", "finite"). If None, uses derivkit.derivative_kit.DerivativeKit.default_method

  • n_workers – Number of workers for per-parameter parallelization/threads. Default 1 (serial).

  • derivatives – Optional dictionary of model derivatives keyed by order. Existing derivatives are reused. Missing orders are computed and added to the supplied dictionary in place, modifying the input. If None, an internal dictionary is used.

  • **dk_kwargs – Additional keyword arguments passed to derivkit.derivative_kit.DerivativeKit.differentiate().

Returns:

An array containing the predicted systematic shift of the posterior mean, calculated by summing the response terms through response_order. The shift is the posterior mean with the mismatch minus the posterior mean without it.

If return_metadata=True, returns (shift, metadata) instead. The metadata dictionary contains:

estimator: Predicted posterior mean with the mismatch.

This is the reference posterior mean plus the predicted shift.

baseline: Posterior mean without the mismatch, calculated from

the supplied samples and weights.

displacement: Predicted posterior mean with the mismatch minus

the fiducial parameters theta0.

terms: Dictionary mapping each response order to its contribution

to the predicted posterior mean shift for the supplied mismatch. Their sum equals the returned shift.

Raises:

  • TypeError – If forecast_order or response_order is not an integer.

  • ValueError – If either order is unsupported, or if the supplied reference samples or weights have invalid shapes or weights are not finite, nonnegative, or have zero total weight.

derivkit.forecasting.dali_bias.build_dali_mean_shift(function: Callable[[Sequence[float] | NDArray[floating]], floating | NDArray[floating]], theta0: Sequence[float] | NDArray[floating], cov: Sequence[Sequence[float]] | NDArray[floating], delta_nu: Sequence[float] | NDArray[floating] | Sequence[Sequence[float]], *, return_metadata: bool = False, method: str | None = None, n_workers: int = 1, derivatives: dict[int, NDArray[float64]] | None = None, **dk_kwargs: Any)#

Predict the systematic shift of the posterior mean analytically.

Uses third-order DALI tensors to calculate the perturbative MAP shift and the change in the posterior mean relative to the MAP.

Parameters:

  • function – Model predicting observables from parameters.

  • theta0 – Fiducial parameter values and assumed baseline MAP.

  • cov – Covariance matrix of the observables.

  • delta_nu – Difference between the systematically affected data vector and the fiducial model prediction, including its magnitude.

  • return_metadata – If True, return the shift and explanatory metadata.

  • method – Numerical differentiation method or alias (e.g., "adaptive", "finite"). If None, uses derivkit.derivative_kit.DerivativeKit.default_method.

  • n_workers – Number of workers for per-parameter parallelization/threads. Default 1 (serial).

  • derivatives – Optional dictionary of model derivatives keyed by order. Existing derivatives are reused. Missing orders are computed and added to the supplied dictionary in place, modifying the input. If None, an internal dictionary is used.

  • **dk_kwargs – Additional keyword arguments passed to derivkit.derivative_kit.DerivativeKit.differentiate().

Returns:

An array containing the predicted systematic shift of the posterior mean, defined as the posterior mean with the mismatch minus the posterior mean without it.

If return_metadata=True, returns (shift, metadata) instead. metadata is a dictionary containing:

estimator: Predicted posterior mean with the mismatch.

baseline: Predicted posterior mean without the mismatch.

This need not equal theta0.

displacement: Predicted posterior mean with the mismatch minus

the fiducial parameters theta0.

map_shift: Predicted systematic shift of the posterior MAP.