Skip to content

halfnormal

halfnormal

Half-Normal distribution for ngboost-lightning.

HalfNormal 

HalfNormal(params: NDArray[floating])

Bases: Distribution

Half-Normal distribution with log-scale parameterization.

Internal parameter is [log_scale] where scale = exp(log_scale). The PDF is sqrt(2/pi) / scale * exp(-y^2 / (2*scale^2)) for y >= 0.

Note on Fisher Information

For HalfNormal(log_scale), the Fisher Information is the constant [[2]] for every sample. This means the natural gradient is simply d_score / 2.

ATTRIBUTE DESCRIPTION
n_params

Always 1 (log_scale).
scale

Scale parameter values, shape [n_samples].

TYPE: NDArray[floating]

Construct HalfNormal from internal parameters.

PARAMETER DESCRIPTION
params

Internal parameters, shape [n_samples, 1]. Column 0 is log(scale).

TYPE: NDArray[floating]

Source code in ngboost_lightning/distributions/halfnormal.py 
30
31
32
33
34
35
36
37
38
39
40
def __init__(self, params: NDArray[np.floating]) -> None:
    """Construct HalfNormal from internal parameters.

    Args:
        params: Internal parameters, shape ``[n_samples, 1]``.
            Column 0 is log(scale).
    """
    self.log_scale: NDArray[np.floating] = params[:, 0]
    self.scale: NDArray[np.floating] = np.exp(self.log_scale)
    self._dist = sp_halfnorm(scale=self.scale)
    self._params = params

fit staticmethod 

fit(
    y: NDArray[floating],
    sample_weight: NDArray[floating] | None = None,
) -> NDArray[floating]

Estimate initial log_scale from non-negative target data.

Uses scale = sqrt(E[y^2]) which is the MLE for HalfNormal.

PARAMETER DESCRIPTION
y

Target values, shape [n_samples]. Should be >= 0.

TYPE: NDArray[floating]

sample_weight

Per-sample weights, shape [n_samples].

TYPE: NDArray[floating] | None DEFAULT: None

RETURNS DESCRIPTION
NDArray[floating]

Parameter vector [log(scale)], shape [1].
Source code in ngboost_lightning/distributions/halfnormal.py 
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
@staticmethod
def fit(
    y: NDArray[np.floating],
    sample_weight: NDArray[np.floating] | None = None,
) -> NDArray[np.floating]:
    """Estimate initial log_scale from non-negative target data.

    Uses ``scale = sqrt(E[y^2])`` which is the MLE for HalfNormal.

    Args:
        y: Target values, shape ``[n_samples]``. Should be >= 0.
        sample_weight: Per-sample weights, shape ``[n_samples]``.

    Returns:
        Parameter vector ``[log(scale)]``, shape ``[1]``.
    """
    mean_sq = float(np.average(y**2, weights=sample_weight))
    scale = max(np.sqrt(mean_sq), 1e-6)
    return np.array([np.log(scale)])

score 

score(y: NDArray[floating]) -> NDArray[floating]

Per-sample negative log-likelihood.

PARAMETER DESCRIPTION
y

Observed target values, shape [n_samples]. Must be >= 0.

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

NLL values, shape [n_samples].
Source code in ngboost_lightning/distributions/halfnormal.py 
62
63
64
65
66
67
68
69
70
71
def score(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Per-sample negative log-likelihood.

    Args:
        y: Observed target values, shape ``[n_samples]``. Must be >= 0.

    Returns:
        NLL values, shape ``[n_samples]``.
    """
    return -self._dist.logpdf(y)

d_score 

d_score(y: NDArray[floating]) -> NDArray[floating]

Analytical gradient of NLL w.r.t. [log_scale].

Derivation

NLL = -log(sqrt(2/pi)) + log(scale) + y^2 / (2*scale^2) d(NLL)/d(scale) = 1/scale - y^2/scale^3 d(NLL)/d(log_scale) = scale * d(NLL)/d(scale) = 1 - y^2/scale^2

PARAMETER DESCRIPTION
y

Observed target values, shape [n_samples].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Gradient array, shape [n_samples, 1].
Source code in ngboost_lightning/distributions/halfnormal.py 
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
def d_score(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Analytical gradient of NLL w.r.t. [log_scale].

    Derivation:
        NLL = -log(sqrt(2/pi)) + log(scale) + y^2 / (2*scale^2)
        d(NLL)/d(scale) = 1/scale - y^2/scale^3
        d(NLL)/d(log_scale) = scale * d(NLL)/d(scale)
            = 1 - y^2/scale^2

    Args:
        y: Observed target values, shape ``[n_samples]``.

    Returns:
        Gradient array, shape ``[n_samples, 1]``.
    """
    n = len(y)
    grad = np.empty((n, 1))
    grad[:, 0] = 1.0 - y**2 / self.scale**2
    return grad

metric 

metric() -> NDArray[floating]

Fisher Information: [[2]] for each sample.

Derivation

E[(d(NLL)/d(log_scale))^2] = E[(1 - Y^2/s^2)^2] = 1 - 2*E[Y^2]/s^2 + E[Y^4]/s^4 = 1 - 2 + 3 = 2

RETURNS DESCRIPTION
NDArray[floating]

FI tensor, shape [n_samples, 1, 1].
Source code in ngboost_lightning/distributions/halfnormal.py 
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
def metric(self) -> NDArray[np.floating]:
    """Fisher Information: ``[[2]]`` for each sample.

    Derivation:
        E[(d(NLL)/d(log_scale))^2] = E[(1 - Y^2/s^2)^2]
        = 1 - 2*E[Y^2]/s^2 + E[Y^4]/s^4
        = 1 - 2 + 3 = 2

    Returns:
        FI tensor, shape ``[n_samples, 1, 1]``.
    """
    n = len(self.scale)
    fi = np.full((n, 1, 1), 2.0)
    return fi

natural_gradient 

natural_gradient(y: NDArray[floating]) -> NDArray[floating]

Natural gradient (fast path since FI is constant 2).

PARAMETER DESCRIPTION
y

Observed target values, shape [n_samples].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Natural gradient, shape [n_samples, 1].
Source code in ngboost_lightning/distributions/halfnormal.py 
108
109
110
111
112
113
114
115
116
117
def natural_gradient(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Natural gradient (fast path since FI is constant 2).

    Args:
        y: Observed target values, shape ``[n_samples]``.

    Returns:
        Natural gradient, shape ``[n_samples, 1]``.
    """
    return self.d_score(y) / 2.0

mean 

mean() -> NDArray[floating]

Conditional mean: scale * sqrt(2/pi).

RETURNS DESCRIPTION
NDArray[floating]

Mean values, shape [n_samples].
Source code in ngboost_lightning/distributions/halfnormal.py 
119
120
121
122
123
124
125
126
def mean(self) -> NDArray[np.floating]:
    """Conditional mean: ``scale * sqrt(2/pi)``.

    Returns:
        Mean values, shape ``[n_samples]``.
    """
    result: NDArray[np.floating] = self.scale * np.sqrt(2.0 / np.pi)
    return result

sample 

sample(n: int) -> NDArray[floating]

Draw n samples per distribution instance.

PARAMETER DESCRIPTION
n

Number of samples to draw.

TYPE: int

RETURNS DESCRIPTION
NDArray[floating]

Samples, shape [n, n_samples].
Source code in ngboost_lightning/distributions/halfnormal.py 
128
129
130
131
132
133
134
135
136
137
def sample(self, n: int) -> NDArray[np.floating]:
    """Draw n samples per distribution instance.

    Args:
        n: Number of samples to draw.

    Returns:
        Samples, shape ``[n, n_samples]``.
    """
    return self._dist.rvs(size=(n, len(self)))

cdf 

cdf(y: NDArray[floating]) -> NDArray[floating]

Cumulative distribution function.

PARAMETER DESCRIPTION
y

Values at which to evaluate the CDF.

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

CDF values, same shape as y.
Source code in ngboost_lightning/distributions/halfnormal.py 
139
140
141
142
143
144
145
146
147
148
def cdf(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Cumulative distribution function.

    Args:
        y: Values at which to evaluate the CDF.

    Returns:
        CDF values, same shape as ``y``.
    """
    return self._dist.cdf(y)

ppf 

ppf(q: NDArray[floating]) -> NDArray[floating]

Percent point function (inverse CDF / quantile function).

PARAMETER DESCRIPTION
q

Quantiles, values in [0, 1].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Values at the given quantiles, same shape as q.
Source code in ngboost_lightning/distributions/halfnormal.py 
150
151
152
153
154
155
156
157
158
159
def ppf(self, q: NDArray[np.floating]) -> NDArray[np.floating]:
    """Percent point function (inverse CDF / quantile function).

    Args:
        q: Quantiles, values in [0, 1].

    Returns:
        Values at the given quantiles, same shape as ``q``.
    """
    return self._dist.ppf(q)

logpdf 

logpdf(y: NDArray[floating]) -> NDArray[floating]

Log probability density function.

PARAMETER DESCRIPTION
y

Values at which to evaluate.

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Log-density values, same shape as y.
Source code in ngboost_lightning/distributions/halfnormal.py 
161
162
163
164
165
166
167
168
169
170
def logpdf(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Log probability density function.

    Args:
        y: Values at which to evaluate.

    Returns:
        Log-density values, same shape as ``y``.
    """
    return self._dist.logpdf(y)