Skip to content

exponential

exponential

Exponential distribution for ngboost-lightning.

Exponential 

Exponential(params: NDArray[floating])

Bases: Distribution

Exponential distribution with log-rate parameterization.

Internal parameter is [log_rate] where rate = exp(log_rate). The log-link for rate avoids constrained optimization during boosting.

Note on Fisher Information

For Exponential(log_rate), the Fisher Information is [[1]] (the identity) for every sample. This means the natural gradient equals the ordinary gradient.

ATTRIBUTE DESCRIPTION
n_params

Always 1 (log_rate).
rate

Rate values, shape [n_samples].

TYPE: NDArray[floating]

Construct Exponential from internal parameters.

PARAMETER DESCRIPTION
params

Internal parameters, shape [n_samples, 1]. Column 0 is log(rate).

TYPE: NDArray[floating]

Source code in ngboost_lightning/distributions/exponential.py 
30
31
32
33
34
35
36
37
38
39
40
def __init__(self, params: NDArray[np.floating]) -> None:
    """Construct Exponential from internal parameters.

    Args:
        params: Internal parameters, shape ``[n_samples, 1]``.
            Column 0 is log(rate).
    """
    self.log_rate: NDArray[np.floating] = params[:, 0]
    self.rate: NDArray[np.floating] = np.exp(self.log_rate)
    self._dist = sp_expon(scale=1.0 / self.rate)
    self._params = params

fit staticmethod 

fit(
    y: NDArray[floating],
    sample_weight: NDArray[floating] | None = None,
) -> NDArray[floating]

Estimate initial log_rate from target data.

PARAMETER DESCRIPTION
y

Target values, shape [n_samples].

TYPE: NDArray[floating]

sample_weight

Per-sample weights, shape [n_samples].

TYPE: NDArray[floating] | None DEFAULT: None

RETURNS DESCRIPTION
NDArray[floating]

Parameter vector [log(rate)], shape [1].
Source code in ngboost_lightning/distributions/exponential.py 
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
@staticmethod
def fit(
    y: NDArray[np.floating],
    sample_weight: NDArray[np.floating] | None = None,
) -> NDArray[np.floating]:
    """Estimate initial log_rate from target data.

    Args:
        y: Target values, shape ``[n_samples]``.
        sample_weight: Per-sample weights, shape ``[n_samples]``.

    Returns:
        Parameter vector ``[log(rate)]``, shape ``[1]``.
    """
    rate = 1.0 / max(float(np.average(y, weights=sample_weight)), 1e-10)
    return np.array([np.log(rate)])

score 

score(y: NDArray[floating]) -> NDArray[floating]

Per-sample negative log-likelihood.

PARAMETER DESCRIPTION
y

Observed target values, shape [n_samples].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

NLL values, shape [n_samples].
Source code in ngboost_lightning/distributions/exponential.py 
59
60
61
62
63
64
65
66
67
68
def score(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Per-sample negative log-likelihood.

    Args:
        y: Observed target values, shape ``[n_samples]``.

    Returns:
        NLL values, shape ``[n_samples]``.
    """
    return -self._dist.logpdf(y)

d_score 

d_score(y: NDArray[floating]) -> NDArray[floating]

Analytical gradient of NLL w.r.t. [log_rate].

Derivation

NLL = -log(rate) + rate * y = -log_rate + exp(log_rate) * y

d(NLL)/d(log_rate) = -1 + exp(log_rate) * y = -1 + rate * y = -(1 - rate * y)

PARAMETER DESCRIPTION
y

Observed target values, shape [n_samples].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Gradient array, shape [n_samples, 1].
Source code in ngboost_lightning/distributions/exponential.py 
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
def d_score(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Analytical gradient of NLL w.r.t. [log_rate].

    Derivation:
        NLL = -log(rate) + rate * y
            = -log_rate + exp(log_rate) * y

        d(NLL)/d(log_rate) = -1 + exp(log_rate) * y
                           = -1 + rate * y
                           = -(1 - rate * y)

    Args:
        y: Observed target values, shape ``[n_samples]``.

    Returns:
        Gradient array, shape ``[n_samples, 1]``.
    """
    n = len(y)
    grad = np.empty((n, 1))
    grad[:, 0] = -1.0 + self.rate * y
    return grad

metric 

metric() -> NDArray[floating]

Fisher Information: [[1]] for each sample.

RETURNS DESCRIPTION
NDArray[floating]

FI tensor, shape [n_samples, 1, 1].
Source code in ngboost_lightning/distributions/exponential.py 
 92
 93
 94
 95
 96
 97
 98
 99
100
def metric(self) -> NDArray[np.floating]:
    """Fisher Information: ``[[1]]`` for each sample.

    Returns:
        FI tensor, shape ``[n_samples, 1, 1]``.
    """
    n = len(self.rate)
    fi = np.ones((n, 1, 1))
    return fi

natural_gradient 

natural_gradient(y: NDArray[floating]) -> NDArray[floating]

Natural gradient (fast path since FI is identity).

Since the Fisher Information is the identity matrix, the natural gradient equals the ordinary gradient.

PARAMETER DESCRIPTION
y

Observed target values, shape [n_samples].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Natural gradient, shape [n_samples, 1].
Source code in ngboost_lightning/distributions/exponential.py 
102
103
104
105
106
107
108
109
110
111
112
113
114
def natural_gradient(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Natural gradient (fast path since FI is identity).

    Since the Fisher Information is the identity matrix, the natural
    gradient equals the ordinary gradient.

    Args:
        y: Observed target values, shape ``[n_samples]``.

    Returns:
        Natural gradient, shape ``[n_samples, 1]``.
    """
    return self.d_score(y)

mean 

mean() -> NDArray[floating]

Conditional mean (point prediction).

RETURNS DESCRIPTION
NDArray[floating]

Mean values 1/rate, shape [n_samples].
Source code in ngboost_lightning/distributions/exponential.py 
116
117
118
119
120
121
122
def mean(self) -> NDArray[np.floating]:
    """Conditional mean (point prediction).

    Returns:
        Mean values ``1/rate``, shape ``[n_samples]``.
    """
    return 1.0 / self.rate

sample 

sample(n: int) -> NDArray[floating]

Draw n samples per distribution instance.

PARAMETER DESCRIPTION
n

Number of samples to draw.

TYPE: int

RETURNS DESCRIPTION
NDArray[floating]

Samples, shape [n, n_samples].
Source code in ngboost_lightning/distributions/exponential.py 
124
125
126
127
128
129
130
131
132
133
def sample(self, n: int) -> NDArray[np.floating]:
    """Draw n samples per distribution instance.

    Args:
        n: Number of samples to draw.

    Returns:
        Samples, shape ``[n, n_samples]``.
    """
    return self._dist.rvs(size=(n, len(self)))

cdf 

cdf(y: NDArray[floating]) -> NDArray[floating]

Cumulative distribution function.

PARAMETER DESCRIPTION
y

Values at which to evaluate the CDF.

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

CDF values, same shape as y.
Source code in ngboost_lightning/distributions/exponential.py 
135
136
137
138
139
140
141
142
143
144
def cdf(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Cumulative distribution function.

    Args:
        y: Values at which to evaluate the CDF.

    Returns:
        CDF values, same shape as ``y``.
    """
    return self._dist.cdf(y)

ppf 

ppf(q: NDArray[floating]) -> NDArray[floating]

Percent point function (inverse CDF / quantile function).

PARAMETER DESCRIPTION
q

Quantiles, values in [0, 1].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Values at the given quantiles, same shape as q.
Source code in ngboost_lightning/distributions/exponential.py 
146
147
148
149
150
151
152
153
154
155
def ppf(self, q: NDArray[np.floating]) -> NDArray[np.floating]:
    """Percent point function (inverse CDF / quantile function).

    Args:
        q: Quantiles, values in [0, 1].

    Returns:
        Values at the given quantiles, same shape as ``q``.
    """
    return self._dist.ppf(q)

logpdf 

logpdf(y: NDArray[floating]) -> NDArray[floating]

Log probability density function.

PARAMETER DESCRIPTION
y

Values at which to evaluate.

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Log-density values, same shape as y.
Source code in ngboost_lightning/distributions/exponential.py 
157
158
159
160
161
162
163
164
165
166
def logpdf(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Log probability density function.

    Args:
        y: Values at which to evaluate.

    Returns:
        Log-density values, same shape as ``y``.
    """
    return self._dist.logpdf(y)

logsf 

logsf(y: NDArray[floating]) -> NDArray[floating]

Log survival function: log(1 - CDF(y)).

For Exponential(rate), logsf(y) = -rate * y.

PARAMETER DESCRIPTION
y

Values at which to evaluate.

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Log-survival values, same shape as y.
Source code in ngboost_lightning/distributions/exponential.py 
168
169
170
171
172
173
174
175
176
177
178
179
def logsf(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Log survival function: log(1 - CDF(y)).

    For Exponential(rate), ``logsf(y) = -rate * y``.

    Args:
        y: Values at which to evaluate.

    Returns:
        Log-survival values, same shape as ``y``.
    """
    return self._dist.logsf(y)

crps_score 

crps_score(y: NDArray[floating]) -> NDArray[floating]

Per-sample CRPS for Exponential.

Closed form (uncensored): CRPS = y + 2*scale*exp(-y/scale) - 1.5*scale where scale = 1/rate.

Derivation

CRPS = int_0^y F(x)^2 dx + int_y^inf (1 - F(x))^2 dx Part 1 gives y + 2*s*exp(-y/s) - s/2*exp(-2y/s) - 1.5*s Part 2 gives s/2*exp(-2y/s) The exp(-2y/s) terms cancel.

PARAMETER DESCRIPTION
y

Observed target values, shape [n_samples].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

CRPS values, shape [n_samples].
Source code in ngboost_lightning/distributions/exponential.py 
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
def crps_score(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Per-sample CRPS for Exponential.

    Closed form (uncensored):
        ``CRPS = y + 2*scale*exp(-y/scale) - 1.5*scale``
    where ``scale = 1/rate``.

    Derivation:
        ``CRPS = int_0^y F(x)^2 dx + int_y^inf (1 - F(x))^2 dx``
        Part 1 gives ``y + 2*s*exp(-y/s) - s/2*exp(-2y/s) - 1.5*s``
        Part 2 gives ``s/2*exp(-2y/s)``
        The ``exp(-2y/s)`` terms cancel.

    Args:
        y: Observed target values, shape ``[n_samples]``.

    Returns:
        CRPS values, shape ``[n_samples]``.
    """
    scale = 1.0 / self.rate
    result: NDArray[np.floating] = (
        y + 2.0 * scale * np.exp(-y * self.rate) - 1.5 * scale
    )
    return result

crps_d_score 

crps_d_score(y: NDArray[floating]) -> NDArray[floating]

Gradient of CRPS w.r.t. [log_rate].

Derivation

Let s = 1/rate = scale. d(CRPS)/d(s) = 2exp(-y/s)(1 + y/s) - 1.5

Chain rule to log_rate: d(s)/d(log_rate) = -s d(CRPS)/d(log_rate) = d(CRPS)/d(s) * (-s)

PARAMETER DESCRIPTION
y

Observed target values, shape [n_samples].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Gradient array, shape [n_samples, 1].
Source code in ngboost_lightning/distributions/exponential.py 
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
def crps_d_score(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Gradient of CRPS w.r.t. [log_rate].

    Derivation:
        Let ``s = 1/rate = scale``.
            d(CRPS)/d(s) = 2*exp(-y/s)*(1 + y/s) - 1.5

        Chain rule to log_rate:
            d(s)/d(log_rate) = -s
            d(CRPS)/d(log_rate) = d(CRPS)/d(s) * (-s)

    Args:
        y: Observed target values, shape ``[n_samples]``.

    Returns:
        Gradient array, shape ``[n_samples, 1]``.
    """
    scale = 1.0 / self.rate
    exp_neg = np.exp(-y * self.rate)
    # d(CRPS)/d(scale)
    d_scale = 2.0 * exp_neg * (1.0 + y / scale) - 1.5
    # Chain to log_rate
    n = len(y)
    grad = np.empty((n, 1))
    grad[:, 0] = d_scale * (-scale)
    return grad

crps_metric 

crps_metric() -> NDArray[floating]

Riemannian metric for CRPS natural gradient.

The CRPS metric in the scale parameterization is the constant 29/108 (derived analytically). Applying the Jacobian for the log_rate parameterization ds/d(log_rate) = -s: M_log_rate = s^2 * 29/108 = (29/108) * scale^2

RETURNS DESCRIPTION
NDArray[floating]

Metric tensor, shape [n_samples, 1, 1].
Source code in ngboost_lightning/distributions/exponential.py 
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
def crps_metric(self) -> NDArray[np.floating]:
    """Riemannian metric for CRPS natural gradient.

    The CRPS metric in the scale parameterization is the constant
    ``29/108`` (derived analytically).  Applying the Jacobian for the
    log_rate parameterization ``ds/d(log_rate) = -s``:
        ``M_log_rate = s^2 * 29/108 = (29/108) * scale^2``

    Returns:
        Metric tensor, shape ``[n_samples, 1, 1]``.
    """
    n = len(self.rate)
    scale = 1.0 / self.rate
    met = np.empty((n, 1, 1))
    met[:, 0, 0] = (29.0 / 108.0) * scale**2
    return met

crps_natural_gradient 

crps_natural_gradient(
    y: NDArray[floating],
) -> NDArray[floating]

Natural gradient under CRPS metric (scalar fast path).

PARAMETER DESCRIPTION
y

Observed target values, shape [n_samples].

TYPE: NDArray[floating]

RETURNS DESCRIPTION
NDArray[floating]

Natural gradient, shape [n_samples, 1].
Source code in ngboost_lightning/distributions/exponential.py 
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
def crps_natural_gradient(self, y: NDArray[np.floating]) -> NDArray[np.floating]:
    """Natural gradient under CRPS metric (scalar fast path).

    Args:
        y: Observed target values, shape ``[n_samples]``.

    Returns:
        Natural gradient, shape ``[n_samples, 1]``.
    """
    grad = self.crps_d_score(y)
    scale = 1.0 / self.rate
    n = len(y)
    nat_grad = np.empty((n, 1))
    nat_grad[:, 0] = grad[:, 0] / ((29.0 / 108.0) * scale**2)
    return nat_grad