Automatic Initialization

The nwave SDK provides two dataset-driven initializers that work together to give hardware-aware SNNs a strong starting point before training begins:

Initializer	What it sets	When to use
`frontend_firing_init`	Frontend diagonal weights	Any model with a frontend stage
`fluct_init`	Dense Synapse weights	All `(Synapse, Layer)` pairs in the model

Both initializers are automatic — they read a few batches from your DataLoader and calibrate weights to the actual data distribution. No manual statistics required.

Recommended order

Always apply frontend_firing_init before fluct_init when your model has a frontend stage. fluct_init propagates firing rates layer-to-layer, and that propagation depends on the frontend output rate being correct first.

```python from nwavesdk.init import frontend_firing_init, fluct_init

frontend_firing_init(model, dataloader, target_fr=0.20, n_batches=8) fluct_init(model, dataloader, xi_target=3.0, alpha=1.0, n_batches=4) ```

Both initializers are quantization-aware: they work correctly when layers are built with quantization_bit set. The binary search in frontend_firing_init runs in continuous weight space and restores quantization after convergence.

Architecture constraint: No non-hardware modules (Dropout, BatchNorm, etc.) may sit before or between hardware stages at any level of the module tree. For parallel or non-sequential architectures, set model.layer_pairs (and model.frontend_stage) explicitly — see Layer pair discovery below.

`frontend_firing_init`

frontend_firing_init sets each frontend neuron's diagonal weight so that the downstream spiking layer reaches a chosen firing rate on real input data. It uses a per-neuron binary search driven by a DataLoader.

It is the natural complement to fluct_init, which skips the frontend stage entirely because of its diagonal (1-to-1) connectivity — weight variance cannot be meaningfully tuned with a single input per neuron.

Supported hardware: H1v1 and H1v2 frontends (H1v1Frontend / H1v2Frontend).

Import

from nwavesdk.init import frontend_firing_init

Quick start

H1v1 frontend-first network

import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset

from nwavesdk.layers import H1v1Frontend, H1v1Synapse, H1v1Layer, prepare_net
from nwavesdk.surrogate import fast_sigmoid
from nwavesdk.init import frontend_firing_init


class FrontendH1Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.frontend      = H1v1Frontend(nb_inputs=13, device="cpu")
        self.frontend_lyr  = H1v1Layer(
            n_neurons=13, taus=32e-3, dt=8e-3,
            layer_topology="FF", spike_grad=fast_sigmoid(), device="cpu",
        )
        self.synapse       = H1v1Synapse(nb_inputs=13, nb_outputs=64, device="cpu")
        self.layer         = H1v1Layer(
            n_neurons=64, taus=64e-3, dt=8e-3,
            layer_topology="FF", spike_grad=fast_sigmoid(), device="cpu",
        )
        # Expose frontend stage explicitly for reliable detection
        self.frontend_stage = (self.frontend, self.frontend_lyr)

    def forward(self, x):
        prepare_net(self, collect_metrics=False)
        spk_fe, _   = self.frontend_lyr(self.frontend(x))
        cur         = self.synapse(spk_fe)
        spk, mem    = self.layer(cur)
        return spk, mem


x        = (torch.rand(64, 125, 13) < 0.3).float()
train_dl = DataLoader(TensorDataset(x), batch_size=16, shuffle=True)
model    = FrontendH1Net()

frontend_firing_init(model, train_dl, target_fr=0.20, n_batches=8, verbose=True)

H1v2 frontend-first network

from nwavesdk.layers import H1v2Frontend, H1v2Synapse, H1v2Layer, prepare_net
from nwavesdk.init import frontend_firing_init


class FrontendH2Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.frontend      = H1v2Frontend(nb_inputs=13, device="cpu")
        self.frontend_lyr  = H1v2Layer(
            n_neurons=13, taus=20e-3, dt=1e-3,
            layer_topology="FF", spike_grad=fast_sigmoid(), device="cpu",
        )
        self.synapse       = H1v2Synapse(nb_inputs=13, nb_outputs=32, device="cpu")
        self.layer         = H1v2Layer(
            n_neurons=32, taus=20e-3, dt=1e-3,
            layer_topology="FF", spike_grad=fast_sigmoid(), device="cpu",
        )
        self.frontend_stage = (self.frontend, self.frontend_lyr)

    def forward(self, x):
        prepare_net(self, collect_metrics=False)
        spk_fe, _   = self.frontend_lyr(self.frontend(x))
        cur         = self.synapse(spk_fe)
        spk, mem    = self.layer(cur)
        return spk, mem


frontend_firing_init(model, train_dl, target_fr=0.20, n_batches=8, verbose=True)

frontend_firing_init modifies the model in-place and returns None.

Parameters

Parameter	Type	Default	Description
`model`	`nn.Module`	—	Network containing exactly one `(H1v1/H1v2Frontend, H1v1/H1v2Layer)` frontend stage. Only the frontend weights are modified.
`dataloader`	`DataLoader` or iterable	—	Input batches `(B, T, N)` or tuples whose first element is that tensor. A few batches are pre-collected and reused for every binary-search step.
`target_fr`	`float`	`0.20`	Desired mean firing rate per frontend neuron as a fraction of timesteps (e.g. `0.20` = 20 %). Must be in (0, 1).
`n_batches`	`int`	`1`	Number of batches consumed per firing-rate measurement. Raise to 4–8 for noisy or short datasets.
`epsilon`	`float`	`0.02`	Convergence tolerance. A `UserWarning` is raised for neurons whose final rate deviates by more than this value.
`verbose`	`bool`	`True`	Print per-neuron weight and firing-rate summary after convergence.

How the binary search works

Runs 16 iterations of bisection independently for each frontend neuron in parallel:

Weight of neuron i set to midpoint between its current lower and upper bound.
Frontend + frontend layer run over pre-collected batches; mean firing rate measured per neuron.
If neuron i fires below target_fr, lower bound raised; if at or above, upper bound lowered.

Quantization is disabled during the search so bisection converges smoothly. If the model had quantization enabled, a post-convergence measurement is taken with quantization restored.

Verbose output

[frontend_firing_init] target_fr=0.20  ε=0.02  n_batches=8  [H1v1]
  Neuron  0 | w=0.5273  fr_cont=0.201  fr_quant=0.198  OK
  Neuron  1 | w=0.4961  fr_cont=0.199  fr_quant=0.200  OK
  ...
  Neuron 12 | w=0.5234  fr_cont=0.201  fr_quant=0.197  OK
[frontend_firing_init] done.

Column	Meaning
`w`	Final diagonal frontend weight for this neuron.
`fr_cont`	Firing rate in continuous weight space (quantization off).
`fr_quant`	Firing rate after quantization restored. Printed only when quantization is enabled.
`OK` / `WARN`	Whether the neuron converged within `epsilon` of `target_fr`.

Warnings

Neuron cannot reach target firing rate

If a neuron cannot reach target_fr even at the maximum hardware weight (w_max), a UserWarning is raised. Mitigations: lower target_fr, increase taus, or feed richer input data.

Convergence tolerance not met

After 16 bisection steps, if a neuron's rate still deviates by more than epsilon, a UserWarning is raised. Increasing n_batches usually resolves this.

Hardware weight bounds

Weights are checked against hardware limits after convergence — H1v1: [−0.9, 0.9] · H1v2: [−1.66, 1.66]. Out-of-range weights emit a UserWarning and will be flagged by is_net_deployable().

Quantization and firing rate

The post-quantization rate (fr_quant) may differ from the continuous-space target. This is expected and no further adjustment is made.

Supported architectures

Feature	Supported
H1v1 (H1v1Frontend + H1v1Layer)	✓
H1v2 (H1v2Frontend + H1v2Layer)	✓
Quantized frontends	✓ — search runs in continuous space; quantization re-applied after
RC frontend layer	✓ — recurrent path suppressed during search; restored after
Multi-layer models (dense layers after frontend)	✓ — only the frontend stage is modified
Frontend-only models (no dense pairs)	✓
Models without a frontend stage	✗ — raises `ValueError`
Mixed H1v1/H1v2 in one model	✗ — raises `ValueError`

Frontend stage discovery

Explicit frontend_stage attribute (recommended): python self.frontend_stage = (self.frontend, self.frontend_lyr)
Auto-discovery: walks model.named_modules() in registration order and picks the first (H1v1Frontend|H1v2Frontend, H1v1Layer|H1v2Layer) consecutive pair.

`fluct_init`

fluct_init initialises hardware layer weights to place the sub-threshold membrane potential near threshold at initialization, maximising surrogate-gradient signal from the first optimization step.

\[\xi = \frac{\theta - \mu_U}{\sigma_U}\]

Reference: Rossbroich, Gygax & Zenke (2022), Fluctuation-driven initialization for spiking neural network training

Supported hardware: H1v1 and H1v2. All layers in a model must belong to the same family.

Relation to Rossbroich et al.

NWAVE's implementation is derived from Rossbroich et al., not a verbatim port:

The PSP kernel ε(t) is measured numerically from the actual HW layer dynamics, rather than assumed to be exponential. For H1v1 the nonlinear synapse makes a small-probe measurement exact; for H1v2 the linear synapse is exact by design.
NWAVE adds an adaptive binary search on the feed-forward mean weight µ_W to ensure no dead neurons at initialization. This step is not in the original paper.

H1v1/H1v2 Differences

H1v2 weights are weaker than H1v1, meaning more charge is needed to drive the membrane. Initializations that worked on H1v1 may produce out-of-bounds weights for H1v2. Use higher xi_target values and ensure each layer has sufficient input neurons.

Import

from nwavesdk.init import fluct_init

Quick start

H1v1 network (FF)

import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset

from nwavesdk.layers import H1v1Synapse, H1v1Layer, prepare_net
from nwavesdk.surrogate import fast_sigmoid
from nwavesdk.init import fluct_init


class TinyH1Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.synapse = H1v1Synapse(nb_inputs=16, nb_outputs=32, device="cpu")
        self.layer   = H1v1Layer(
            n_neurons=32, taus=10e-3, dt=1e-3,
            layer_topology="FF", spike_grad=fast_sigmoid(slope=5.0), device="cpu",
        )

    def forward(self, x):
        prepare_net(self, collect_metrics=False)
        cur      = self.synapse(x)
        spk, mem = self.layer(cur)
        return spk, mem


x        = (torch.rand(64, 100, 16) < 0.3).float()
train_dl = DataLoader(TensorDataset(x), batch_size=16, shuffle=True)
model    = TinyH1Net()

fluct_init(model, train_dl, xi_target=1.0, alpha=1.0, n_batches=4, verbose=True)

H1v2 network (RC)

from nwavesdk.layers import H1v2Synapse, H1v2Layer, prepare_net
from nwavesdk.init import fluct_init


class TinyH1v2Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.syn = H1v2Synapse(nb_inputs=8, nb_outputs=16, device="cpu")
        self.lyr = H1v2Layer(
            n_neurons=16, taus=10e-3, dt=1e-3,
            layer_topology="RC", spike_grad=fast_sigmoid(slope=5.0), device="cpu",
        )
        self.layer_pairs = [(self.syn, self.lyr)]   # explicit — recommended for RC nets

    def forward(self, x):
        prepare_net(self, collect_metrics=False)
        cur      = self.syn(x)
        spk, mem = self.lyr(cur)
        return spk, mem


fluct_init(model, train_dl, xi_target=1.0, alpha=0.85, n_batches=4, verbose=True)

fluct_init modifies the model in-place and returns None.

Parameters

Parameter	Type	Default	Description
`model`	`nn.Module`	—	Initialized in-place. Must contain at least one `(H1v1/H1v2 Synapse, H1v1/H1v2 Layer)` dense pair. All pairs must belong to the same hardware family.
`dataloader`	`DataLoader` or iterable	—	Input batches `(B, T, n_F)` or tuples whose first element is that tensor. Used to estimate firing-rate moments and check for dead neurons.
`xi_target`	`float`	`2.0`	Target ξ = (θ − µ_U) / σ_U. Recommended range: 1–3. Lower = more fluctuation-driven.
`alpha`	`float`	`1`	Fraction of σ_U² budget for FF weights; remaining (1 − alpha) goes to RC weights. Set to `1.0` for FF-only networks.
`n_batches`	`int`	`1`	Batches per firing-rate estimate. Raise to 4–8 for noisy datasets.
`verbose`	`bool`	`True`	Print per-layer init summary.

Verbose output

[fluct_init] ξ=1.0  α=0.85  dt=1.0ms  (stacked, adaptive µ)  [H1v1]
  Input → ν_mean=392.0Hz  ν_var=392.0Hz  ratio=1.0x
  Layer 1 | ν_in=392.0Hz  µ_W=0.5158  σ_FF=0.0516  µ_U=0.104
[fluct_init] done.

For a frontend-first model:

[fluct_init] ξ=1.0  α=0.85  dt=1.0ms  (stacked, adaptive µ)  [H1v1]
  Frontend → ν_out=145.2Hz  (used as ν_in for layer 1)
  Layer 1 | ν_in=145.2Hz  µ_W=0.3841  σ_FF=0.0384  σ_RC=0.0572  µ_U=0.097
[fluct_init] done.

Field	Meaning
`ν_mean`	Mean input firing rate estimated from data [Hz].
`ν_var`	Second moment for variance-budget calculation. Equals ν_mean for binary spikes.
`ν_in`	Input firing rate for this layer (DataLoader input or previous layer output).
`µ_W`	Mean FF weight after adaptive search + 10 % safety margin.
`σ_FF`	Std of FF weights. If the FF budget is exhausted the floor `0.1 · µ_W` is applied.
`σ_RC`	Std of RC weights (zero-mean). Printed only for recurrent layers.
`µ_U`	Resulting mean membrane potential = n_F · µ_W · ν_in · ε̄.

Warnings

xi_target below 1.0

xi_target < 1.0 triggers a UserWarning. Very small ξ places the mean membrane extremely close to threshold, which can cause very high firing rates and unstable training. Recommended: 1 ≤ ξ ≤ 3.

Single-input layers — exhausted FF variance budget (n_F = 1)

Low fan-in layers may need a large µ_W to keep all neurons active, exhausting the FF variance budget. fluct_init detects this and applies a symmetry-breaking floor σ_FF = 0.1 · µ_W instead of leaving all FF weights identical. The warning message calls this init mean-driven and suggests using a smaller ξ.

Hardware weight range

After init, weights are checked against hardware limits: H1v1: [−0.9, 0.9] · H1v2: [−1.66, 1.66]. Out-of-range weights emit a UserWarning and will be flagged by is_net_deployable(). Add weight_magnitude_loss(model) to your training loss to softly enforce the constraint during training.

Dead neurons after init

If dead neurons remain after the adaptive search and symmetry-breaking floor, fluct_init emits a warning with the layer index and count. Mitigations: smaller xi_target, lower alpha, or more input neurons (increase n_F).

Supported architectures

Feature	Supported
H1v1 (H1v1Synapse + H1v1Layer)	✓
H1v2 (H1v2Synapse + H1v2Layer)	✓
Mixed H1v1/H1v2 in one model	✗ — raises ValueError
FF topology	✓
RC topology	✓
Heterogeneous τ per layer	✓
Analog inputs	✓
Binary spike inputs	✓
Multi-layer stacked init	✓ — output ν propagated layer-to-layer
Frontend-first models	✓ — frontend stage skipped; its output ν used as input for first dense pair
Frontend-only (no dense pairs after frontend)	✗ — raises ValueError
LIFSynapse / LIFLayer	✗ — use manual init for LIF networks

Layer pair discovery

fluct_init finds dense (Synapse, Layer) pairs in two ways:

Explicit layer_pairs attribute (recommended for RC nets and complex architectures): python self.layer_pairs = [(self.syn1, self.lyr1), (self.syn2, self.lyr2)]
Auto-discovery: walks the module tree with named_children() in registration order and collects consecutive (H1v1Synapse|H1v2Synapse, H1v1Layer|H1v2Layer) pairs at each level. Non-hw modules between hw siblings at the same level raise ValueError.

Automatic Initialization

frontend_firing_init

Import

Quick start

H1v1 frontend-first network

H1v2 frontend-first network

Parameters

How the binary search works

Verbose output

Warnings

Supported architectures

Frontend stage discovery

fluct_init

Relation to Rossbroich et al.

Import

Quick start

H1v1 network (FF)

H1v2 network (RC)

Parameters

Verbose output

Warnings

Supported architectures

Layer pair discovery

See Also

`frontend_firing_init`

`fluct_init`