Skip to content

Layers

This page documents the core layer building blocks in NWAVE, split into:

  • Hardware layers (Abstractions of the H1 neuronal model)
  • LIF layers (generic software LIF layers, with optional GPU path via comPaSSo)

The goal is to make it clear how to use each layer, what it returns, and how to run simulations that respect hardware non-idealities (mismatch, quantization).

Conventions (inputs/outputs)

NWAVE networks are typically simulated one timestep at a time:

  • Synapses map spikes → charge/current
    spk[t] → cur[t]
  • Neuron layers map charge/current → spikes (+ membrane)
    cur[t] → (spk[t], mem[t])

Shapes

At a single timestep:

  • Spike tensor: [B, N]
  • Charge/current tensor: [B, N]
  • Weight matrices are divided in:
    • Feedforward synapse: [N_in, N_out]
    • Recurrent: [N, N]
  • Time series of the NWaveDataloaders (input of the network) are usually [B, T, N] and you iterate over t.

Note

Layers accept 1D inputs of shape [N] and internally convert them to [1, N]. But it is a general practice to pass [B, N] consistently.

Preparing layers each batch: prepare_net

Many layers build internal quantities (e.g., effective synapse matrix, quantized weights, sampled mismatch noise) that must be refreshed before simulation.

Call prepare_net(model) once per batch, before you loop over timesteps.

from nwavesdk.layers import prepare_net

prepare_net(model)
for t in range(T):
    ...

For every submodule of the network it resets the states, and optionally applies mismatch effects, charge conversions and quantization. Even if you don't want to simulate non-idealities, you generally still want to call prepare_net before running a sequence, to avoid stale buffers/states.

Warning

If you forget to call prepare_net, you will likely see runtime errors such as missing tensors, NaNs, or you may simulate with stale synaptic/mismatch values.

Hardware Layers

Frontend

Module: Frontend(nb_inputs, quantization_bits=None, stddev=None, init=..., lif_threshold=...)

Hardware-oriented abstraction of the filterbank frontend. It is implemented as a diagonal synapse: each input channel is scaled by its own trainable parameter.

On the H1 Neuronova chip, the frontend is 16 filters → 16 neurons (one-to-one).

Import with:

from nwavesdk.layers import Frontend

Parameters

  • nb_inputs (int): number of input channels / neurons.
    If nb_inputs != 16, a warning is emitted (the layer will run in software, but is not chip-faithful). However, smaller channels can be mapped on chip by zeroing weights of non supported channels.
  • quantization_bits (int | None): enables quantization of the weights.
  • stddev (float | None): enables empirical mismatch on the weight matrix.
  • init (callable / InitSpec): initialization used for the diagonal trainable weights (see Weight initialization).
  • lif_threshold (float | int | Tensor): reference threshold used by the hardware initializer scaling.

Frontend stores a learnable diagonal weight vector weight of shape [N] and builds a respective charge matrix.

  • With quantization enabled, w is replaced by a quantized version after calling prepare_net.
  • With mismatch enabled, noise is added to the matrix.

Input / Output

Input: input_spikes of shape [B, nb_inputs]
Output: tensor of shape [B, nb_inputs]

Typical usage

Charge matrix must be built before running a simulation (via prepare_net(model)).

from nwavesdk.layers import Frontend, prepare_net

syn = Frontend(nb_inputs=16, quantization_bits=6, stddev=0.02, 
init=lambda w: nn.init.normal_(w, 0.1, 0.01))

# before iterating over timesteps:
prepare_net(syn)          # or prepare_net(model) if syn is inside a model
y = syn(x_t)              # x_t: [B, 16]

Warning

BUG ALERT: In first alpha release v0.0.1 there is a known issue: if the layer is built with default params for init will raise a "ValueError: Provided init spec is not a valid torch initializer call". This is due to the fact that default init (xavier) expects a multiple dimension tensor for weights, but Frontend has a single dimension tensor. Therefore, when using Frontend layer, always use only either normal init or uniform. For other custom initializations of weight access to the weight as a layer's param and assign custom initialization. This bug will be fixed in the next sdk release.

Non-idealities (per-layer)

Quantization

Enable reduced precision by setting quantization_bits.

  • During training/inference layers will use quantized weights.
  • To permanently snap the stored parameters to the quantized representation (e.g., for export), call:
syn.get_quant_weights()
Synaptic mismatch

Enable mismatch by setting stddev.

  • If you call prepare_net() once per batch, mismatch is resampled each time (stochastic robustness), effectively simulating as many chips as many batches in the dataloader.

Errors / common gotchas

  • Forgetting to build charge matrix: If you call Frontend without prepare_net(model), you may get a runtime failure. Fix: call prepare_net(model) before running timesteps.

HWSynapse

Module: HWSynapse(nb_inputs, nb_outputs, quantization_bit=None, stddev=None, init=..., lif_threshold=...)

A dense hardware-compatible synapse. Internally it builds an effective synapse for charge transfers with optional mismatch noise.

Import with:

from nwavesdk.layers import HWSynapse

Parameters

  • nb_inputs, nb_outputs (int): matrix size.
  • init (callable / InitSpec): torch-style initializer applied then mapped to hardware weight meaning (see Weight initialization).
  • lif_threshold (float | int | Tensor): reference LIF threshold used only to scale the hardware init.
  • quantization_bit (int | None): if set, W is quantized.
  • stddev (float | None): if set, empirical mismatch is sampled with that standard deviation.

Input / Output

Input: input_spikes of shape [B, nb_inputs]
Output: charge/current of shape [B, nb_outputs]

Non-idealities (per-layer)

  • Synaptic mismatch: set stddev.
    Mismatch is sampled from the empirical distribution of noise and added to the effective synapse.
  • Quantization-aware training: set quantization_bit.
    Effective synapse is built using the quantized weights.
syn = HWSynapse(64, 64, quantization_bit=6, stddev=0.02)

Tip

You can mix settings across layers freely: e.g., mismatch on early layers only (often useful since early layers carry finer-grain information), quantization on all layers, etc.

Common error / gotcha

  • Forgetting prepare_net call → charge matrix may not exist yet.

HWLayer

Module: HWLayer(n_neurons, taus, dt, layer_topology="FF" | "RC", detach_reset = False, ...)

Hardware neuron layer (high-level model of Neuronova neurons). It simulates:

  • membrane integration with a fixed threshold (chip parameter)
  • non-negative membrane (negative voltages are clipped)
  • optional leak (tau) mismatch
  • optional recurrent dynamics (within-layer all-to-all) when layer_topology="RC"

Import with:

from nwavesdk.layers import HWLayer

Parameters (core)

  • n_neurons (int): number of neurons.
  • taus (float | Tensor): time constant(s). Can be scalar or per-neuron.
  • dt (float): timestep in seconds.
  • layer_topology (str):
  • "FF": feedforward-only neuron dynamics
  • "RC": includes a recurrent synapse on spikes of the previous timestep
  • detach_reset (bool): if set to true the reset contribution on gradient is not considered. By default it consider that in the training process.

Output behavior

At each timestep the layer returns:

  • spk: spikes [B, n_neurons]
  • mem: membrane [B, n_neurons]
spk, mem = hw(cur)

Input / Output

Input: input_charge of shape [B, n_neurons] (or [n_neurons], promoted to [1, n_neurons], not recommended)
Outputs: (spk, mem) each of shape [B, n_neurons]

Recurrent mode (layer_topology="RC")

When "RC" is enabled:

  • A trainable recurrent matrix of shape [n_neurons, n_neurons] is created.
  • It is converted to an effective recurrent synapse (with optional quantization/mismatch) through the prepare_net call.
  • The recurrent matrix supports init=... and lif_threshold=... exactly like HWSynapse (see Weight initialization).

Note

Recurrent connections are all-to-all within the layer and do not implement routing constraints by themselves. Either use the hardware-aware losses/metrics (e.g., topology loss/coherence) to train a mappable configuration or initialize weight matrix by accessing recurrent_weights from the layer.

Non-idealities (per-layer)

  • Leak (tau) mismatch: set ileak_mismatch=True.
    Mismatch is sampled from the empirical leak distribution.
  • Recurrent synaptic mismatch: set stddev=<float> only in RC mode.
  • Recurrent quantization: set quantization_bit=<int> only in RC mode.
  • Surrogate gradient: set spike_grad=<surrogate module>. Default is fast_sigmoid() with slope = 25 as snntorch's default.
hw = HWLayer(
    n_neurons=64,
    taus=20e-3,
    dt=1e-3,
    layer_topology="RC",
    ileak_mismatch=True,
    stddev=0.02,
    quantization_bit=6,
)

Errors and inconsistencies to expect

  • If layer_topology="FF" and you pass any of the RC-only arguments, the constructor raises KeyError:
  • quantization_bit (for recurrent weights)
  • stddev (for recurrent mismatch)
  • custom init / lif_threshold (used only for recurrent initialization)
  • If layer_topology is not "FF" or "RC" → KeyError.
  • If you call forward() without prepare_net() in RC mode, the effective recurrent synapse may be missing. Always use prepare_net().

Weight initialization (LIF vs Hardware)

NWAVE exposes a common init interface across layers, built around PyTorch initializers (e.g., torch.nn.init.xavier_uniform_).

How to pass a custom initializer

Every layer that accepts init=... supports one of these forms:

  • Callable: init(tensor)
    Example: init=torch.nn.init.kaiming_uniform_
  • (callable, kwargs): (init_fn, {"a": ..., "mode": ...})
  • (callable, args, kwargs): (init_fn, (arg1, arg2, ...), {"kw": ...})

This mirrors how most torch.nn.init.* functions are called. 

import torch.nn as nn

# simple callable
init = nn.init.xavier_uniform_

# callable + kwargs
init = (nn.init.kaiming_uniform_, {"a": 0.1, "mode": "fan_in", "nonlinearity": "relu"})

LIF weights: standard meaning

For LIF layers/synapses, weights have the usual “software” meaning (they scale currents/charges in the update).
So your init behaves as you expect from PyTorch: it directly sets the scale of the synaptic effect and can easily make a neuron cross threshold in one step depending on your input data, dt, taus, and thresholds.

Hardware weights: membrane-jump mapping

For hardware synapses/layers, the weight parameter does not represent an arbitrary “current gain”.

On H1, weights are mapped through a hardware transfer function into an effective charge/voltage jump on the membrane. This implies:

  • A hardware weight is conceptually a target membrane jump, not a free gain.
  • The hardware model cannot arbitrarily increase membrane jump the way a software LIF weight can. A single step that would easily exceed threshold in LIF is physically unrealistic in HW due to saturation effects.

To make initialization user-friendly, hardware synapses use a LIF-equivalent target during init: the initializer produces a “software-like” weight sample, then NWAVE rescales it so that (roughly) a spike would cause a membrane jump proportional to the lif threshold.

lif_threshold (hardware init only)

lif_threshold is only used for hardware weight initialization: it is the reference LIF threshold you want the initializer to assume when setting the initial synaptic strength.

  • Larger lif_threshold → smaller intended membrane jump (more conservative init)
  • Smaller lif_threshold → larger intended membrane jump (more aggressive init)

This does not change the hardware neuron threshold (that is a chip parameter); it only changes how the trainable weights are initialized.

from nwavesdk.layers import HWSynapse
import torch.nn as nn

syn = HWSynapse(
    64, 64,
    init=nn.init.xavier_uniform_,
    lif_threshold=1.0,   # reference LIF threshold used ONLY for init scaling
)

Tip

If you are porting a working LIF model to HW simulation, set lif_threshold close to your LIF layer threshold(s) to start from a comparable effective synaptic strength (then fine-tune with HW-aware losses / mismatch / quantization).

LIF Layers

LIFSynapse

Module: LIFSynapse(nb_inputs, nb_outputs, device="cpu"|"gpu", use_bias=False, quantization_bit=None, init=...)

A generic (non-hardware) synapse used with LIFLayer.

Import with:

from nwavesdk.layers import LIFSynapse

Parameters

  • nb_inputs, nb_outputs (int)
  • device (str):
  • "cpu" uses a standard synapse
  • "gpu" uses batched matmul
  • use_bias / bias_learn: optional bias term
  • quantization_bit (int | None): quantization bit applied
  • init (callable / InitSpec): torch-style initializer applied directly to the synaptic weight matrix (see Weight initialization).

Input / Output

CPU (serial):

  • Input: [B, nb_inputs]
  • Output: [B, nb_outputs]

GPU path:

  • Designed to support batched matmul (@). Typically used with time-series tensors:
  • Input: [B, T, nb_inputs] (or [B, nb_inputs])
  • Output: [B, T, nb_outputs] (or [B, nb_outputs])

Warning

Like other synapses, LIFSynapse expects you to run prepare_net() first if you enabled quantization.

LIFLayer

Module: LIFLayer(n_neurons, taus, thresholds, reset_mechanism, dt, device="cpu"|"gpu", detach_reset = False, ...)

A standard LIF neuron layer supporting:

  • configurable thresholds (optionally learnable)
  • configurable taus (optionally learnable)
  • optional recurrent connections on CPU (layer_topology="RC")
  • an experimental GPU parallel path (comPaSSo) optimized for long sequences

Import with:

from nwavesdk.layers import LIFLayer

Input / Output

CPU (serial timestep simulation):

  • Input (per timestep): x of shape [B, n_neurons]
  • Output: (spk, mem) each [B, n_neurons]

GPU (comPaSSo):

  • Input (whole sequence): x of shape [B, T, n_neurons]
  • Output: (spk_seq, mem_seq) each [B, T, n_neurons]

Reset mechanisms

  • "zero": reset membrane to zero on spike
  • "subtraction": subtract threshold on spike
  • "none": no reset

Recurrent mode (CPU only)

If layer_topology="RC" (CPU):

  • recurrent_weights of shape [n_neurons, n_neurons] is created
  • init=... controls the initialization of recurrent_weights (torch-style)

If device="gpu" and layer_topology="RC" → NotImplementedError (recurrent not supported on comPaSSo).

About comPaSSo (GPU) - experimental

When device="gpu":

  • The layer computes the same LIF update in a GPU-parallel way.
  • Only feedforward connections are supported (no recurrent).
  • Reset mechanism is effectively constrained to subtraction (other choices are overridden).
  • It is optimized for long sequences where serial timestep loops are slow.

Warning

comPaSSo is experimental. Small numerical differences vs. CPU can occur due to floating-point operation ordering and GPU kernel behavior.

Surrogates on GPU

In the comPaSSo path, surrogate behavior is constrained:

  • Custom surrogate modules are bypassed
  • A FastSigmoid-like surrogate with fixed slope = 25 is used internally for numerical stability

So, setting spike_grad=... has effect on CPU serial, but not on GPU comPaSSo.

Example: hardware network definition (minimal)

import torch
import torch.nn as nn
from nwavesdk.layers import Frontend, HWSynapse, HWLayer, prepare_net

class HWSNN(nn.Module):
    def __init__(self, num_classes=2, dt=1e-3):
        super().__init__()
        taus = 10e-3

        self.syn1 = Frontend(nb_inputs=16)          # chip-faithful frontend
        self.hw1  = HWLayer(n_neurons=16, taus=taus, dt=dt)

        self.syn2 = HWSynapse(16, num_classes)      # dense synapse
        self.hw2  = HWLayer(n_neurons=num_classes, taus=taus, dt=dt)

    def forward(self, x):
        # x: [B, T, 16]
        B, T, _ = x.shape
        prepare_net(self)  # IMPORTANT: call once per batch/sequence

        spk2_trace = []
        for t in range(T):
            cur1 = self.syn1(x[:, t, :])
            spk1, _ = self.hw1(cur1)

            cur2 = self.syn2(spk1)
            spk2, _ = self.hw2(cur2)

            spk2_trace.append(spk2)

        return torch.stack(spk2_trace, dim=1)  # [B, T, num_classes]

Practical training knobs (mismatch & quantization)

You can enable or disable non-idealities per layer, without changing the rest of the model:

Quantization-aware training (QAT)

  • Frontend(quantization_bits=...)
  • HWSynapse(quantization_bit=...)
  • HWLayer(..., layer_topology="RC", quantization_bit=...) (recurrent only)
  • LIFSynapse(quantization_bit=...)

For deployment, you can optionally snap weights:

  • Frontend.get_quant_weights()
  • HWSynapse.get_quant_weights()
  • LIFSynapse.get_quant_weights()

Mismatch-aware training

  • HWSynapse(stddev=...): synaptic mismatch
  • HWLayer(ileak_mismatch=True): leak mismatch
  • HWLayer(..., layer_topology="RC", stddev=...): recurrent synaptic mismatch
  • Frontend(stddev=...): synaptic mismatch

Mismatch is resampled by prepare_net(), which makes training robust to variability by training the net to be independent from empirical noises.

Tip

If you want deterministic evaluation, set stddev=None / disable mismatch. If you want stochastic robustness, keep prepare_net() in the training loop.