multigeom.encoders.reference#
Image-to-response encoders f_φ : x ↦ r̂ used throughout the paper.
Each encoder is a frozen feature-extracting backbone followed by a per-neuron
Gaussian spatial-attention readout (the same readout structure across all
three encoders, as discussed in Appendix C / D of the paper).
The canonical input is a [B, 1, 64, 64] tensor in [-1, 1] (matching the
diffusion-process input convention). Each encoder rescales / replicates
channels internally to its backbone's expected input format.
- InceptionEncoder: [-1, 1] → 3-channel → InceptionV3 → readout
(torchvision's Inception weights expect [-1, 1] input)
- ResNet50Encoder: [-1, 1] → 3-channel → ImageNet mean/std
standardisation → ResNet50 → readout
- OnOffEncoder: [-1, 1] → [0, 1] → 1-channel → plenoptic OnOff → readout
Trainable parameters live only in the readout. Backbone parameters are frozen
(requires_grad = False), so `forward` builds no graph for them unless the input
requires grad, in which case gradients with respect to the input image flow
(the encoder contract of `multigeom.encoders.contract`, rule R5).
Readout parameter counts (n_neurons = 50):
- Inception (C=192): 50 * (2 + 1 + 192 + 1) = 9,800
- ResNet50 (C=256): 50 * (2 + 1 + 256 + 1) = 13,000
- OnOff (C=2): 50 * (2 + 1 + 2 + 1) = 300
Functions
|
Construct an encoder from a config namespace. |
|
A reference encoder (fresh readout) from its description, {"reference": "inception", "n_neurons": 50, ...}. |
Classes
|
Encoder of the paper: frozen InceptionV3 backbone (truncated at layer) and a Gaussian readout. |
|
Encoder of the paper: frozen ON-OFF front-end (plenoptic) and a Gaussian readout. |
|
Encoder of the paper: frozen ResNet-50 backbone (truncated at layer) and a Gaussian readout. |