multigeom.encoders.contract#
The encoder contract: what the pipeline requires of an encoder f: image -> responses, and a check that enforces it.
A user can bring any fitted PyTorch model, as long as it satisfies these rules.
`validate_encoder` runs a few forward and backward passes on random images and raises
`EncoderContractError` (naming the rule and what to change) at the first violation;
softer problems are returned as warnings.
R1 an `nn.Module`
R2 input [B, 1, H, W] float in [-1, 1] (the model does any rescaling itself)
R3 output [B, N] float, finite, the same N for every call and batch size
(and `n_neurons`, if present, == N)
R4 deterministic in eval mode (no dropout or noise at inference)
R5 differentiable w.r.t. the input through `forward` (no `torch.no_grad()`, no
`detach()`): needed for J(x)
R6 samples independent within a batch (no batch-statistics layers in train mode)
R7 does not modify its input in place
R8 not constant across images (warning)
R9 response scale comparable to the noise sigma: responses of order 1 (warning)
Which step needs what: training the diffusion model on the encoder's responses needs
R1-R4, R6 and R9; estimating G(x) and J(x) needs all of them (R5 for J(x)). Not
checkable here: the quality of the fit, how the responses were measured, and that the
images are 64x64 grayscale natural images.
Functions
|
validate_encoder once per encoder object and settings; warnings are issued with warnings.warn. |
|
Check encoder against the contract. |
Classes
|
What the pipeline asks of an encoder f: image -> responses (see the rules above). |
|
Result of validate_encoder: the number of neurons, the scale of the responses and any warnings. |
Exceptions
|
The encoder breaks a rule of the contract; rule is its id (R1-R9). |