multigeom.fit.layer_sweep#

Truncation-layer sweep for the frozen-backbone encoders (paper App. “InceptionV3 model”):

    "We chose the truncation layer empirically, by sweeping every candidate
     Inception block against each area under a fixed readout architecture and
     training schedule. Conv2d_4a_3x3 maximised test correlation in both V1
     and V4; deeper blocks did not help V4 and hurt V1."

The same sweep is available for the ResNet-50 backbone (blocks conv1, maxpool,
layer1.0 ... layer4.2), whose chosen truncation is layer1.0.

For every candidate layer ℓ the encoder is

    r̂_n(x) = Σ_c w_{n,c} Σ_{h,w} a_n(h,w) φ^ℓ_c(x)_{h,w} + b_n     (Eq. (eq:readout))

with φ^ℓ the frozen backbone truncated after ℓ and a_n the normalised Gaussian
attention of `multigeom.encoders.GaussianReadout`. Everything except ℓ is held
fixed: the canonical [-1, 1] 64×64 grayscale input, the backbone's input
conversion (`multigeom.encoders._BaseEncoder._to_backbone_input`), the readout, and
the Stage-1 recipe read from the encoder config (AdamW, cosine schedule, MSE,
batch size, gradient clipping at 1.0, 5% image-level validation split with a
fixed seed, model selection on mean validation correlation). The reported
score is the mean Pearson correlation over the n target neurons on the Papale
test set (and its noise-corrected version r_n/ρ_n).

Truncating at an arbitrary layer. `multigeom.encoders.InceptionBackbone` keeps the
torchvision children up to the named layer, which is correct up to Mixed_6e
but would include the auxiliary classifier (`AuxLogits`) for deeper layers,
and `multigeom.encoders.ResNet50Backbone` accepts only "<stage>.<block>" names.
`TruncatedBackbone` in this file walks the same child list, skips `AuxLogits`
and expands each ResNet stage into its blocks, so `layer1.0` and
`Conv2d_4a_3x3` give exactly the modules of the multigeom backbones. Layers whose
output is empty for a 64×64 input (Inception Mixed_7a onwards, after the
feature map has shrunk to 2×2) are recorded as skipped.

Features are extracted for one layer at a time, from images decoded once and
kept in memory, so peak memory is that of the largest single layer.

Output (JSON): per layer the feature shape, best validation correlation, test
correlation (raw and noise-corrected), best epoch and wall time; the best
layer; and a provenance record.

Usage: python -m multigeom.fit.layer_sweep --backbone inception --area V1 \ --out
runs/analysis/layer_sweep_inception_V1.json python -m multigeom.fit.layer_sweep
--backbone resnet50 --area V4 --layers layer1.0 layer2.0

Functions

SweepEncoder(backbone, layer, n_neurons, dropout)

Sweep encoder with the input conversion of the matching multigeom encoder.

build_torchvision_net(backbone, pretrained)

A torchvision InceptionV3 or ResNet-50, with ImageNet weights if pretrained.

main([argv])

Fit and score an encoder at every candidate layer (command-line entry point).

parse_args([argv])

Command-line arguments of python -m multigeom.fit.layer_sweep.

sweep(layers, backbone, images, responses, ...)

Train one readout per layer and score it on the test split.

Classes

InceptionSweepEncoder(*args, **kwargs)

Encoder on an InceptionV3 truncated at any layer.

ResNetSweepEncoder(*args, **kwargs)

Encoder on a ResNet-50 truncated at any block.

TruncatedBackbone(*args, **kwargs)

Frozen torchvision Inception-V3 or ResNet-50 truncated after layer.