Semantic Feature Modulation for Mammographic Lesion Classification

Shahar Mahpod and Gil Ben Artzi

Ariel University, Israel

Oral - International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 2026

Presentation

Poster

Paper

Github

Dedicated MLP modulation networks map each BI-RADS descriptor (breast density, mass shape, mass margins, ...) to per-sample (γ, β) parameters, computed from that sample's descriptors, that replace the static affine parameters of selected Layer Normalization layers in a ConvNeXt backbone. Descriptors are injected either in block mode (each descriptor modulates consecutive layers) or in interleaved mode (descriptors alternate across layers within a stage).

Highlights

Context-dependent descriptors. BI-RADS descriptors condition the network through Conditional Layer Normalization: each descriptor predicts a per-channel scale and shift that is applied to the image's normalized intermediate features, so the effect of a descriptor on the representation depends on the image.
Gains over image-only and descriptor-aware baselines. Compared with the image-only baseline and descriptor-aware baselines (Concat, SE-Gate, Multi-Task), Cond-LN Inter achieves the highest or tied-highest AUC in 5 of 6 dataset × setting combinations on CBIS-DDSM and CMMD, with up to +8.3% AUC and +5.0% F1 over the image-only baseline. On CMMD mass it does not improve over the image-only baseline.
Exact sub-category interpolation. Continuous descriptor embeddings support mixtures such as "60% spiculated, 40% ill-defined", and yield monotonic risk paths aligned with clinical severity, despite training only on hard labels.

Abstract

Most mammography AI systems predict benign versus malignant from the image alone, ignoring the structured reasoning radiologists perform through the BI-RADS descriptor lexicon. We introduce a conditional modulation framework that dynamically shifts intermediate feature statistics based on BI-RADS descriptors. Rather than considering descriptors as static labels, our method produces sample-specific modulation parameters from each sample's descriptors; because these parameters act on image-dependent intermediate features, the same attribute affects the representation differently depending on the image, capturing context-dependent diagnostic meaning. We show that our approach is highly effective. Across CBIS-DDSM and CMMD, our model improves AUC by up to +8.3% and F1 by up to +5.0% over the image-only baseline. By representing descriptors as conditional continuous embeddings, our framework enables exact interpolation between attributes within the same BI-RADS category; interpolation experiments confirm monotonic risk paths aligned with clinical severity, despite training only on hard categorical labels. Our model enables a post-detection decision-support mode using probabilistic outputs from emerging BI-RADS prediction models. Perturbation analysis confirms approximately 25% of joint-setting predictions change when all descriptors are removed.

Method

We adopt ConvNeXt-Base as backbone. ConvNeXt uses Layer Normalization (LN) throughout, and since LN operates per-sample, its affine parameters are natural control points for sample-specific conditioning. We replace the static (γ, β) of selected LN layers with parameters predicted from clinical descriptors: for each descriptor type t, a dedicated two-layer MLP g^(t) maps the descriptor embedding z^(t) to modulation parameters,

CLN(h, z^(t)) = γ^(t)(z^(t)) ⊙ h − μ(h)σ(h) + β^(t)(z^(t))
Global descriptors (e.g., breast density) modulate earlier layers and lesion-level descriptors (e.g., margins, morphology) modulate later layers. Training uses hard one-hot descriptor labels, yet at inference a descriptor may be any convex combination z^(t) = (1−α) z_A^(t) + α z_B^(t), allowing exact interpolation between adjacent BI-RADS attributes without retraining.

Results

All models share the same ConvNeXt-Base backbone and training protocol. Base: image only. Concat: descriptor embeddings concatenated with pooled features. SE-Gate: SE-style channel gating conditioned on descriptors. MTL: multi-task learning with auxiliary descriptor prediction heads. Cond-LN Block / Inter: ours, with block and interleaved conditioning. Values are mean ± std (%) over 5 runs; best per setting in bold.

CBIS-DDSM

	Model	Acc	AUC	F1	Rec
Mass	Base	80.3±1.1	88.7±0.8	78.5±1.4	79.1±2.2
	Concat	79.5±2.1	88.3±0.7	77.5±2.6	77.4±3.5
	SE-Gate	80.7±1.4	87.8±0.5	78.4±1.7	77.1±2.2
	MTL	81.2±1.9	87.8±1.2	79.4±2.2	79.6±2.4
	Cond-LN Block	83.4±1.5	89.9±0.4	82.1±1.6	84.1±3.3
	Cond-LN Inter	84.7±0.9	90.9±1.3	83.0±0.9	81.8±2.7
Calc	Base	80.0±1.2	86.4±0.8	69.1±2.2	72.9±3.9
	Concat	79.1±1.8	86.5±1.1	67.7±2.5	71.4±2.6
	SE-Gate	79.5±0.8	86.6±0.7	68.9±1.0	74.0±1.6
	MTL	79.4±1.4	86.7±0.9	68.5±2.2	73.1±2.8
	Cond-LN Block	80.6±1.6	88.8±0.7	70.9±2.3	77.1±2.5
	Cond-LN Inter	82.8±1.1	90.9±0.4	74.1±0.7	80.2±4.0
Both	Base	78.8±0.6	86.9±0.1	72.3±1.1	73.9±2.0
	Concat	78.9±1.1	87.1±0.7	72.8±1.1	75.2±1.1
	SE-Gate	79.4±1.4	87.0±0.7	73.2±1.6	75.2±1.0
	MTL	80.2±0.8	88.2±0.4	74.7±0.9	78.2±1.5
	Cond-LN Block	81.2±1.5	90.3±0.3	74.2±2.4	72.2±3.8
	Cond-LN Inter	81.2±0.7	90.3±0.4	75.0±1.7	75.4±4.3

CMMD

	Model	Acc	AUC	F1	Rec
Mass	Base	85.9±0.7	86.9±0.3	91.8±0.5	94.5±1.7
	Concat	86.8±0.8	85.4±3.7	92.3±0.5	94.8±1.1
	SE-Gate	86.8±0.8	86.8±1.4	92.3±0.5	95.1±0.6
	MTL	86.9±1.2	85.4±1.6	92.3±0.7	94.4±1.4
	Cond-LN Block	86.3±1.4	85.9±1.5	92.0±0.8	94.8±1.1
	Cond-LN Inter	86.6±0.7	86.3±0.6	92.1±0.4	94.3±0.7
Calc	Base	86.5±2.0	83.3±1.7	92.5±1.1	95.3±1.0
	Concat	86.5±1.5	83.2±2.0	92.5±0.8	94.6±1.2
	SE-Gate	87.6±1.0	85.6±1.1	93.1±0.6	95.7±0.7
	MTL	88.0±0.9	82.8±1.5	93.4±0.5	96.2±1.0
	Cond-LN Block	87.3±2.4	86.4±2.7	92.8±1.4	94.2±3.0
	Cond-LN Inter	89.7±1.7	91.6±1.1	94.1±1.0	93.4±3.2
Both	Base	85.8±0.8	87.3±1.3	91.8±0.5	94.2±1.0
	Concat	85.5±0.6	85.8±1.2	91.7±0.3	93.9±0.9
	SE-Gate	86.5±0.7	87.3±0.7	92.2±0.4	94.0±0.8
	MTL	86.5±0.3	87.6±0.8	92.2±0.2	93.7±0.3
	Cond-LN Block	88.7±0.5	90.2±0.4	93.6±0.3	97.2±0.6
	Cond-LN Inter	88.1±0.5	91.7±0.8	93.2±0.3	96.6±0.4

Cond-LN Inter achieves the highest (or tied-highest) AUC in 5 of 6 dataset × setting combinations. CMMD-mass illustrates a regime where descriptor headroom is limited; the subset's imbalanced malignant prior may further constrain how much descriptor signal can add.

Descriptor Interpolation and Monotonicity

We sweep descriptor inputs as convex combinations z = (1−α) z_A + α z_B, α ∈ {0, 0.1, ..., 1}, across all CBIS-DDSM test images, regardless of their original descriptor, pathology, or lesion type. All adjacent-pair interpolation curves increase monotonically, and the ordering aligns with clinical knowledge: irregular shapes and spiculated margins are the strongest malignancy indicators. The model was trained only on hard one-hot labels and never saw soft inputs.

Mass shape

Mass margins

Mean malignancy probability rises from 0.29 (ROUND) to 0.55 (IRREGULAR) for mass shape, and from 0.22 (CIRCUMSCRIBED) to 0.69 (SPICULATED) for mass margins.

Descriptor Sensitivity

Zeroing individual descriptors on the CBIS-DDSM joint setting shows that the lesion-specific attributes, mass margins and shape, produce the largest probability shifts (left) and prediction flip rates (right).

When all descriptors are removed, accuracy drops from 81.0% to 68.0% and about 25% of predictions change.

Qualitative Analysis

Grad-CAM++ at multiple ConvNeXt depths for CBIS-DDSM mass cases. The benign case (top) exhibits diffuse patterns; the malignant case (bottom) shows increasingly localized activations, with deep-stage activations aligning with the lesion boundary.

Paper

Semantic Feature Modulation for Mammographic Lesion Classification, Shahar Mahpod, Gil Ben Artzi
MICCAI 2026
PDF | Code

@inproceedings{mahpod2026semantic,
  title     = {Semantic Feature Modulation for Mammographic Lesion Classification},
  author    = {Mahpod, Shahar and Ben Artzi, Gil},
  booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
  year      = {2026}
}

Acknowledgements

We acknowledge the Ariel HPC Center at Ariel University for providing computing resources. The website template is from here.