Region of Interest Detection in Melanocytic Skin Tumor Whole Slide Images — Nevus and Melanoma

Bibliographic information

1. Executive summary

2. Clinical background and motivation

Why melanocytic tumors matter. The American Cancer Society projected that in 2022 an estimated 99,780 invasive and 97,920 in-situ melanoma cases would be newly diagnosed in the US, with ~7,650 deaths. The gold standard remains a pathologist's visual assessment of H&E-stained tissue.

The diagnostic problem. The classification of melanocytic tumors is diagnostically challenging: the paper cites literature reporting 25–26% inter-observer discordance among pathologists for distinguishing benign nevus from malignant melanoma. The clinical consequences are asymmetric: under-diagnosis of melanoma delays surgical excision and adjuvant therapy and thereby risks metastasis, whereas over-diagnosis leads to unnecessary procedures and to toxic adjuvant therapies.

The opportunity. Computational pathology can make assessment faster, cheaper, and more consistent. Prior deep-learning work scored tumor grade, classified histologic patterns, and predicted treatment response — but the authors note that prior skin-cancer literature did not address ROI detection for melanocytic tumors and had limited classification accuracy.

The goal. Build a method that both detects the ROI (what a pathologist would circle) and classifies the slide, leveraging the partial annotations pathologists already produce rather than requiring complete pixel-level labels.

3. Problem definition

Two outputs are required per WSI:

• Slide classification — ŷ ∈ {Melanoma, Nevus}.
• ROI mask B — the set of patches predicted to lie in the diagnostic region, compared against the pathologist-annotated region A.

The supervision challenge. During training, only slide-level labels and partial ROI annotations are available. Every training slide has a label, but pathologists annotated only some regions — not all ROIs were annotated. An unannotated region may still contain tumor, so naively labeling "everything outside an annotation" as non-tumor would inject label noise.

The key idea. Rather than pixel-level segmentation (which lacks exhaustive labels), PCLA-3C reframes the task as 3-class patch classification, where a manually curated "Other" class supplies clean background examples. Both slide classification and ROI detection then fall out of the same per-patch scores — the paper's "semi-supervised" use of partial annotations.

4. Dataset: the UNC Melanocytic Tumor Dataset

Effective data size. Only 160 slides exist, but each gigapixel WSI yields thousands of patches, so the effective training set is hundreds of thousands of patches — enough for CNN training and better within-slide generalization.

5. Method: the PCLA-3C framework

The framework (Figure 2 of the paper) has a Training stage and a Testing stage. Each step's code location appears in Section 6.

5.1 Preprocessing: color normalization

Scans vary in color across labs and even across time within one lab; a CNN may exploit these spurious differences rather than genuine tissue morphology. All WSIs are therefore color-normalized into a common space using the two established stain-normalization methods the paper cites: Ruifrok & Johnston color deconvolution [28] and Macenko et al. [29].

Both methods rest on the Beer–Lambert law. For incident light of intensity I₀ transmitted through stained tissue at measured intensity I (per RGB channel), the optical density (OD) is

and OD is linearly proportional to stain concentration: each stain has a characteristic, constant absorption (OD) signature across the three RGB channels. This linearity is what makes the stains separable.

Ruifrok & Johnston [28] — color deconvolution (fixed stain vectors). The specimen is modeled as a mixture of up to three stains (for H&E: hematoxylin, eosin, and a residual/background channel).

1. Each stain is assigned a normalized OD vector — its RGB color signature — originally measured from control slides stained with a single stain.
2. These vectors are stacked into a 3×3 stain matrix M (one row per stain).
3. For a pixel with measured optical density OD, the relationship is OD = M^Tc, where c is the vector of per-stain concentrations. Because M is (approximately) invertible, the concentrations are recovered by deconvolution, c = (M^T)⁻¹OD, separating the RGB image into independent per-stain density channels.
4. Normalization re-projects those concentrations onto a set of reference/standard stain vectors, so slides processed under different conditions share a common stain appearance.

Macenko et al. [29] — automatic, data-driven stain estimation. Rather than using pre-measured stain vectors, this method estimates them per slide directly from the image:

1. Convert RGB → OD and discard near-transparent background pixels (OD below a small threshold, e.g. β = 0.15).
2. Because H&E optical-density vectors lie approximately in a 2-D plane, take the singular value decomposition (SVD) of the OD pixel matrix and keep the two directions with the largest singular values — the plane that best explains stain variation.
3. Project pixels onto that plane, compute each pixel's angle, and take the robust extreme angles (e.g. the 1st and 99th percentiles) as the two stain vectors (hematoxylin and eosin) — estimated automatically, with no control slides.
4. Solve the linear system (non-negative least squares) for the per-pixel stain concentrations, and take a robust maximum (e.g. the 99th percentile) as each stain's reference concentration.
5. Normalize by rescaling every slide's stain concentrations to a chosen target (reference-slide) stain matrix and maximum concentrations, then recompose OD → RGB — mapping all slides into the reference color space, removing inter-scanner and inter-batch variation while preserving morphology.

In this study, one of these normalizations is applied to every WSI before patch extraction, so the downstream VGG16 classifier learns morphology rather than staining artifacts.

5.2 Patch extraction and labeling

• Tiling. Non-overlapping 256×256 patches at 20×; tissue detection first discards empty background.
• Augmentation. Random crop, random horizontal flip, and normalization preserve edge features and improve generalization.
• Labeling rule (slide label + XML annotations):
  • Nevus slide → patches inside annotations labeled Nevus.
  • Melanoma slide → patches inside annotations labeled Melanoma.
  • Patches outside annotations labeled Other.
• Handling incomplete annotations. Because not all ROIs were annotated, some tumor patches lie outside annotations and would be wrongly labeled "Other." The authors instead manually extracted "Other" patches from curated regions to avoid contaminating the background class.

5.3 Model and training

• Architecture: VGG16 [10] base with the final fully connected classification layer replaced by a three-unit output head, fine-tuned by backpropagation.
• Output: three per-patch scores (melanoma, nevus, other).
• Optimization (from the code): cross-entropy loss, Adam, learning rate 5×10⁻⁴, weight decay 1×10⁻⁴, early stopping (patience 20) on validation loss.

5.3.1 Detailed model architecture (VGG16)

The paper states only that VGG16 is the base and that "the last layer's parameters" were changed; the per-layer specification below follows the canonical VGG16, configuration "D" (Simonyan & Zisserman [10]) as instantiated by torchvision. The concrete instantiate_model('vgg16', True, 3) routine is not shown in the released scripts, so the specifics are reconstructed from the paper together with standard torchvision/VGG conventions.

• Input. An RGB patch of 224 × 224 × 3 during training (after RandomCrop(224, padding=4)) and 256 × 256 × 3 at inference. Channel-wise normalization uses the dataset-specific H&E statistics — mean (0.6632, 0.4123, 0.5529), std (0.1618, 0.1749, 0.1478) — not ImageNet statistics.
• Feature extractor. Thirteen 3 × 3 convolutions (stride 1, padding 1), each followed by ReLU, in five blocks separated by 2 × 2 max-pooling (stride 2):

• Adaptive pooling. AdaptiveAvgPool2d((7, 7)) fixes the feature map to 7 × 7 × 512 regardless of input size — precisely why a 256 × 256 test patch (which pools to 8 × 8 × 512 before this layer) is accepted despite the 224 × 224 training crop.
• Classifier head. Flatten (25 088) → Linear(25088, 4096) → ReLU → Dropout(0.5) → Linear(4096, 4096) → ReLU → Dropout(0.5) → Linear(4096, 3). The original ImageNet head's final Linear(4096, 1000) is replaced by Linear(4096, 3) — the "changed last layer." The three outputs are raw class logits {Melanoma (0), Nevus (1), Other (2)}; no softmax is applied (see §6.2).
• Depth. 13 convolutional + 3 fully connected = 16 weight layers ("VGG-16"). With the 3-way head the network has ≈ 134 million trainable parameters (vs. ≈ 138 M for the 1000-class original).
• Transfer learning. The second argument of instantiate_model('vgg16', True, 3) denotes ImageNet-pretrained initialization; the backbone is fine-tuned end-to-end on the H&E patches rather than trained from scratch.
• Training configuration. CrossEntropyLoss on the logits; Adam (lr = 5e-4, weight_decay = 1e-4); batch size 100; up to 20 epochs. Note that early-stopping patience = 20 equals n_epochs = 20, so under the default configuration early stopping can never actually fire — training runs the full 20 epochs and every per-epoch checkpoint is retained for later selection (§6.3).

5.4 Slide classification (testing stage)

All patches of a WSI are classified by taking, for each patch, the class with the highest predicted score (the argmax over the three class scores). "Other" patches are ignored. The slide label is a majority vote: if #melanoma patches > #nevus patches → melanoma, else nevus (ties resolved in favor of nevus, consistent with the strict inequality in the code).

5.5 ROI ranking and detection

After the slide class is decided, all patches are ranked by the winning class's score — by melanoma score for a melanoma slide, by nevus score for a nevus slide. The predicted ROI is the set of top-ranked patches, sized by the annotated ratio β.

5.6 The two key metrics: β and IoU

Annotated ratio β — for a slide C with annotated region A:

where A_p is the number of patches inside the annotation and C_p is the total number of patches in the slide. Equivalently, β is the proportion of the slide's extracted patches (by patch count, not area) that fall within the pathologist-annotated region.

Top-nβ selection. With n = total patches in the slide, the model designates the top nβ highest-scoring patches as the predicted ROI B. Example: if β = 0.2, the model predicts its top 20% of patches as ROI.

Intersection over Union (IoU) — patch-counted:

As implemented in analysis.py:

• num_inregion = patches inside the annotation (|A|).
• num_highlight = patches predicted as ROI (|B|).
• num_inregion_highlight = patches both inside and predicted (|A ∩ B|).

5.7 Visualization (three map types)

1. Overlay map. Highlights top-ranked patches, masks the rest with a transparent blue tint. When ground-truth XML is available the threshold is set from β (via get_percent); with --no_xml it falls back to --percent (default 0.2). The highlighted fraction is therefore approximately β (owing to the per-class-threshold and argmax gating of §6.2), so the highlighted region closely approximates the predicted ROI.
2. Boundary map. Draws the boundary of the largest ROI cluster. Highlighted patches are grouped with the OPTICS density-based clustering algorithm [30]; the largest cluster's outline is rendered via an alpha-shape hull — showing how precisely tumor margins are captured.
3. Heatmap. A continuous color gradient (matplotlib coolwarm): red = high predicted score, blue = low — conveying model confidence across the slide.

Figure 1 of the paper shows the goal: the pathologist's ROI (black dots) on the left, reproduced as PCLA-3C's green boundary on the right.

6. The code: repository walkthrough and paper-to-code mapping

6.1 Repository layout

understand-deeplearningproject/
├── paper/cancers-16-02616.pdf     # the paper
└── codes/                         # all source; run commands from here
    ├── extract_patches_3class.py  # Stage 1: patch extraction + labeling
    ├── method_pcla_3class.py      # Stage 2: train VGG16 3-class classifier
    ├── score_pcla_3class.py       # Stage 3: score all patches, slide classification
    ├── visual.py                  # Stage 4: overlay / boundary / heatmap
    ├── analysis.py                # Stage 5: IoU computation
    ├── automated_pipeline.py      # config-driven orchestrator (subprocess wrapper)
    ├── batch_process.py           # run the pipeline over multiple configs
    ├── config.yaml / config_example.yaml
    ├── setup.py / requirements.txt
    └── utils/                     # CLAM-derived helpers (not referenced by the active pipeline)

Each stage is an independent argparse script; stages communicate only through files on disk (patch PNGs, HDF5 score files, text logs, CSVs).

6.2 The five-stage pipeline

Stage 1 — extract_patches_3class.py (patch extraction & labeling)

Implements §5.2. Key functions:

• makemask(w, h, xml_path) — parses the Aperio XML, iterates each <Region>'s <Vertex> coordinates, and rasterizes the polygons into a binary mask via cv2.fillPoly. This is the code realization of the annotated region A. Note that makemask() exists in both extract_patches_3class.py and visual.py with different internal conventions: the extraction copy allocates np.zeros((h, w)) and is called makemask(w, h, ...), whereas the visual copy allocates np.zeros((w, h)) but is called makemask(image.height, image.width, ...); both yield a (height × width) mask, but the parameter order and names differ.
• checkinout(patch_mask, annotation_ratio=1.00) — a patch is in-region if the fraction of annotated pixels meets the ratio. Extraction is effectively hardcoded to 1.00: compute_w_loader calls checkinout(img_mask) without passing annotation_ratio, so the config.yaml value annotation_ratio=0.5 affects only visual.py, not extraction.
• compute_w_loader(...) — iterates patch coords from the slide's HDF5 bag, checks each against the mask, and saves in-region patches as PNGs named {output}_{x}_{y}.png (coordinates encoded in the filename).
• make_dirs(...) — builds the feat_dir/{train,val,test}/{Melanoma,Nevi,Other}/ tree the trainer consumes.

The two annotation sources map to the labeling rule: --xml_annotation_new sets the Melanoma/Nevus label for in-region patches, while --xml_annotation_other routes patches into the manually curated Other class.

Stage 2 — method_pcla_3class.py (train PCLA-3C)

• Loads patches with torchvision.datasets.ImageFolder. Alphabetical class indexing gives {Melanoma: 0, Nevi: 1, Other: 2} — consistent with later decision logic.
• Train transform: RandomCrop(224, padding=4) → RandomHorizontalFlip → ToTensor → Normalize(mean=(0.6632, 0.4123, 0.5529), std=(0.1618, 0.1749, 0.1478)). (Note: 224-px training crop vs. full 256-px test patches; VGG16's adaptive pooling accepts both.)
• Model: instantiate_model('vgg16', True, num_class=3).
• Loop: cross-entropy, Adam (lr=5e-4, weight_decay=1e-4), validation each epoch, early stopping (patience=20). Checkpoints saved as {exp_name}_epoch{N}_loss{L}_acc{A}.pt; the accuracy token records per-epoch training accuracy and is later parsed to auto-select a checkpoint.

Stage 3 — score_pcla_3class.py (score patches, slide classification)

• Loads each slide's patches from {slide_id}.h5, runs the model, and accumulates class_mel (argmax==0) and class_nev (argmax==1).
• Slide label = majority vote: label = 0 (melanoma) if class_mel > class_nev else 1 (nevus) — §5.4 in code. check_correct encodes Melanoma↔0 / Nevi↔1 to tally slide accuracy.
• Per-slide output results_dir/{exp_name}/score/{slide_id}.h5 with datasets scores (raw 3-class logits), coord (patch coordinates), pred (slide label). These files feed visualization and analysis. No softmax is applied anywhere in the pipeline: ScoreToColor, get_thrd, and ScoreToRank all threshold or rank these raw per-class logits (the visual.py variables named probs/prob0/prob1 are logits, not probabilities).
• Writes classification_{exp_name}.csv, whose classification_result column is recomputed independently from the counts as Melanoma if mel_con > nev_con else nevi (note the lowercase nevi; ties fall to nevi), and prints overall slide accuracy. Normalization mean/std arrive via --mean1..3/--std1..3 flags (from config.yaml) and must be kept in sync with the training values.

Stage 4 — visual.py (overlay, boundary, heatmap)

• get_percent(...) computes β from the ground-truth mask (fraction of in-region patches) — the code form of β = A_p/C_p.
• get_thrd(probs, percent) computes a per-class percentile threshold at the 100 − percent·100 percentile (thrd0 on the melanoma logit, thrd1 on the nevus logit) — approximately selecting the top nβ patches.
• ScoreToColor(...) (overlay/boundary) highlights a patch only when its winning-class logit exceeds that per-class threshold and its argmax agrees with the slide label; because of the extra argmax gate, the highlighted set is at most the top nβ patches. It returns a discrete decision (0 = highlight-melanoma, 1 = highlight-nevus, −1 = not highlighted), whereas ScoreToRank(...) (heatmap) returns a continuous min–max-normalized percentile rank mapped to a coolwarm color.
• Overlay: masks the slide with transparent blue (plt.cm.coolwarm(0.05, alpha=0.5)), reveals only highlighted patches, and logs per-slide num_inregion_highlight, num_inregion, num_highlight to {exp_name}.txt — exactly the counts IoU needs.
• Boundary: clusters highlighted patches with sklearn.cluster.OPTICS, keeps the largest cluster, fits an alphashape hull, draws the green contour with cv2.drawContours.
• Heatmap: blends the colored score map with the slide (original_image * 0.5 + heatmap_image * 0.5).
• Outputs are pyramidal .tiff written with pyvips for WSI-scale rendering.

Stage 5 — analysis.py (IoU)

Parses the {exp_name}.txt log and computes, per slide,

union = num_inregion + num_highlight − num_inregion_highlight
IoU   = num_inregion_highlight / union

skipping slides with zero in-region patches, and writes summary_iou_final.csv. (It hardcodes reading a label file melanomal.csv and a single experiment name pcla_3class.)

6.3 Orchestration layer

• automated_pipeline.py — ROIDetectionPipeline reads config.yaml and runs all five stages via subprocess, mapping config keys to CLI flags. It skips stages whose outputs exist (skip_existing) and auto-selects a checkpoint by parsing the _acc filename token (the highest training-accuracy checkpoint, not necessarily the best-validation model). --validate-only checks the config without running.
• batch_process.py — runs the pipeline once per config_*.yaml in a directory and prints a pass/fail summary.
• config.yaml — single source of truth (architecture; batch_size=100; n_epochs=20; learning_rate=5e-4; patch_size=256; annotation_ratio=0.5; percent=0.2; normalization mean/std). The validator rejects any path still set to a PATH_TO_* placeholder.

6.4 Paper → code mapping (quick reference)

7. Results

7.1 Method comparison — full training set (Table 1)

PCLA-3C raises slide classification accuracy from 69.2% to 92.3% and IoU from 0.112 to 0.382, a more than threefold increase in ROI overlap relative to CLAM.

Comparison protocol. Both methods were trained on the identical 134-WSI split and evaluated on the identical 26-WSI test set. CLAM is label-free (slide labels only), whereas PCLA-3C additionally consumes the pathologists' partial ROI annotations; the comparison is therefore best read as quantifying the value that those partial annotations add, rather than as an architecture-only contest (see Reviewer 1 in reply_to_reviewer.md).

7.2 Confusion matrix (Table 2)

The paper designates nevi as positive and melanoma as negative, reflecting the clinical priority of correctly identifying benign cases.

Reported summary metrics: accuracy 93.5%, sensitivity 81.8%, specificity 100%. The reported specificity of 100% indicates that no malignant case was misclassified as benign on this test set, a property of particular clinical importance given the asymmetric cost of under-diagnosis. (See Section 11 for a note reconciling these counts with the 26-slide test set.)

7.3 Robustness across training-set subsampling (Table 3)

The model was retrained on random subsets (20/40/60/80%) while the test set stayed fixed. CLAM has no patch-classification row. Values are mean [95% CI].

Interpretation. PCLA-3C exhibits a gradual, monotonic decline in performance as the training set is reduced: even at a 20% subset its IoU (0.30) already exceeds CLAM's best (0.11, at the 80% subset), and slide classification accuracy increases monotonically with training-set size (0.74 → 0.89). The paper also reports, at 80% (107 WSIs), a patch-level accuracy of 0.7866 [0.761, 0.813] and slide-level accuracy of 0.885 [0.857, 0.914]. The consistent margin over CLAM across all subsampling levels indicates that combining partial annotations with an explicit background class is the principal source of the improvement, and that PCLA-3C is comparatively data-efficient and robust.

8. Misclassification analysis

PCLA-3C misclassified only two slides — both true nevi predicted as melanoma:

• Figure 4 case — not a typical nevus: features of a pigmented spindle cell nevus, a recognized diagnostic challenge that is hard even for experts.
• Figure 5 case — a routine nevus misled by color: melanoma ROIs are generally dark and nevus regions light, but this slide had dark areas outside the annotated ROI, pushing the model toward a melanoma prediction and an incorrect ROI.

The analysis explicitly acknowledges this limitation: the model can be confounded by staining and darkness cues and by rare morphological variants — the same cases that challenge human readers.

9. Discussion, related work, and limitations

9.1 Positioning against related work

• CLAM [31] (clustering-constrained attention MIL; Lu et al., Nat. Biomed. Eng. 2021) is the primary baseline — strong on renal/lung cancer but with unsatisfactory ROI detection on melanocytic tumors, motivating PCLA-3C.
• CNN classifiers for breast/skin cancer [13–16, 37, 38] mostly do classification only; PCLA-3C adds ROI detection with higher accuracy.
• Transformers — Feng et al.'s T2Net [34] (MRI reconstruction/super-resolution) and Wang et al.'s TED-Net [35] (low-dose CT denoising), plus transfer-learning skin classifiers (Khalid et al. [36]) — cited as complementary directions.
• Weakly-supervised segmentation — Lerousseau et al.'s WMIL [39] generates patch pseudo-labels from slide labels, highlighted as a promising future avenue.

9.2 Strengths (in detail)

1. Detection without complete labels — the central novelty. PCLA-3C learns ROI localization from partial annotations plus a manually curated "Other" background class, rather than from exhaustive pixel/patch masks (which do not exist for this cohort). This addresses the defining obstacle in melanocytic pathology: exhaustive annotation is infeasible and the ground truth is itself uncertain given the documented 25–26% inter-pathologist discordance [40].
2. One model, two tasks. Slide classification (majority vote over patch predictions) and ROI localization (ranking patches by winning-class score) both emerge from the same patch classifier — no separate detector or segmentation head.
3. Large margin over the state-of-the-art baseline. On the fixed 26-WSI test set, slide accuracy improves 0.692 → 0.923 and IoU 0.112 → 0.382 (a more than threefold gain in ROI overlap) relative to CLAM.
4. Data efficiency and robustness. In Table 3, PCLA-3C trained on only 20% of the data already surpasses CLAM's best IoU (at 80%), and accuracy/IoU improve monotonically with data, with comparatively tight 95% CIs.
5. Clinically aligned evaluation. With nevi as the positive class, the 100% specificity means no malignant case in the test set was labeled benign — the error direction that matters most clinically.
6. Interpretability. Three visualization modes (overlay, boundary, heatmap) give spatially grounded, human-readable explanations of where the model sees tumor, supporting review and trust.
7. Accessibility. A VGG16 patch classifier runs on a single RTX 3090 and is compatible with open tooling such as FastPathology — a low barrier to reproduction and deployment.

9.3 Weaknesses (in detail)

1. Not autonomous. Slide-classification accuracy is 92.3%, so the model cannot replace a pathologist; two true nevi were classified as melanoma. It is framed as decision support.
2. Residual color/stain confounding. Even after color normalization, dark tissue outside the annotated ROI can bias the model toward "melanoma" (Figure 5), because melanoma regions are typically darker than nevi.
3. Small, single-institution cohort. Only 160 WSIs from one institution (UNC), with 26 test slides and no external, multi-scanner validation in the published work; generalization is unproven.
4. Annotation and curation dependence. Gains rely on the partial annotations, and the "Other"-class curation is manual and not fully specified, limiting exact reproducibility; performance degrades as training data shrinks.
5. Dated backbone with no domain pretraining. VGG16 (2014) predates modern vision Transformers and pathology foundation models, and is initialized from ImageNet rather than self-supervised pathology pretraining, bounding its representational power (see §9.5).
6. No slide-level context. Patches are classified independently; no inter-patch or global-slide context is modeled (unlike attention-MIL or slide encoders), which can fragment the predicted ROI.
7. Evaluation coupling. IoU is computed at the ground-truth annotated ratio β (which sizes the prediction), and ranking uses raw logits with a per-class-threshold-plus-argmax gate — defensible, but to keep in mind when comparing protocols.

9.4 Conclusions and future work

The framework produces an accurate and robust approach to detect skin tumors and predict tumor type, which the authors suggest could reduce clinician workload and improve diagnostic efficiency. The authors anticipate that the approach may generalize to other cancers and to vision-based treatment-outcome prediction. Future work stated in the paper: extract richer WSI information and incorporate extra modalities (gene expression, clinical data) to further improve detection and prediction.

9.5 Beyond the paper: post-2024 medical-imaging models and how they could extend PCLA-3C

The most impactful development since this paper is the arrival of pathology foundation models and promptable segmentation models. The VGG16 patch classifier is the natural component to modernize; the 2024 models below are the leading candidates, with their architectures summarized.

• UNI (Chen et al., Nature Medicine 2024) — a general-purpose tile encoder. Architecture: a Vision Transformer, ViT-Large/16 (~307 M params) pretrained with the DINOv2 self-supervised objective on Mass-100K (~100 M H&E tiles from ~100,000 WSIs across 20 tissue types); emits a 1024-dim embedding per 256×256 tile. Extension: replace the VGG16 backbone with a frozen/lightly-tuned UNI encoder and train only a small 3-class head — typically far higher accuracy and data efficiency than an ImageNet CNN.
• Virchow / Virchow2 (Paige, 2024) — a large pathology tile encoder. Architecture: a ViT-Huge/14 (~632 M params) trained with DINOv2 on ~1.5 M WSIs (billions of tiles); Virchow2 adds pathology-specific augmentations and multi-magnification training. Extension: a stronger drop-in patch encoder where compute permits.
• Prov-GigaPath (Xu et al., Nature 2024) — a whole-slide foundation model supplying the slide-level context PCLA-3C lacks. Architecture: (i) a ViT tile encoder (DINOv2) over 256×256 tiles, then (ii) a slide encoder built on LongNet, a Transformer with dilated attention scaling to tens of thousands of tiles per gigapixel slide; pretrained on ~1.3 B tiles from ~171,000 WSIs. Extension: a LongNet-style aggregator would let the model reason about global slide layout instead of voting over independent patches.
• MedSAM (Ma et al., Nature Communications 2024) — the medical Segment Anything Model, directly relevant to ROI detection. Architecture: a ViT-Base image encoder, a prompt encoder (bounding-box prompts), and a lightweight transformer mask decoder with transposed-convolution upsampling; trained on ~1.5 M image–mask pairs across ~10 modalities. Extension: PCLA-3C's coarse patch ROI could seed box prompts for MedSAM to produce a pixel-precise boundary, improving IoU beyond the patch grid.
• CONCH (Lu et al., Nature Medicine 2024) — a vision–language pathology model. Architecture: a CoCa-style dual encoder (ViT image encoder + text encoder) trained with contrastive and image-captioning objectives on ~1.17 M pathology image–caption pairs. Extension: enables zero-shot / text-promptable classification and report-style outputs, reducing dependence on curated patch labels.

Synthesis for future work. A modern successor to PCLA-3C would (a) swap VGG16 for a DINOv2-pretrained pathology tile encoder (UNI or Virchow2), (b) add a slide-level LongNet aggregator (Prov-GigaPath) for global context, and (c) refine the coarse patch ROI into a precise boundary with a promptable segmenter (MedSAM) — while retaining this paper's key insight that partial annotations plus an explicit background class are enough to supervise ROI detection.

10. Reproducibility and computational environment

Typical run order (from codes/, once the models/ and datasets/ helper packages are on the PYTHONPATH and a color-normalized dataset is available):

1. python extract_patches_3class.py --data_dir ... --csv_path ... \
      --xml_annotation_new ... --xml_annotation_other ... --feat_dir ...
2. python method_pcla_3class.py --exp_name pcla_3class \
      --data_folder <feat_dir> --batch_size 100 --n_epochs 20
3. python score_pcla_3class.py --exp_name pcla_3class --model_load <ckpt> \
      --csv_path ... --patch_path ... --results_dir ... --classification_save_dir ...
4. python visual.py --exp_name pcla_3class --csv_path ... --wsi_dir ... \
      --results_dir ... --xml_dir ... [--heatmap|--boundary] --percent 0.2
5. python analysis.py --results_dir <results_dir>/pcla_3class --csv_dir <dir with melanomal.csv>

Or run everything via the orchestrator: python automated_pipeline.py --config config.yaml (use --validate-only first).

Metadata CSV contract: columns slide_id, data_split (train/val/test), label_name (Melanoma/Nevi/Other). WSIs are .svs; per-slide patch bags are .h5 keyed by slide_id.

11. Reconciling the reported numbers

For a fully faithful reading, a few figures in the paper do not perfectly reconcile with one another. These are noted for the careful reader; none change the paper's conclusions.

• 160 vs. 165 WSIs. The paper consistently states 160 WSIs (86 + 74). The repository README's overview mentions 165 — treat 160 as authoritative.
• Training-set composition. The main text gives 71 melanoma + 63 nevus = 134 and test 15 + 11 = 26 (self-consistent with 86/74). Figure 2a's panel labels the training set "Melanoma 67, Nevus 57" (sums to 124) — the main-text 71/63 numbers are the consistent ones.
• Confusion-matrix totals vs. 26-slide test set. Table 2's cells (20 / 0 / 2 / 9) sum to 31 and give 93.5%, whereas the headline slide accuracy is 92.3% (i.e. 24 of 26, with exactly 2 nevi misclassified as melanoma). The 2-false-melanoma count is consistent throughout; the absolute cell counts / 93.5% do not fully reconcile with 26 slides. The 92.3% / 24-of-26 / 2-misclassified framing is the internally consistent one. (The companion reply_to_reviewer.md, Reviewer 2, presents a corrected confusion matrix that sums to 26 and matches 92.3%.)

12. Glossary and abbreviations

13. Key references

Selected references cited here (numbering follows the paper):

• [10] Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition (VGG). ICLR 2015.
• [28] Ruifrok, A.C.; Johnston, D.A. Quantification of Histochemical Staining by Color Deconvolution. Anal. Quant. Cytol. Histol. 2001.
• [29] Macenko, M. et al. A Method for Normalizing Histology Slides for Quantitative Analysis. IEEE ISBI 2009.
• [30] Ankerst, M.; Breunig, M.M.; Kriegel, H.P.; Sander, J. OPTICS: Ordering Points to Identify the Clustering Structure. SIGMOD 1999.
• [31] Lu, M.Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images (CLAM). Nat. Biomed. Eng. 2021.
• [32] Pedersen, A. et al. FastPathology. IEEE Access 2021.
• [39] Lerousseau, M. et al. Weakly Supervised Multiple Instance Learning Histopathological Tumor Segmentation (WMIL). MICCAI 2020.
• [40] Hekler, A. et al. Pathologist-level classification of histopathological melanoma images with deep neural networks. Eur. J. Cancer 2019.

The full reference list (41 entries) is in the paper, pages 11–12.

Appendix A. Peer-review clarifications

A companion document, reply_to_reviewer.md, gives point-by-point responses to three Cancers reviewers. The substantive points relevant to this explainer are summarized below:

• Reviewer 1 (ML methodology). Requested a full architecture specification (addressed in §5.3.1), clarification of the CLAM comparison protocol (§7.1), confirmation that ranking uses raw logits (§6.2), and documentation of the "Other"-class curation (§5.2).
• Reviewer 2 (clinical). Raised single-institution / small-cohort external validity, the Table 2 confusion-matrix inconsistency (§11), color/stain confounding (§8), and the intended decision-support role.
• Reviewer 3 (statistics). Asked for the confidence-interval resampling protocol (§7.3), correction of the Figure 2a training-split counts (§11), justification of the β-sized top-nβ IoU protocol (§5.6), and broader baselines (§9.1).