1 Introduction
Knee osteoarthritis (OA) is a prevalent degenerative disease in which progressive cartilage loss narrows the radiographic joint space [1]. In routine practice, joint-space morphology on weight-bearing X-rays is used to assess disease severity through the Kellgren–Lawrence (KL) grading system [1, 2]. Manual assessment remains common but is labor-intensive and prone to interobserver variability, particularly in large longitudinal cohorts [3, 4].
Although deep learning has improved automated analysis of knee radiographs [5, 6, 7], two barriers remain important in data-scarce settings. First, high-performing segmentation models typically depend on dense pixel-wise annotations [8, 9]. Second, zero-shot foundation models such as MedSAM are sensitive to prompt placement and can leak into adjacent bony structures when boundaries are weak or degraded [10, 11]. These limitations motivate a pipeline that reduces annotation burden while stabilizing zero-shot segmentation.
We therefore formulate knee joint segmentation as an integrated three-stage pipeline: few-shot JROI localization, zero-shot Med-SAM segmentation, and PO-IoU-based prompt calibration. The proposed method uses histogram-derived structural information inside the detected JROI to identify the population-level prompt offset that yields the most consistent masks. Every stage is deployable at modest annotation cost: sparse box annotations are sufficient for localization, zero-shot MedSAM provides candidate masks, and the final selection step is driven by image statistics. This makes the overall framework appealing for retrospective cohort analysis where curated segmentation masks are unavailable but radiographs and severity labels are abundant.
The main contributions of this study are threefold. First, we present PO-IoU, a prompt-calibration criterion that scores Med-SAM candidate masks against a histogram-derived structural proxy in place of unavailable pixel-wise ground truth. The design is deliberately minimal—an off-the-shelf detector, a zero-shot segmenter, and a single population-level offset—so that the entire pipeline can be transferred to a new cohort without any segmentation label, additional training, or per-image tuning. Second, we combine few-shot YOLO localization and zero-shot prompt-based segmentation into a single end-to-end practical workflow for knee radiographs. Third, we show that the resulting PO-IoU values remain clinically informative at scale by exhibiting a significant negative relation-ship with KL grade in a large OA cohort.
2 Related Work
Before deep learning, knee joint-space assessment relied on semi-automated and classical computervision methods. Early approaches applied Canny edge detection and morphological operations to locate the tibia–femur boundary, from which joint-space width (JSW) was estimated along a series of vertical profiles [32, 33]. Active shape and active appearance models later enabled more flexible bone contour extraction [34], and atlas-based registration methods aligned a template skeleton to each new radiograph to derive JSW measurements automatically [35]. While these methods achieve clinically acceptable precision in controlled settings, they require careful parameter tuning, depend on consistent acquisition geometry, and typically need expert review before deployment on large heterogeneous cohorts. Their reliance on precise bone boundary localization also makes them sensitive to the same low-contrast and sclerotic conditions that challenge deep learning approaches. The proposed pipeline sidesteps explicit bone contour estimation entirely: by anchoring MedSAM to the radiolucent joint-space interior via a histogram-derived proxy, PO-IoU operates on intensity statistics rather than boundary geometry, offering a complementary route that is less sensitive to boundary degradation.
Recent studies have shown that deep neural networks are effective for knee OA analysis from plain radiographs, including severity grading, ROI localization, and structural assessment [5, 12, 13]. Yet many of these methods rely on carefully curated labels or dense annotations that are difficult to scale in clinical practice. Follow-up studies have explored few-shot learning, external validation, and broader review perspectives, underscoring both the promise and the remaining generalization gap of current OA systems [14, 15, 16, 7]. Detection-oriented pipelines have also improved structural focus by combining localized analysis with classification or grading [17, 18, 19].
Foundation segmentation models such as SAM and MedSAM offer a promising alternative by enabling prompt-based zero-shot inference [20, 10, 21]. In medical imaging, however, prompt sensitivity remains a major limitation: small changes in prompt position can substantially alter the predicted mask, especially under low contrast and weak anatomical boundaries [11, 22, 23]. In practice, many prompt-automation approaches reduce user burden but still depend on task-specific heuristics, additional training signals, or domain adaptation, which can weaken portability across cohorts. Our work addresses this limitation by replacing manual prompt selection with a histogram-guided objective tailored to knee radiographs. By grounding prompt selection in the local brightness distribution of the target image itself, the proposed method preserves the zero-shot nature of MedSAM while introducing a simple anatomical prior tied to the radiolucent joint space. This distinction is important for clinical translation: instead of retraining a new segmentation model for every institutional dataset, the refinement logic can be reused as long as the local radiographic appearance of the joint space remains diagnostically meaningful.
3 Proposed Method
The proposed framework consists of three stages, as summarized in Fig. 1: (1) image preprocessing and few-shot JROI localization, (2) zero-shot MedSAM segmentation, and (3) PO-IoU-based prompt calibration.
In the first stage, each input radiograph undergoes Z-score normalization followed by CLAHE (Contrast Limited Adaptive Histogram Equalization, clip limit 0.01) to reduce acquisition-dependent intensity variation and improve local contrast around the knee joint [24, 25]. This preprocessing standardizes the visual conditions under which both the histogram proxy and the segmenter operate, consistent with prior work on adaptive ROI selection for knee radiographs [26]. The preprocessed image is then intensity-inverted before YOLO inference to increase contrast around the radiolucent joint space, which empirically improves localization recall. A YOLOv8n detector trained with only N = 52 bounding-box annotations [6] then localizes the JROI, inheriting the practical design philosophy of the YOLO family [27]. Bounding-box labels were collected using LabelImg [28]. The detector is trained for 200 epochs with batch size 4, SGD optimizer, learning rate 0.01, and momentum 0.937 at input size 640 pixels; Table 1 summarizes the implementation parameters of both the detector and the PO-IoU calibration stage. During inference the confidence threshold is set to 0.2 to prioritize recall: missing the joint region is more harmful than including a slightly larger crop, because downstream prompt calibration can absorb residual background ambiguity but cannot recover from a failed localization. The detector therefore serves as a robust structural gate that constrains MedSAM to the anatomically relevant neighborhood rather than as a precise delineator. We empirically verified that this threshold choice affects coverage but not segmentation quality: at confidence 0.2 the detector localizes the JROI in 94.8% of images, compared with 76.2% at confidence 0.5, while the resulting PO-IoU is essentially unchanged between the two settings (mean difference below 0.002 over the full cohort, and 0.000 on the images localized at both thresholds). The lower threshold therefore maximizes coverage without degrading downstream segmentation.
In the second stage, MedSAM [10] generates a zero-shot segmentation mask within the detected JROI. The SAM predictor receives the preprocessed (non-inverted) image, the YOLO bounding box as a spatial constraint, and a single positive-class point prompt at (cx + ε̃, cy + θ̃), where (cx, cy) is the bounding-box center and (ε̃ = 0, θ̃ = 10) is the calibrated offset derived through PO-IoU optimization (Stage 3 below). Using both the box and the point jointly grounds MedSAM to the anatomically relevant region while allowing the point to steer the mask toward the joint space rather than adjacent bone.
In the third stage, the optimal prompt offset is determined through PO-IoU calibration. Let µJROI and σJROI denote the mean and standard deviation of pixel intensities within the detected JROI. We define an adaptive threshold as
and binarize pixels below τ to construct the proxy region Ωhist. This proxy approximates the radiolucent joint space by exploiting the fact that the joint space tends to occupy the darker local region after contrast normalization, adapting naturally to patient-specific intensity ranges and acquisition conditions (Fig. 2).
Prompt candidates are generated through a two-stage grid search centered on the JROI. Let ΩSAM(p) denote the SAM mask produced from prompt p = (cx + ε, cy + θ). The PO-IoU score for each candidate is defined as
In Stage 1 (coarse search), horizontal offsets ε ∈ [−20, +20] and vertical offsets θ spanning both upward and downward directions are evaluated at a coarse step to map the full optimization land-scape. The resulting mean PO-IoU heatmap reveals that warm-color (high-scoring) regions concentrate consistently in the downward half of the search space, while upward shifts (θ < 0) yield uniformly lower scores. This observation is anatomically consistent: the radiolucent joint space lies in the lower portion of the detected bounding box, between the femoral condyle above and the tibial plateau below. Motivated by this finding, Stage 2 (refined downward search) restricts the vertical range to θ ∈ [0, +30] pixels and sweeps ε ∈ [−20, +20] at step size 2 px, yielding a 21 × 16 candidate grid. Fig. 3 visualizes both search stages: the coarse landscape in panel (a) and the refined landscape with the identified global maximum in panel (b).
The grid search is conducted once across the full dataset. The global maximum identified in Stage 2, (ε̃ = 0, θ̃ = 10), is then applied as a fixed bias at every inference, making deployment deterministic and eliminating perimage grid search overhead. After MedSAM inference, connected-component analysis retains only the largest foreground region to suppress noise blobs. PO-IoU thus serves as a prompt calibration criterion: rather than requiring a manually placed prompt, the framework identifies the population-level offset at which the joint space most consistently aligns with the histogram proxy, and fixes that offset for all subsequent inference.
4 Experimental Results
All experiments were run on a single workstation with an NVIDIA GeForce RTX 3090 GPU (24 GB) under Python 3.11 and PyTorch 2.5.1 with CUDA 12.1. Detection used Ultralytics YOLOv8n (v8.3.235), and zero-shot segmentation used the ViT-B backbone [10, 11]; image I/O and processing used OpenCV 4.12 and NumPy 1.26. Because the pipeline performs inference only—no per-image optimization is carried out and a single population-level offset is applied at test time—the procedure is deterministic and reproducible given the same inputs and model weights.
Experiments were conducted on 8,252 knee radiographs annotated with KL grades. The radiographs were drawn from a publicly available knee osteoarthritis grading dataset [36], derived from the Osteoarthritis Initiative (OAI) repository and comprising frontal knee X-rays graded KL 0–4, where left and right knees are treated as independent images. The grade distribution is markedly imbalanced—approximately 3,252, 1,494, 2,174, 1,085, and 250 images for KL 0 through KL 4, respectively—reflecting the natural prevalence of mild-to-moderate disease. Because the segmentation stage is zero-shot and performs no per-image training, the evaluation does not depend on a patient-level train/test partition; the only trained component is the lightweight YOLO localizer, fit from 52 bounding-box annotations. Public knee OA grading resources and cohort-oriented evaluation settings provide useful context for interpreting this type of large-scale analysis [29, 4]. Instead of evaluating direct geometric measurements, we examined how PO-IoU varies with disease severity. This evaluation strategy is motivated by the fact that projection-dependent geometric measurements can fluctuate with positioning and acquisition setup, whereas a structure-consistency score can still provide useful population-level information even when individual measurements are noisy. Linear regression on individual images (N = 8,252) revealed a statistically significant negative association between PO-IoU and KL grade (p < 0.001, R2 = 0.048). The low sample-level R2 is expected given large image-quality variation, class imbalance, and anatomical diversity. A separate regression on the pergrade mean values showed a clear decreasing trend with R2 = 0.819, as illustrated in Fig. 4. Because a regression fitted to five aggregated points carries limited information, we characterize the same association at the individual-image level using several independent tests, summarized in Table 2. Rank correlations, a non-parametric comparison across the five grades with its effect size, and the discrimination between early and advanced disease all confirm the negative trend on the full cohort; the grade-level regression is reported alongside them for reference rather than as primary evidence. Because the KL grade never enters the calibration objective, which is defined solely on the overlap with Ωhist, these results are measured against an external clinical label rather than against the quantity being optimized.
Linear-regression slope = −0.031; Kruskal–Wallis df = 4; AUC 95% CI [0.699, 0.727]. The grade-mean regression is fitted to five aggregated points. Pairwise post-hoc tests (Mann–Whitney, Bonferroni-corrected over ten comparisons) are significant for nine of the ten grade pairs, the sole exception being KL 0 versus KL 1; KL 3 and KL 4 differ significantly (p = 1.4 × 10−19, Cliff’s δ = 0.379) with KL 4 lowest. Grade-wise bootstrap 95% confidence intervals of the mean (5,000 resamples) are disjoint from KL 2 onward.
The mean PO-IoU decreases with disease severity, a relative decline of approximately 36% from KL 0 to KL 4, with KL 4 showing the lowest mean. The individual-level analyses establish statistical significance across the full cohort, and the grade-level summary complements them by describing the population trend. This inter-pretation follows recent recommendations that medical AI results should be read with careful attention to imbalance and evaluation metrics [30, 31]. The proposed score is therefore best understood not as a precise per-image surrogate for manual measurement but as a stable cohort-level indicator of structural degradation that becomes clearer after aggregation by disease grade.
Qualitative and configuration-ablation comparisons further support the design choices. Fig. 5 compares, for a representative case at each KL grade (rows), four prompt/segmentation configurations (columns): the vanilla center-point prompt (ε = 0, θ = 0), the PO-IoU-calibrated prompt (ε̃ = 0, θ̃ = 10), an extreme offset (ε = 20, θ = 20), and MedSAM prompted without the YOLO localizer. Relative to the vanilla center-point baseline, the calibrated prompt reduces left–right lateral leakage into the medial and lateral femoral condyles and the adjacent tibial cortex while preserving the thin, elongated morphology of the joint space in these examples. Beyond this baseline, we add two ablations that isolate the contribution of each component. First, displacing the prompt to an extreme offset (20, 20) causes the mask to over-spread well beyond the joint space into surrounding bone, illustrating that a moderate, population-level calibrated offset—rather than an aggressive shift—is what keeps the mask anatomically constrained. Second, removing the YOLO localizer and prompting MedSAM with a single image-center point alone fails to recover the joint region, showing that the detection stage is essential for restricting the zero-shot segmenter to the anatomically relevant neighborhood. These comparisons are illustrative rather than quantitative: without pixel-level ground truth the four configurations cannot be ranked by Dice or Hausdorff distance, as discussed in Section 6.
5 Conclusion
We presented PO-IoU, a segmentation-label-free knee joint segmentation framework that combines few-shot YOLO localization, zero-shot MedSAM inference, and point-offset prompt calibration. By directly exploiting image-derived structural cues instead of training a separate correction network, the method improves the stability of zero-shot segmentation while remaining lightweight. Experiments on 8,252 radiographs show a statistically significant negative association between PO-IoU and KL grade at the individual level (Spearman ρ = −0.221, p < 0.001), with early disease separated from advanced disease at an AUC of 0.713, suggesting that the proposed approach can serve as a practical structure-aware tool for scalable OA analysis in sparse-annotation settings. Qualitatively, the calibrated masks exhibit substantially reduced left–right lateral leakage into the femoral condyles and tibial cortex compared to the vanilla center-point baseline, improving the clinical plausibility of the output.
6 Limitations and Future Work
The following considerations delimit the scope within which the present results should be interpreted, and define the agenda of our ongoing work.
Nature of the score. PO-IoU quantifies the structural consistency between a predicted mask and a histogram-derived prior, and the same quantity is used both to calibrate the prompt offset and to summarize the resulting masks; it should accordingly be read as a consistency measure rather than as a substitute for accuracy metrics such as Dice or Hausdorff distance. Two aspects of the design delimit this dependence: the KL grade used for validation does not enter the calibration objective, which is defined solely on the overlap with Ωhist, and the calibration is applied as a single population-level bias rather than optimized per image. The criterion remains meaningful within an anatomically plausible range of prompt displacements; establishing accuracy in the conventional sense nonetheless requires expert pixel-level annotation.
Scope of comparison. Quantitative comparison against classical joint-space-width pipelines, supervised segmentation networks, or alternative prompt-automation strategies requires pixel-level ground truth on a common subset; the configuration comparison in Fig. 5 is accordingly illustrative in nature.
Level of aggregation. The association with KL grade is robust in aggregate (H = 635.4, p = 3.4 × 10−136) while individual-image variation remains large (R2 = 0.048). The score is therefore best used as a cohort-level indicator of structural degradation rather than as a per-patient measurement.
Partition, cohort, and proxy assumptions. The calibration offset (ε̃, θ̃) was determined on the same cohort on which results are reported; left and right knees are treated as independent images; and all radiographs derive from a single publicly available OAI-derived source, leaving robustness across acquisition protocols untested. The proxy threshold assumes that the joint space forms the darkest local region after normalization, which may not hold under severe sclerosis, radiopaque implants, or end-stage ankylosis. Finally, the framework operates on 2D frontal radiographs and models neither bilateral correspondence, longitudinal change, nor 3D modalities such as MRI.
These considerations define our ongoing work. We are assembling an expert-annotated subset with pixel-level masks stratified across KL grades, which will allow the pipeline and its configurations to be scored with Dice, IoU, and 95% Hausdorff distance and compared directly against classical and supervised baselines. Extending the criterion with explicit plausibility terms—containment within the detected joint region, single-connected-component structure, and an elongation prior consistent with joint-space morphology—would allow the offset to be optimized rather than bounded. Beyond a single global offset, a lightweight per-image predictor fitted under a patient-level train/validation/test partition, together with validation on an independent cohort and on the longitudinal structure of the OAI cohort, would establish whether the calibration transfers and whether the score tracks joint-space narrowing within subjects over time. The same proxy-guided calibration principle may also transfer to other low-contrast muscu-loskeletal segmentation tasks where annotation cost and prompt sensitivity pose similar challenges.
