Skip to main content
EJNMMI Research
Download PDF

Inter-rater and intra-rater agreement of [99mTc]-labelled NM-01, a single-domain programmed death-ligand 1 (PD-L1) antibody, using quantitative SPECT/CT in non-small cell lung cancer

EJNMMI Research volume 13, Article number: 51 (2023) Cite this article

Abstract

Background

Immune checkpoint inhibitors, including those against programmed cell death protein-1 (PD-1) or its ligand (PD-L1), are routinely used to treat non-small cell lung cancer (NSCLC). PD-L1 is a validated prognostic and predictive immunohistochemical biomarker of anti-PD-1/PD-L1 therapy but displays temporospatial heterogeneity of expression. Non-invasive radiopharmaceutical techniques, including technetium-99m [99mTc]-labelled anti-PD-L1 single-domain antibody (NM-01) SPECT/CT, have the potential to improve the predictive value of PD-L1 assessment. This study aims to determine the inter- and intra-rater agreement of the quantitative measurement of [99mTc]NM-01 SPECT/CT in NSCLC.

Methods

Participants (n = 14) with untreated advanced NSCLC underwent [99mTc]NM-01 SPECT/CT at baseline (n = 3) or at baseline plus 9-week follow-up (n = 11). [99mTc]NM-01 uptake (of primary lung, lymph node, thoracic and distant metastases, and healthy reference tissues) was measured using SUVmax and malignant lesion-to-blood pool ratios with Siemens xSPECT Broad Quantification software by three independent raters. Intraclass correlation coefficients (ICC) were calculated and Bland–Altman plot analysis performed to determine inter- and intra-rater agreement.

Results

There was excellent inter-rater agreement of manual freehand SUVmax scores of primary lung tumour (T; n = 25; ICC 1.00; 95% CI 0.99–1.00), individual lymph node metastases (LN; n = 56; ICC 0.97; 95% CI 0.95–0.98), thoracic metastases (ThMet; n = 9; ICC 0.94; 95% CI 0.83–0.99) and distant metastases (DisMet; n = 21; ICC 0.91; 95% CI 0.83–0.96). The inter-rater ICCs of tumour-to-blood pool (T:BP), LN:BP, ThMet:BP and DisMet:BP measures of [99mTc]NM-01 uptake also demonstrated good or excellent agreement. Manual freehand scoring of T, LN, ThMet, DisMet and their ratios using [99mTc]NM-01 SPECT/CT following a 28-day interval was consistent for all raters with good or excellent intra-rater agreement demonstrated (ICCs range 0.86–1.00).

Conclusion

Quantitative assessment of [99mTc]NM-01 SPECT/CT in NSCLC, using SUVmax of malignant primary or metastatic lesions and their ratios with healthy reference tissues, demonstrated good or excellent inter- and intra-rater agreement in this study. Further validation with ongoing and future larger cohort studies is now warranted.

Clinical trial registration

ClinicalTrials.gov identifier no. NCT04436406 (registered 18th June 2020; available at https://clinicaltrials.gov/ct2/show/NCT04436406) and NCT04992715 (registered 5th August 2021; available at https://clinicaltrials.gov/ct2/show/NCT04992715).

Background

Lung cancer is the second most commonly diagnosed cancer and a major cause of mortality globally with over 1.8 million deaths in 2020 alone [1]. Over the past decade, the treatment paradigm of advanced non-small cell lung cancer (NSCLC) has shifted with the introduction of therapies targeting immune checkpoint molecules, including programmed cell death protein 1 (PD-1) and its ligand (PD-L1). An important mechanism of immune escape involves the upregulation of co-inhibitory molecule PD-L1 by tumour cells, which on interaction with PD-1, expressed by effector T cells, lead to their dysfunction. PD-1/PD-L1 monoclonal antibodies are now widely used in the management of advanced NSCLC with significant improvements in median overall survival demonstrated in both first- and second-line treatment compared to standard cytotoxic chemotherapy [2,3,4,5]. Importantly, even in the advanced setting, durable responses can be seen in around 20% of patients [6].

Anti-PD-1/PD-L1 therapeutic response and survival in NSCLC are associated with PD-L1 tumour proportion score (TPS) measured using immunohistochemistry, a widely available and validated biomarker [2,3,4,5,6,7]. Those with metastatic NSCLC and a PD-L1 TPS ≥ 50% are likely to respond to single agent immunotherapy, such as anti-PD-1 pembrolizumab, whereas in those with < 1% (negative) or 1–49% (low) TPS, first-line treatment would usually include a combination approach with cytotoxic chemotherapy [2, 6, 8]. This ability to predict response allows many patients to avoid the treatment burden and toxicity associated with combination therapy. However, up to 10% of patients deemed ‘non-expressors’ by immunohistochemistry may respond to anti-PD-1/PD-L1 therapy and vice versa, resulting in a proportion of patients potentially being over- or under-treated [3]. Importantly, PD-L1 expression within and between tumours is heterogenous, as well as being dynamic, with changes over time particularly following exposure to anti-cancer therapies [9, 10]. Mapping PD-L1 expression within and across tumour sites over the course of an individual’s cancer journey with multiple and serial biopsies is impractical and exposes them to additional risk. Non-invasive imaging techniques present a potential solution in overcoming the limitations of PD-L1 expression measured by immunohistochemistry and provide opportunity to improve the predictive value of PD-L1 assessment.

The 14.3 kDa camelid single-domain PD-L1 antibody, NM-01, can be radiolabelled with technetium-99m ([99mTc]), administered and subsequently detected by single-photon emission computed tomography (SPECT). A first-in-human study of [99mTc]NM-01 demonstrated both safety and acceptable dosimetry in 16 participants with NSCLC [11]. SPECT/computed tomography (CT) scans were obtained 1 and 2 h following [99mTc]NM-01 injection with 2-h primary tumour-to-blood pool ratio (T:BP) assessment correlating with PD-L1 immunohistochemistry. [99mTc]NM-01 uptake greater than blood pool was also demonstrated in nodal and bone metastases with intertumoural heterogeneity in 30% of participants. As such, non-invasive PD-L1 assessment using this novel single-domain antibody could help oncologists better stratify patients to receive the most appropriate anti-cancer therapy at the right time in their disease course and facilitate the advancement of novel imaging biomarker driven clinical trials. We hypothesised that PD-L1 expression measured using [99mTc]NM-01 with a quantitative SPECT/CT imaging approach is consistent and reproducible between individual raters and over time. Our aim was to determine the inter-rater and intra-rater agreement of [99mTc]NM-01 uptake measurements in experienced and less experienced raters in nuclear medicine within a cohort of patients with NSCLC.

Methods

Participants in the PD-L1 Expression in Cancer (PECan; NCT04436406) and PD-L1 Expression in Lung Cancer (PELICAN; NCT04992715) PD-L1 imaging studies scheduled to undergo [99mTc]NM-01 SPECT/CT imaging were included in this single-centre prospective study conducted at Guy’s and St Thomas’ NHS Foundation Trust, London, UK. Participants aged 18 years and over with histologically confirmed, untreated advanced NSCLC scheduled for systemic anti-cancer therapy, tissue available for PD-L1 analysis and an Eastern Cooperative Oncology Group (ECOG) performance score of 1 or less were eligible. Exclusion criteria included pregnant or lactating females, severe infection, and prognosis of < 3 months. All participants provided written informed consent. Participants in the PECan study underwent [99mTc]NM-01 SPECT/CT at baseline, prior to starting immune checkpoint inhibitor containing regimen, and at 9-week follow-up. Participants in the PELICAN study underwent [99mTc]NM-01 SPECT/CT at baseline only and were eligible to participate independently of planned systemic therapy. Diagnostic samples of all recruited participants underwent PD-L1 immunohistochemical assessment as standard of care using the Ventana PD-L1 (SP263) assay.

SPECT/CT protocol

SPECT/CT examinations were performed on a Siemens Symbia Intevo Bold SPECT/CT scanner calibrated for use of xSPECT Broad Quantification quantitative analysis software (Siemens Healthcare GmBH; Erlangen, Germany). Participants (n = 14) were administered an intravenous bolus of [99mTc]NM-01, median 584 MBq (range 343–721 MBq) [99mTc] corresponding to approximately 100 μg of NM-01. Participants were asked to drink 500 mL water post-injection and void their bladder prior to imaging. Whole-body planar imaging was performed with the patient supine at 2 h post-injection using a 256 × 1024 matrix at 10 cm/slice/min. Single field of view SPECT/CT imaging, focussing on primary tumour (thorax) and site(s) of suspected metastases was subsequently performed with the patient supine. Scans were acquired on a 256 × 256 matrix using low-energy high-resolution collimators, with a 15% energy window centred at 140 keV (the Tc-99m photopeak). A 15% energy window centred at 120 keV was also used for tomographic image acquisition for scatter correction. SPECT was performed over 180° with 128 projections (64 views), in step and shoot mode, 20 s per projection. A low-dose CT (130 kV, effective mAs determined using CARE Dose4D) was performed for anatomical correlation and attenuation correction. Images were reconstructed within an xSPECT Broad Quantification reconstruction workflow using OSEM iterative reconstruction (2 iterations, 10 subsets) with an additive update mechanism, at a matrix size of 128 × 128, with scatter correction. An xSPECT Broad Quantification analysis workflow was then utilised for the presentation of quantitative uptake data, gathered utilising inputted injected activity and patient weight [12].

Image analysis

Images were reviewed by three independent raters blinded to patient details and each other’s assessments using Hermes GOLD™ (Hermes Medical Solutions; Stockholm, Sweden). The raters included 1 nuclear medicine physician, 1 nuclear medicine clinical fellow and 1 oncology clinical fellow PhD student with 30-, 5- and 3-year experience in nuclear medicine image analysis, respectively. Regions of interest, including primary tumour, metastatic lesions (including thoracic lymph nodes, thoracic metastases—pleural or lung, and distant metastases), and normal tissue references (lung, blood pool, bone marrow, liver and spleen), were identified on fused SPECT/CT. SUVmax was chosen as it represents the highest voxel within a given region of interest and is independent of the exact definition of the region, as long as the highest voxel is included in the region. Whilst SUVmean, the average SUV across all voxels in a region, is less sensitive to noise, it could be significantly affected by differences in the manual segmentation and thus subject to greater inter- and intra-rater variability [13]. It is also more likely to be affected by the partial volume effect, which is of particular importance when measuring small tumours and lymph nodes in this study of early NSCLC. Additionally, the previously reported study of [99mTc]NM-01 SPECT/CT demonstrated maximum region of interest (ROImax) correlated with PD-L1 immunohistochemistry [11]. Although quantitative SPECT/CT permits the measurement of SUVmax for lesions, its methods are not fully validated and may be subject to less accuracy than PET/CT, such as that due to the partial volume effect associated with small regions of interest. As such, ratios of primary tumour and metastatic lesions to healthy reference tissues were also evaluated to allow for analysis on SPECT scanners that do not have the ability to calculate SUVmax, as with the previously reported study of [99mTc]NM-01 SPECT/CT.

Using a freehand manual technique, the maximum standardised uptake values (SUVmax) were recorded from the SPECT images (n = 25; baseline only images in n = 3, baseline and 9-week follow-up images in n = 11). Freehand SUVmax was recorded for normal lung in the right upper lobe (or contralateral upper lobe if pathology present) for calculation of tumour-to-lung (T:L) ratio and for blood pool within the aortic arch for calculation of tumour-to-blood pool (T:BP) and metastatic lesion-to-blood pool ratios. Similarly, freehand SUVmax was measured for normal haematopoietic tissues, including bone marrow (thoracic vertebra at the level of the carina), spleen and liver.

To evaluate if rule-based approaches improved consistency of scoring normal tissue references, SUVmax was measured using a standardised 3-cm-diameter sphere for normal lung at the level of the aortic arch and carina, and the liver at the level of the gastroesophageal junction (GOJ) on axial views. SUVmax was measured using a standardised 2-cm-diameter sphere for the spleen and using a 1.5-cm-diameter sphere for the bone marrow in the thoracic vertebra at the level of the carina on axial views. Examples of image analysis are provided in Fig. 1. To determine intra-rater agreement, the 3 independent raters repeated their calculations for all measured regions blind to their initial measurements following a minimum 28-day period.

Fig. 1
figure 1

Image analysis using SUVmax scoring of [99mTc]NM-01 SPECT/CT. a Freehand manual region of interest of a baseline (pre-treatment) primary right lower lobe tumour. b Freehand manual region of interest of a baseline station 7 (subcarinal) thoracic lymph node. c Blood pool SUVmax measured using a freehand manual technique. Healthy reference tissue SUVmax measurements determined by a freehand approach of d left upper lobe lung, e liver and spleen, and f bone marrow. Healthy reference tissue SUVmax measurements determined by volumetric rule-based approaches of g left upper lobe (3 cm sphere at level of aortic arch), h liver (3 cm sphere at level of the GOJ) and spleen (2 cm sphere), and i bone marrow (1.5 cm sphere at level of the carina)

Full size image

Statistical analysis

The intraclass correlation coefficient (ICC) is a widely used reliability index representing both the degree of correlation and the agreement between measurements. ICC values range from 0 to 1, where less than 0.5 indicates poor agreement, 0.5 to < 0.75 moderate, 0.75 to < 0.9 good and values greater than 0.9, i.e. close to 1 represent excellent agreement [14]. A two-way random consistent model was used to determine the ICC and its 95% confidence intervals (CI) for inter-rater agreement of all 3 raters. A two-way mixed effects absolute agreement model was used to determine the ICC and its 95% CI for intra-rater agreement for each rater. Each level of agreement is more accurately defined by their 95% confidence intervals, considering the ICC is an estimated reliability index. Normality was assessed using the Shapiro–Wilk test, and non-normally distributed data were log transformed. One-sample t test was used to assess the difference of means (where two-sided p < 0.05 is significant). Bland–Altman plots and their 95% limits of agreement were used to determine the agreement between raters and their repeat measurements for T:BP, LN:BP, thoracic metastasis-to-blood pool ratio (ThMet:BP) and distant metastasis-to-blood pool ratio (DisMet:BP) scores. Linear regression of Bland–Altman plots was performed to determine the β coefficient of the mean difference and demonstrate any proportional bias (where p < 0.05 is significant). Statistical analysis was performed using IBM SPSS Statistics for Windows, version 28.0 (Armonk, NY: IBM Corp).

Results

Participant characteristics

Participants were recruited to the study between October 2020 and September 2022 (n = 14). The median age was 64 years (range 52–75 years); all were of white ethnicity. All had a histologically confirmed diagnosis of NSCLC (adenocarcinoma n = 10, squamous cell carcinoma n = 2, not otherwise specified n = 2) with 12 participants having metastatic disease at diagnostic staging. Detailed and summarised participant characteristics are provided in Table 1 and Additional file 1: Table S1.

Table 1 Summary of participant characteristics
Full size table

Inter-rater agreement

There was excellent agreement of manual freehand SUVmax scores between all three raters of the primary lung tumour (T; n = 25; ICC 1.00; 95% CI 0.99–1.00) and individual lymph node metastases (LN; n = 56; ICC 0.97; 95% CI 0.95–0.98) (Table 2). There was good to excellent inter-rater agreement for freehand SUVmax measurements of individual thoracic metastases (ThMet; n = 9; ICC 0.94; 95% CI 0.83–0.99) and distant metastases (DisMet; n = 21; ICC 0.91; 95% CI 0.83–0.96).

Table 2 Inter-rater agreement of [99mTc]NM-01 measurements
Full size table

Freehand SUVmax measurements of blood pool (BP; n = 25; ICC 0.87; 95% CI 0.76–0.94) and lung (L; n = 25; ICC 0.93; 95% CI 0.86–0.96) normal reference tissues demonstrated good to excellent inter-rater agreement. The ICC using the volumetric rule-based approach measuring the SUVmax of normal lung at the level of the aortic arch (ICC 0.88; 95%CI 0.78–0.94) and the carina (ICC 0.94; 95% CI 0.88–0.97) also demonstrated good to excellent inter-rater agreement. Calculated T:L ratios demonstrated good to excellent inter-rater agreement with all methods (Table 2).

There was good to excellent inter-rater agreement of manual freehand SUVmax measurements of the normal haematopoietic reference tissues, including of the liver (n = 25; ICC 0.93; 95% CI 0.87–0.97), spleen (n = 25; ICC 0.90; 95% CI 0.82–0.95) and bone marrow (n = 25; ICC 0.90; 95% CI 0.82–0.95). Applying a rule-based volumetric approach to measure the SUVmax of the liver (ICC 0.90; 95% CI 0.81–0.95), spleen (ICC 0.94; 95% CI 0.88–0.97) and bone marrow (ICC 0.93; 95% CI 0.87–0.97) demonstrated good to excellent inter-rater agreement. Spleen-to-liver (SLR; ICC 0.78; 95% CI 0.63–0.89) and bone marrow-to-liver ratios (BLR; ICC 0.86; 95% CI 0.75–0.93) both demonstrated good inter-rater agreement.

Malignant lesion-to-blood pool ratios used as a quantitative measure of [99mTc]NM-01 uptake demonstrated good or excellent inter-rater agreement. T:BP (ICC 0.85; 95%CI 0.73–0.93), LN:BP (ICC 0.90; 95% CI 0.85–0.94) and DisMet:BP (ICC 0.88; 95% CI 0.77–0.94) all demonstrated good inter-rater agreement. ThMet:BP (ICC 0.94; 95% CI 0.81–0.98) demonstrated excellent inter-rater agreement. Bland–Altman plot analysis demonstrated inter-rater agreement with no proportional bias on linear regression for T:BP scores (Fig. 2) and DisMet:BP scores (Additional file 2: Fig. S1). However, one-sample t-test of LN:BP scores demonstrated statistical difference between the means of rater A versus B (p < 0.05) and rater B versus C (p < 0.05). The Bland–Altman analysis plots and associated β coefficients for LN:BP scores are included (Fig. 2). There was no statistical significance, and as such, acceptable agreement between the means of rater A versus C (p = 0.04) for LN:BP scores; however, there was proportional bias on linear regression (β = 0.20, p < 0.05). Inter-rater agreement was demonstrated for ThMet:BP scores; however, with proportional bias for rater B versus C (β = 0.34, p < 0.05; Additional file 2: Fig. S1).

Fig. 2
figure 2

Inter-rater Bland–Altman level of agreement plots for T:BP (a–c) and log10 LN:BP (d–f) scores. Solid horizontal lines represent between-rater mean difference. Upper and lower 95% limits of agreement are represented by dashed lines. a T:BP scores of rater A versus B (t-test p = 0.55; β = -0.06, p = 0.51); b T:BP scores rater A versus C (p = 0.46; β = 0.05, p = 0.74); c T:BP scores rater B versus C (p = 0.18; β = 0.11, p = 0.34); d LN:BP scores rater A versus B (p < 0.05; β = 0.51, p < 0.05); e LN:BP scores rater A versus C (p = 0.44; β = 0.20, p < 0.05); f LN:BP scores rater B versus C (p < 0.05; β = 0.22, p < 0.05)

Full size image

Intra-rater agreement

Freehand SUVmax scoring using [99mTc]NM-01 SPECT/CT following a 28-day interval of all malignant lesions including primary lung tumour, lymph node, thoracic and distant metastases was consistent for all 3 raters (Table 3). The intra-rater ICC for primary lung tumour (T; n = 25) SUVmax measurements for rater A (ICC 1.00; 95% CI 0.99–1.00), rater B (ICC 1.00; 95% CI 1.00–1.00), and rater C (ICC 1.00; 95% CI 1.00–1.00) demonstrated excellent agreement. SUVmax measurements for lymph node (n = 56; ICCs 0.99–1.00), thoracic (n = 9; ICCs 0.97–0.98) and distant (n = 21; ICCs 0.94–0.99) metastases demonstrated excellent intra-rater agreement for all raters (see Table 3 for individual intra-rater ICCs and their 95% CI).

Table 3 Intra-rater agreement of [99mTc]NM-01 measurements
Full size table

Scoring of healthy reference tissue blood pool (n = 25) demonstrated good or excellent intra-rater agreement for raters A, B and C (ICCs 0.90, 0.98, 0.97 respectively). The intra-rater ICC for freehand SUVmax scores of healthy lung tissue (n = 25; ICCs 0.90, 0.86, 1.00) were good or excellent. Applying a 3-cm-diameter sphere volumetric rule-based approach at the level of the aortic arch (ICCs 0.96, 0.96, 0.99) or the carina (ICCs 0.98, 0.94, 0.98) demonstrated excellent intra-rater agreement for all raters. Calculated T:L ratios using freehand and volumetric spheres at the aortic arch or the carina all demonstrated either good or excellent intra-rater agreement for all three raters (Table 3).

There was excellent intra-rater agreement of manual freehand SUVmax scores of normal liver (n = 25; ICCs 0.94, 0.99, 0.99), spleen (n = 25; ICCs 0.91, 0.99, 0.97) and bone marrow (n = 25; ICCs 0.91, 0.99, 0.90) for all raters. Applying the rule-based volumetric approach to measuring the SUVmax of the liver (ICCs 0.90, 0.98, 0.96), spleen (ICCs 0.95, 0.96, 0.97) and bone marrow (ICCs 0.91, 0.98, 0.91) demonstrated good or excellent agreement. Spleen-to-liver ratio (SLR) measured using freehand (ICCs 0.94, 0.96, 0.99) and volumetric (ICCs 0.84, 0.93, 0.96) approaches both demonstrated good or excellent intra-rater agreement of SUVmax scores for raters A, B and C. Similarly, bone marrow-to-liver ratios measured using freehand (ICCs 0.86, 0.97, 0.90) and volumetric (ICCs 0.77, 0.98, 0.90) approaches demonstrated good or excellent intra-rater agreement for all raters.

Good or excellent intra-rater agreement was demonstrated for T:BP measurements for rater A (ICC 0.86; 95% CI 0.71–0.94), rater B (ICC 0.97; 95% CI 0.92–0.98), and rater C (ICC 0.99; 95% CI 0.97–0.99). LN:BP measurements demonstrated excellent intra-rater agreement for all three raters (ICCs 0.94, 0.98, 0.98). Thoracic metastasis-to-blood pool (ThMet:BP; ICCs 0.95, 0.95, 0.99) and distant metastasis-to-blood pool (DisMet:BP; ICCs 0.89, 0.96, 0.98) ratios also demonstrated good or excellent intra-rater agreement for each rater. Bland–Altman plot analysis demonstrated intra-rater agreement with no proportional bias for T:BP scores for raters A (β coefficient = 0.11, p = 0.36) and B (β = − 0.03, p = 0.64) (Fig. 3). However, one-sample t test of T:BP scores demonstrated statistical difference between the means of rater C time-point 1 and time-point 2 (p < 0.05). Comparison of the means of the separate time-point LN:BP scores were statistically significant for each rater (p-values < 0.05). Bland–Altman analysis for ThMet:BP scores for raters A (β = 0.2, p < 0.05) and C (β = 0.11, p < 0.05) demonstrated intra-rater agreement but with proportional bias (Additional file 3: Fig. S2). A t-test comparison for rater B demonstrated significantly different ThMet:BP scores between time-point 1 and 2 (p < 0.05). Bland–Altman analysis demonstrated intra-rater agreement for all three raters for DisMet:BP scores; however, there was proportional bias on linear regression for rater B (β = − 0.15, p < 0.05).

Fig. 3
figure 3

Intra-rater Bland–Altman level of agreement plots for T:BP (a–c) and log10 LN:BP (d–f) scores. Solid horizontal lines represent between-timepoints mean difference. Upper and lower 95% limits of agreement represented by dashed lines. a T:BP scores rater A, time 1 versus time 2 (t-test p = 0.35; β = 0.11, p = 0.36); b T:BP scores rater B, time 1 versus time 2 (p = 0.3; β = -0.03, p = 0.64); c T:BP scores rater C, time 1 versus time 2 (p < 0.05; β = -0.01, p = 0.71); d LN:BP scores rater A, time 1 versus time 2 (p < 0.05; β = 0.04, p = 0.35); e LN:BP scores rater B, time 1 versus time 2 (p < 0.05; β = 0.04, p = 0.07); f LN:BP scores rater C, time 1 versus time 2 (p < 0.05; β = -0.09, p < 0.05)

Full size image

Discussion

In this study, we demonstrate that quantitative assessment of [99mTc]NM-01 SPECT/CT is reliable and reproducible within and between independent raters of variable experience in nuclear medicine. Inter-rater agreement was demonstrated for all malignant lesions (primary lung, lymph node, thoracic and distant metastases) using xSPECT BroadQuantification assessed SUVmax scores or their equivalent malignant lesion-to-blood pool ratios of [99mTc]NM-01 SPECT/CT. Additionally, intra-rater agreement was also demonstrated for malignant lesion SUVmax scores and their ratios. Measurements of normal tissue using freehand or volumetric approaches demonstrated good or excellent inter- and intra-rater agreement, suggesting simplified and standardised manual measurement techniques (volumetric) could be utilised without negatively impacting the reproducibility of results between and within individual raters and over time. [99mTc]NM-01 uptake measured by T:BP on SPECT/CT has already been shown to correlate with PD-L1 expression measured by immunohistochemistry (r = 0.68, p = 0.014) [11]. Importantly, that study also documented both inter- and intra-tumoural heterogeneity of uptake. This study’s findings provide further evidence that [99mTc]NM-01 SPECT/CT has the potential for reliably quantifying and as such, clinical utility as a diagnostic agent for PD-L1 assessment. PD-L1 expression is both dynamic and heterogenous, therefore, non-invasive assessment is an attractive possibility, with the potential to better stratify patients for anti-PD-1/PD-L1 therapies, as well as to determine changes of expression in response to therapy and heterogeneity of expression in responding versus non-responding lesions.

Several other radiopharmaceuticals are under development for imaging the PD-1/PD-L1 axis as potential predictive and/or prognostic biomarkers [15]. Uptake of 18Fluorine-labelled anti-PD-L1 Adnectin (18F-BMS-986192) measured using SUVpeak on positron emission tomography (PET)/CT correlated with PD-L1 expression ≥ 50% by immunohistochemistry in NSCLC [16, 17]. Drug-labelled 89Zirconium-Atezolizumab, including participants with NSCLC, bladder and breast cancers, demonstrated better prediction of clinical response using mean SUVmax on PET compared with either SP263 or SP142 PD-L1 immunohistochemical assays [18]. In both studies, along with several other novel radiopharmaceuticals in early phase clinical trials, both inter- and intra-tumoural heterogeneity of uptake (i.e. PD-L1 expression) was demonstrated (consistent with findings reported in the early phase study of [99mTc]NM-01 SPECT/CT) [11].

In the majority of studies, PET/CT imaging approaches have been taken, with benefits including greater spatial resolution and standardised quantitative assessment. Drug-labelled radiopharmaceuticals offer additional information about drug distribution and are site-binding with the potential of theranostic applications. However, optimal imaging time is usually in the region of days post-administration due to the large size of monoclonal antibodies and longitudinal assessment may be hampered by therapeutic drug binding site occupancy. [99mTc]NM-01 is a small (14.3 kDa) antigen-binding fragment radiopharmaceutical with rapid blood clearance, and optimal SPECT/CT imaging at just 2 h following administration [11, 19]. Pre-clinical studies have demonstrated that it does not directly block the PD-L1 binding site nor interfere with the PD-1/PD-L1 axis, and therefore, it has the potential to assess whole-body PD-L1 status before, during and after anti-PD-1/PD-L1 therapy [19]. The ongoing PECan (NCT04436406) study involves [99mTc]NM-01 SPECT/CT at both baseline and following anti-PD-1/PD-L1 therapy, and aims to demonstrate this in vivo. Whilst tumour-to-background ratios appear to be relatively lower than published examples using PET monoclonal antibody tracers, likely due to a partial sink effect, increasing the nanobody dose from 100 to 400 µg was not associated with any significant differences in tumour to background in the phase 1 study [11]. Despite this, the inter- and intra-rater agreement remains good to excellent.

There are additional benefits to SPECT/CT imaging with both [99mTc] radioisotope and SPECT being widely available and relatively inexpensive. Here we have demonstrated that a quantifiable SPECT/CT approach produces reproducible and reliable [99mTc]NM-01 uptake measurements in malignant NSCLC lesions, as well as in healthy reference tissues. Simple rule-based approaches to measuring healthy reference tissues also demonstrated acceptable inter- and intra-rater agreement, suggesting these methods may be used clinically to standardise and simplify image analysis without any negative impact on quantification.

This study has some limitations, including its sample size; nevertheless, the relatively narrow confidence intervals suggest a reasonable estimate of the agreement. The number of measurable extra-nodal thoracic (lung or pleural) metastases was limited (n = 9, all time points) and, as such, the inter- and intra-rater ICCs, although excellent, should be interpreted with more caution. Semi-quantitative SPECT using CT attenuation correction is a relatively novel methodology compared to PET/CT and is likely to be less accurate and more subject to partial volume effects. Nevertheless, with validation, quantifiable SPECT/CT offers the potential for comparison of [99mTc]NM-01 SPECT/CT PD-L1 assessment with alternative PD-L1 PET radiopharmaceuticals.

Conclusion

Overall, good or excellent inter- and intra-rater agreement of the quantitative assessment of [99mTc]NM-01 SPECT/CT in NSCLC primary and metastatic lesions was demonstrated in this study. As such, there is potential for quantifiable [99mTc]NM-01 SPECT/CT assessment of PD-L1 expression and its inter- and intra-tumoural heterogeneity. With further understanding of the relationship between PD-L1 expression by immunohistochemistry and by [99mTc]NM-01 SPECT/CT, it may also be possible for both quantitative (as described in this study) and qualitative assessments to be made by raters blind to immunohistochemical PD-L1 expression, and their agreement evaluated. Ongoing and subsequent clinical trials are warranted to confirm the utility of [99mTc]NM-01 SPECT/CT in clinical practice.

Availability of data and materials

The datasets used and analysed during the current study are available from the corresponding author on reasonable request. Reuse is permitted where aggregate data will advance research in this field, for example, inter-nation or cross-specialty studies of this kind.

Abbreviations

BP:

Blood pool

BLR:

Bone marrow-to-liver ratio

CI:

Confidence interval

CT:

Computed tomography

DisMet:

Distant metastasis

DisMet:BP:

Distant metastasis-to-blood pool ratio

ECOG:

Eastern Cooperative Oncology Group

GOJ:

Gastroesophageal junction

ICC:

Intraclass correlation coefficient

IHC:

Immunohistochemistry

kDa:

Kilodalton

L:

Lung

LN:

Lymph node

LN:BP:

Lymph node-to-blood pool ratio

NM-01:

A single-domain antibody targeting PD-L1

NSCLC:

Non-small cell lung cancer

PD-1:

Programmed cell death protein 1

PD-L1:

Programmed death-ligand 1

PET:

Positron emission tomography

ROImax :

Maximum region of interest

SLR:

Spleen-to-liver ratio

SPECT:

Single-photon emission compute tomography

SUVmax :

Maximum standardised uptake value

SUVmean :

Mean standardised uptake value

T:

Tumour

T:BP:

Primary tumour-to-blood pool ratio

[99mTc]:

Technetium-99m

ThMet:

Thoracic metastasis

ThMet:BP:

Thoracic metastasis-to-blood pool ratio

TPS:

Tumour proportion score

References

  1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71(3):209–49. https://doi.org/10.3322/caac.21660.

    Article  PubMed  Google Scholar 

  2. Reck M, Rodríguez-Abreu D, Robinson AG, Hui R, Csőszi T, Fülöp A, et al. Updated analysis of KEYNOTE-024: pembrolizumab versus platinum-based chemotherapy for advanced non-small-cell lung cancer with PD-L1 tumor proportion score of 50% or greater. J Clin Oncol. 2019;37(7):537–46. https://doi.org/10.1200/jco.18.00149.

    Article  CAS  PubMed  Google Scholar 

  3. Rittmeyer A, Barlesi F, Waterkamp D, Park K, Ciardiello F, Von Pawel J, et al. Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): a phase 3, open-label, multicentre randomised controlled trial. Lancet. 2017;389(10066):255–65. https://doi.org/10.1016/s0140-6736(16)32517-x.

    Article  PubMed  Google Scholar 

  4. Vokes EE, Ready N, Felip E, Horn L, Burgio MA, Antonia SJ, et al. Nivolumab versus docetaxel in previously treated advanced non-small-cell lung cancer (CheckMate 017 and CheckMate 057): 3-year update and outcomes in patients with liver metastases. Ann Oncol Off J Eur Soc Med Oncol. 2018;29(4):959–65. https://doi.org/10.1093/annonc/mdy041.

    Article  CAS  Google Scholar 

  5. Fehrenbacher L, Spira A, Ballinger M, Kowanetz M, Vansteenkiste J, Mazieres J, et al. Atezolizumab versus docetaxel for patients with previously treated non-small-cell lung cancer (POPLAR): a multicentre, open-label, phase 2 randomised controlled trial. Lancet. 2016;387(10030):1837–46. https://doi.org/10.1016/s0140-6736(16)00587-0.

    Article  CAS  PubMed  Google Scholar 

  6. de Castro G, JKudaba I Jr, Wu YL, Lopes G, Kowalski DM, Turna HZ, et al. Five-year outcomes with pembrolizumab versus chemotherapy as first-line therapy in patients with non-small-cell lung cancer and programmed death ligand-1 tumor proportion score ≥ 1% in the KEYNOTE-042 study. J Clin Oncol. 2022. https://doi.org/10.1200/JCO.21.02885.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Herbst RS, Baas P, Kim D-W, Felip E, Pérez-Gracia JL, Han J-Y, et al. Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial. Lancet. 2016;387(10027):1540–50. https://doi.org/10.1016/S0140-6736(15)01281-7.

    Article  CAS  PubMed  Google Scholar 

  8. Gadgeel S, Rodríguez-Abreu D, Speranza G, Esteban E, Felip E, Dómine M, et al. Updated analysis from KEYNOTE-189: pembrolizumab or placebo plus pemetrexed and platinum for previously untreated metastatic nonsquamous non-small-cell lung cancer. J Clin Oncol. 2020;38(14):1505–17. https://doi.org/10.1200/JCO.19.03136.

    Article  CAS  PubMed  Google Scholar 

  9. McLaughlin J, Han G, Schalper KA, Carvajal-Hausdorf D, Pelekanou V, Rehman J, et al. Quantitative assessment of the heterogeneity of PD-L1 expression in non-small-cell lung cancer. JAMA Oncol. 2016;2(1):46–54. https://doi.org/10.1001/jamaoncol.2015.3638.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Kim T-J, Hong SA, Kim O, Kim SJ, Yang J-H, Joung EK, et al. Changes in PD-L1 expression according to tumor infiltrating lymphocytes of acquired EGFR-TKI resistant EGFR-mutant non-small-cell lung cancer. Oncotarget. 2017. https://doi.org/10.18632/oncotarget.22582.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Xing Y, Chand G, Liu C, Cook GJR, O’Doherty J, Zhao L, et al. Early phase I study of a 99mTc-labeled anti-programmed death ligand-1 (PD-L1) single-domain antibody in SPECT/CT assessment of PD-L1 expression in non-small cell lung cancer. J Nucl Med. 2019;60(9):1213–20. https://doi.org/10.2967/jnumed.118.224170.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Vija AH. Introduction to xSPECT technology: evolving multi-modal SPECT to become context-based and quantitative. In: Vija AH (ed) Molecular imaging white paper: siemens medical solutions USA, Inc., molecular imaging. 2013.

  13. Adams MC, Turkington TG, Wilson JM, Wong TZ. A systematic review of the factors affecting accuracy of SUV measurements. Am J Roentgenol. 2010;195:310–20. https://doi.org/10.2214/AJR.10.4923.

    Article  Google Scholar 

  14. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63. https://doi.org/10.1016/j.jcm.2016.02.012.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Hughes DJ, Subesinghe M, Taylor B, Bille A, Spicer J, Papa S, et al. 18F FDG PET/CT and novel molecular imaging for directing immunotherapy in cancer. Radiology. 2022;304(2):246–64. https://doi.org/10.1148/radiol.212481.

    Article  PubMed  Google Scholar 

  16. Niemeijer AN, Oprea-Lager DE, Huisman MC, Hoekstra OS, Boellaard R, de Wit-van der Veen BJ, et al. Study of 89Zr-pembrolizumab PET/CT in patients with advanced-stage non-small cell lung cancer. J Nucl Med. 2022;63(3):362–7. https://doi.org/10.2967/jnumed.121.261926.

    Article  CAS  PubMed  Google Scholar 

  17. Huisman MC, Niemeijer AN, Windhorst AD, Schuit RC, Leung D, Hayes W, et al. Quantification of PD-L1 expression with 18F-BMS-986192 PET/CT in patients with advanced-stage non-small cell lung cancer. J Nucl Med. 2020;61(10):1455–60. https://doi.org/10.2967/jnumed.119.240895.

    Article  PubMed  Google Scholar 

  18. Bensch F, van der Veen EL, Lub-de Hooge MN, Jorritsma-Smit A, Boellaard R, Kok IC, et al. 89Zr-atezolizumab imaging as a non-invasive approach to assess clinical response to PD-L1 blockade in cancer. Nat Med. 2018;24(12):1852–8. https://doi.org/10.1038/s41591-018-0255-8.

    Article  CAS  PubMed  Google Scholar 

  19. Wong NC, Cai Y, Meszaros LK, Biersack HJ, Cook GJ, Ting HH, Mottaghy FM. Preclinical development and characterization of 99mTc-NM-01 for SPECT/CT imaging of human PD-L1. Am J Nucl Med Mol Imaging. 2021;11(3):154–66.

    CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all the participants involved in this study. Additionally, the authors acknowledge colleagues at King’s College London, Guy’s and St Thomas’ PET Centre and the Cancer Centre at Guy’s, London, UK, for their contributions.

Funding

This research was supported by NanoMab Technology Limited, London, UK. The authors acknowledge financial support from the Cancer Research UK National Cancer Imaging Translational Accelerator (C1519/A28682), and the Wellcome/Engineering and Physical Sciences Research Council Centre for Medical Engineering at King’s College London (WT 203148/Z/16/Z). For the purpose of open access, authors have applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.

Author information

Authors and Affiliations

  1. Department of Cancer Imaging, School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK

    Daniel Johnathan Hughes, Gitasha Chand, Vicky Goh & Gary J. R. Cook

  2. King’s College London and Guy’s and St. Thomas’ PET Centre, Lambeth Wing, St Thomas’ Hospital, Westminster Bridge Road, London, SE1 7EH, UK

    Daniel Johnathan Hughes & Gary J. R. Cook

  3. Department of Nuclear Medicine, Guy’s and St Thomas’ NHS Foundation Trust, London, UK

    Jessica Johnson, Damion Bailey & Kathryn Adamson

  4. Department of Radiology, Guy’s and St. Thomas’ NHS Foundation Trust, London, UK

    Vicky Goh

Authors
  1. Daniel Johnathan Hughes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gitasha Chand
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jessica Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Damion Bailey
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kathryn Adamson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vicky Goh
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gary J. R. Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

DJH, GCh, JJ, DB, KA, VG and GJRC were responsible for study concept and design, data analysis and interpretation, writing, reviewing and editing the manuscript. DJH, GCh and GJRC were responsible for rating. DJH and GJRC made the final decision to approve and submit the final manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Gary J. R. Cook.

Ethics declarations

Ethics approval and consent to participate

This study was conducted in accordance with the Declaration of Helsinki and underwent Research Ethics Committee and Health Research Authority, United Kingdom approval (IRAS 256684 and IRAS 290974). Written informed consent was provided by all participants.

Consent for publication

Not applicable.

Competing interests

DJH has received honoraria/speaker fees from Pfizer, travel/accommodation grants from Bristol-Myers Squibb, Roche and NanoMab Technology, research funding via institute from NanoMab Technology, and is an executive committee member of the Association of Cancer Physicians (UK). GCh is a director and shareholder of NanoMab Technology (UK) Ltd. GJRC has research support via institute from NanoMab Technology, Theragnostics, Serac Healthcare and provides consultancy for GE Healthcare, NanoMab Technology, Amgen and Full-Life Technologies. All other authors report no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1.

Participant demographics.

Additional file 2: Fig. S1.

Inter-rater Bland–Altman level of agreement plots for ThMet:BPand log10 DisMet:BPscores. Solid horizontal lines represent between-timepoints mean difference. Upper and lower 95% limits of agreement represented by dashed lines. a ThMet:BP scores of rater A versus B; b ThMet:BP scores of rater A versus C; c ThMet:BP scores of rater B versus C; d DisMet:BP scores of rater A versus B; e DisMet:BP scores of rater A versus C; f DisMet:BP scores of rater B versus C.

Additional file 3: Fig. S2.

Intra-rater Bland–Altman level of agreement plots for ThMet:BPand log10 DisMet:BPscores. Solid horizontal lines represent between-timepoints mean difference. Upper and lower 95% limits of agreement represented by dashed lines. a ThMet:BP scores rater A, time 1 versus time 2; b ThMet:BP scores rater B, time 1 versus time 2; c ThMet:BP scores rater C, time 1 versus time 2; d DisMet:BP scores rater A, time 1 versus time 2; e DisMet:BP scores rater B, time 1 versus time 2; f DisMet:BP scores rater C, time 1 versus time 2.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hughes, D.J., Chand, G., Johnson, J. et al. Inter-rater and intra-rater agreement of [99mTc]-labelled NM-01, a single-domain programmed death-ligand 1 (PD-L1) antibody, using quantitative SPECT/CT in non-small cell lung cancer. EJNMMI Res 13, 51 (2023). https://doi.org/10.1186/s13550-023-01002-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13550-023-01002-4

Keywords