Kinfitr — an open-source tool for reproducible PET modelling: validation and evaluation of test-retest reliability

Background In positron emission tomography (PET) imaging, binding is typically estimated by fitting pharmacokinetic models to the series of measurements of radioactivity in the target tissue following intravenous injection of a radioligand. However, there are multiple different models to choose from and numerous analytical decisions that must be made when modelling PET data. Therefore, it is important that analysis tools be adapted to the specific circumstances, and that analyses be documented in a transparent manner. Kinfitr, written in the open-source programming language R, is a tool developed for flexible and reproducible kinetic modelling of PET data, i.e. performing all steps using code which can be publicly shared in analysis notebooks. In this study, we compared outcomes obtained using kinfitr with those obtained using PMOD: a widely used commercial tool. Results Using previously collected test-retest data obtained with four different radioligands, a total of six different kinetic models were fitted to time-activity curves derived from different brain regions. We observed good correspondence between the two kinetic modelling tools both for binding estimates and for microparameters. Likewise, no substantial differences were observed in the test-retest reliability estimates between the two tools. Conclusions In summary, we showed excellent agreement between the open-source R package kinfitr, and the widely used commercial application PMOD. We, therefore, conclude that kinfitr is a valid and reliable tool for kinetic modelling of PET data.

PMOD is therefore designed in such a way that fitting a t* value for each PET examination is made very convenient. If one were to wish to use a single t* value across multiple individuals based on the results from everyone, this would take more time and effort. One would sequentially load in the data for each subject individually, fit the model and t* value, and write down the different t* values obtained. Afterward, one could evaluate these values and decide on an appropriate value for t*, and then sequentially load in the data for each subject once more, manually enter the t* value, and fit the model again using this value. For this reason, we suggest that the user interface encourages the use of individual t* values for each measurement, as this approach is much more convenient.
kinfitr Interface and Auditability In kinfitr, the user interacts with the software through R code. We made use of R notebooks, by which writing, code and code outputs are all interspersed with one another (3). These analysis notebooks are shared online (https://github.com/tjerkaskij/agreement_kinfitr_pmod). All user actions are therefore represented in the analysis code and can be checked and audited directly. This is one of the central benefits of computational reproducibility.
One of the other proposed benefits of computational reproducibility is that code can be recycled and repurposed from project to project, or even between analysts. An example of this in practice is that the code within the analysis notebooks for [ 11 C]AZ10419369 and [ 11 C]SCH23390 is nearly identical and could, therefore, be copied from one to the other analysis and modified as necessary.

Weighting schemes
We used the default weighting scheme from kinfitr. This method defines weights as the square root of the product of the frame durations and the non-decay-corrected time-activity curve of a large region. The resulting values are then taken as a proportion of the range between 0.7 and 1 to restrict their range. The whole brain TAC was used for the calculation of weights. Weights were applied for all models in kinfitr apart from the Logan and noninvasive Logan models, owing to their making use of transformed values for the predicted value. For these cases, uniform weights were used.

t* Values
In contrast to PMOD, the selection of t* is not automated in kinfitr. Rather, the user is presented with R 2 values, maximum percentage residuals and changes in binding estimates for each potential value of t*, for three different regions of interest for each PET measurement, recommended to consist of regions with high, medium and low binding (see figure below). In this way, the user must select an appropriate value for t* by examining all of the available information. It is further explicitly recommended in the documentation that these visual aids should be examined for several individuals/measurements to select a single t* value for the entire cohort.
Graphical aids provided by kinfitr for the selection of an appropriate t* value.
In kinfitr, all data is loaded at once, and so producing all t* selection figures for several or all measurements in a cohort is straightforward. As such, kinfitr makes it more convenient to examine these t* visual aids and to make a judgement about the most appropriate value for a t*, across the whole cohort. If, instead, one were to wish to select an individual value for t* for each individual, this would take a great deal more time and effort. One would have to examine the t* aids one-by-one, and write down the selected t* value for each, add the selected t* values to the data, and then re-run the modelling by pointing the model to the newly created t* column for which value to use. Instead, this was a deliberate and opinionated design decision to avoid the potential for overfitting the selection of t* values, and to present the user with as much information as possible to make as informed a choice as possible. For this reason, the user interface encourages the use of a single t* values for a whole cohort, as this approach is much more convenient, and is furthermore explicitly encouraged.

Comparison of BPND values calculated by kinfitr and PMOD.
The relationship between binding estimates calculated by either kinfitr or PMOD All results were derived from the frontal cortex region. The diagonal line represents the line of identity. Each colour corresponds to a different subject, and the dotted lines connect both measurements from the same subject.

Comparison of VT values calculated by kinfitr and PMOD.
The relationship between binding estimates calculated by either kinfitr or PMOD All results were derived from the thalamus region. The diagonal line represents the line of identity. Each colour corresponds to a different subject, and the dotted lines connect both measurements from the same subject.
Agreement between kinfitr and PMOD using fitted or selected t* values for the non-invasive models We also performed an analysis using PMOD weights and t* values in kinfitr for MA1 and MRTM2. We did not perform this analysis using the Logan methods, as these models were fitted using uniform weights in kinfitr (see Supplementary Materials S2). Additionally, 2TCM and SRTM were also run in kinfitr using the constant weights.
Test-retest reliability of kinfitr and PMOD for the non-invasive models using the higherbinding ROI