Computer-aided diagnosis of renal obstruction: utility of log-linear modeling versus standard ROC and kappa analysis

EJNMMI Research

Table 3 Pairwise agreement within experts and between experts and RENEX

	Log-linear model coefficients
	Between experts			RENEX and expert
	E ₁ E ₂	E ₁ E ₃	E ₂ E ₃	*R E* ₁	*R E* ₂	R E ₃
	[ δ _m1 , (SE)]	[ δ _{m 2} , (SE)]	[ δ _{m 3} , (SE)]	[ θ _{m 1} , (SE)]	[ θ _{m 2} , (SE)]	[ θ _{m 3} , (SE)]
Non-obstructed	1.58* (0.42)	0.36 (0.40)	0.91 (0.46)	0.26 (0.40)	1.08* (0.41)	0.84* (0.32)
Equivocal	-0.12 (0.40)	0.89 (0.37)	-0.58 (0.44)	-0.06 (0.37)	-0.28 (0.39)	0.47 (0.33)
Obstructed	0.78 (0.46)	-0.07 (0.43)	1.47* (0.45)	0.48 (0.43)	1.08 (0.44)	0.41 (0.38)
	p values for tests of hypothesis
	Among experts		RENEX and expert		Experts vs. RENEX
	H ₀: δ _{m 1}= δ _{m 2}= δ _{m 3} ^a		H ₀: θ _{m 1}= θ _{m 2}= θ _{m 3} ^b		H ₀: δ _{m 1}+ δ _{m 2}+ δ _{m 3}= θ _{m 1}+ θ _{m 2}+ θ _{m 3} ^c
Non-obstructed	0.15		0.47		0.41
Equivocal	0.08		0.34		0.95
Obstructed	0.11		0.58		0.81

* Significant positive pairwise agreement at α = 0.01. ^aHypothesis that the overall pairwise agreement among experts is the same. The δ s reflect the strength of the beyond-chance agreement between two raters within each specific category. ^bHypothesis that the overall pairwise agreement between RENEX and each expert is the same. The θ s reflect the strength of the beyond-chance agreement between RENEX and an expert within each specific category. ^cHypothesis that the overall pairwise agreement among experts is the same as the overall pairwise agreement between each expert and RENEX.