As the leader of a new primary care group, you notice in your Health Authority’s annual report a steady rise in rates of death from skin cancer, perhaps because people are spending more holidays in sunnier climes. As most forms of skin cancer are preventable if detected early, you wonder whether your GPs have sufficient diagnostic skills for what used to be a rare condition, but which now has a prevalence (“pretest probability”) of about 5%. You formulate the question: “In patients with a skin lesion, how accurate are general physicians at diagnosing skin cancer?”
You open Best Evidence, enter “skin cancer” and click on the diagnosis limit button, and find the abstract for a study comparing the accuracy of clinical examinations conducted by GPs with those of dermatologists. You review the abstract and decide you’d like to read the complete article. Arch Intern Med 1997;157:985-990.
Read the article and decide:
- Are the results of this diagnostic article valid?
- Are the valid results of this diagnostic study important?
- Can you apply this valid, important evidence about a diagnostic test in caring for your patient or population?
Completed Diagnosis Worksheet for Evidence-Based Purchasing
Citation
Whited JD, Hall RP, Simel DL, Horner RD. Primary care clinicians’ performance for detecting actinic keratoses and skin cancer.
Arch Intern Med 1997;157:985-990.
Are the results of this diagnostic study valid?
- Was there an independent, blind comparison with a reference (“gold”) standard of diagnosis?
- Yes – of GP examinations versus those of dermatologists – considered a “pragmatic” gold standard. In a second stage, the classic gold standard, biopsy, was performed only on lesions thought malignant by the dermatologists.
- Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom it would be used in practice)?
- Probably not. The prevalence in this group is quite high for British patients at 32%. Patients were recruited from dermatology and general internal medical out patients. Although neither group had been seen by a dermatologist for 4 months, and those from medical outpatients did not necessarily have a known skin condition, the dermatology patients did.
- Was the reference standard applied regardless of the diagnostic test result?
- Yes- when the dermatologist examination was the reference standard. All patients were examined both by a GP and a dermatologist.
No – when the reference standard was biopsy, to compare with the dermatologist examination. Biopsies were only performed on lesions thought to be malignant by the dermatologist.
Are the valid results of this diagnostic study important?
Your calculations
| Target Disorder (skin cancer, according to dermatologist exam) |
||||
|---|---|---|---|---|
| Present | Absent | |||
| Diagnostic Test Result (GP exam) |
Positive | 35 a |
15 b |
50 a + b |
| Negative | 26 c |
114 d |
140 c + d |
|
| Totals | a + c 61 |
b + d 129 |
190 |
|
begin{align}
mathit{Sensitivity} &= a/(a+c)\
&= 35/61\
&= 57%
end{align}
begin{align}
mathit{Specificity} &= d/(b+d)\
&= 114/129\
&= 88%
end{align}
begin{align}
text{Likelihood Ratio for a positive test result} &= mathit{LR+}\
&= mathit{sens}/(1-mathit{spec})\
&= 57%/12%\
&= 4.9 text{ (rounding off at the final stage)}
end{align}
begin{align}
text{Likelihood Ratio for a negative test result} &= mathit{LR-}\
&= (1-mathit{sens})/mathit{spec}\
&= 43%/88%\
&= 0.48 text{ (rounding off at the final stage)}
end{align}
begin{align}
text{Positive Predictive Value} &= a/(a+b)\
&= 35/50\
&= 70%
end{align}
begin{align}
text{Negative Predictive Value} &= d/(c+d)\
&= 114/140\
&= 81%
end{align}
begin{align}
text{Pre-test Probability ($prevalence$)} &= (a+c)/(a+b+c+d)\
&= 61/190\
&= 32%
end{align}
begin{align}
mathit{Pre-test-odds} &= mathit{prevalence}/(1-mathit{prevalence})\
&= 32%/68%\
&= 0.47
end{align}
begin{align}
text{Post-test odds} &= text{Pre-test odds} times text{Likelihood Ratio for a positive result}\
&= 0.47 times 4.9\
&= 2.3
end{align}
begin{align}
text{Post-test Probability} &= text{Post-test odds}/(text{Post-test odds} + 1)\
&= 2.3/3.3\
&= 0.70
end{align}
Can you apply this valid, important evidence about a diagnostic test in caring for your patient?
- Is the diagnostic test available, affordable, accurate, and precise in your setting?
- It is available and affordable. The GPs’ lack of accuracy and precision would probably be worrying for most GPs and patients.
- Can you generate a clinically sensible estimate of your patient’s pre-test probability (from practice data, from personal experience, from the report itself, or from clinical speculation)?
- Based upon Health Authority data, the population’s pre-test probability is known to be about 5%.
- Will the resulting post-test probabilities affect your management and help your patient? (Could it move you across a test-treatment threshold?; Would your patient be a willing partner in carrying it out?)
- Using the nomogram and figures from the study, a 5% pre-test probability will yield a post-test probability of a positive result of about 23%.
- Would the consequences of the test help your patient?
- Yes – when the GP identifies a problem. However, the low sensitivity and prevalence means that a negative result is not so reassuring. Hence, GPs may need more training, or new ways of identifying skin cancer investigated.
Additional Notes
The paper answered the question “how good are GPs versus dermatologists at diagnosing skin cancer.” It does not answer the question “how good are GPs or dermatologists versus biopsy at diagnosing skin cancer,” as biopsies were only performed on lesions thought to be malignant by dermatologists. The study might have been strengthened by performing biopsies on all lesions, in order to use a real “gold standard” against GP performance, rather than these authors’ “pragmatic gold standard.”
Skin Cancer: General Practitioner examinations are not sufficient for identifying
Purchasing Bottom Line
GP examinations for skin cancer have high specificity but low sensitivity, so are good for reliably referring patients with positive results, but should not be the sole means of identification.
Citation
Whited JD, Hall RP, Simel DL, Horner RD. Primary care clinicians’ performance for detecting actinic keratoses and skin cancer.
Arch Intern Med 1997;157:985-990.
Purchasing/Policy Question
In patients with skin lesions, what is the accuracy of the general practitioner’s diagnosis of skin cancer? Should Primary Care Groups employ alternative means to routine examination for detecting skin cancer?
Search Terms
Best Evidence search using “skin cancer” and “diagnosis.” For MEDLINE search, use skin cancer AND diagnosis AND exp sensitivity and sensitivity.
The Study
Blinded comparison of GP versus dermatologist examinations for skin cancer, single actinic keratoses and multiple actinic keratoses in 190 men recruited from dermatology and general medicine outpatient clinics. The reference standard was dermatologist opinion. Each patient was independently examined by a GP and a dermatologist. Interobserver agreement of patients and of lesions was also measured.
The Evidence
| Target Disorder (skin cancer, according to dermatologist exam) |
||||
|---|---|---|---|---|
| Present | Absent | |||
| Diagnostic Test Result (GP exam) |
Positive | 35 a |
15 b |
50 a + b |
| Negative | 26 c |
114 d |
140 c + d |
|
| Totals | a + c 61 |
b + d 129 |
190 |
|
begin{align}
mathit{Sensitivity} &= a/(a+c)\
&= 35/61\
&= 57%
end{align}
begin{align}
mathit{Specificity} &= d/(b+d)\
&= 114/129\
&= 88%
end{align}
begin{align}
text{Likelihood Ratio for a positive test result} &= mathit{LR+}\
&= mathit{sens}/(1-mathit{spec})\
&= 4.9 text{ (rounding off at the final stage)}
end{align}
begin{align}
text{Likelihood Ratio for a negative test result} &= mathit{LR-}\
&= (1-mathit{sens})/mathit{spec}\
&= 43%/88%\
&= 0.48 text{ (rounding off at the final stage)}
end{align}
begin{align}
text{Positive Predictive Value} &= a/(a+b)\
&= 35/50\
&= 70%
end{align}
begin{align}
text{Negative Predictive Value} &= d/(c+d)\
&= 114/140\
&= 81%
end{align}
begin{align}
text{Pre-test Probability ($prevalence$)} &= (a+c)/(a+b+c+d)\
&= 61/190\
&= 32%
end{align}
begin{align}
mathit{Pre-test-odds} &= mathit{prevalence}/(1-mathit{prevalence})\
&= 32%/68%\
&= 0.47
end{align}
begin{align}
text{Post-test odds} &= text{Pre-test odds} times text{Likelihood Ratio for a positive result}\
&= 0.47 times 4.9\
&= 2.3
end{align}
begin{align}
text{Post-test Probability} &= text{Post-test odds}/(text{Post-test odds} + 1)\
&= 2.3/3.3\
&=0.70
end{align}
Comments
The paper compares GP with dermatologist exams for diagnosing skin cancer. It does not compare GPs or dermatologists reliably against the gold standard of biopsy.
Appraised By
Anna Donald and Sam Vincent
