Systematic Review Scenario

You are the practice manager in an inner city practice. The practice health visitor is about to retire, and you are considering whether to reallocate the resources to other services within the practice. The health visitor claims that she plays an important role in accident prevention and shows you the attached systematic review as evidence. You are aware that 5% of your childhood population (<5yrs) suffer from accidental injuries. What recommendations do you make to the practice management committee report about whether to continue to employ a health visitor in the practice? In children can home visits decrease the risk of injury?

Search strategy for Health Visitors
Type in:
1. explode CHILD/ all subheadings
2. VISIT*
3. SYSTEMATIC*
4. REVIEW*
5. OVERVIEW*
6. REVIEW* OR OVERVIEW*
7. #2 and (#3 and #6)

You find the article: Roberts, I., Kramer, M.S., and Suissa, S. Does home visiting prevent childhood injury? BMJ 312:29-33, 1996.

Read the article and decide:

  • Is the evidence from this systematic review valid?
  • Is this valid evidence from this systematic review important?
  • Can you apply this valid and important evidence from this systematic review in caring for your patient?

Completed Systematic Reviews Worksheet for Child Health

Clinical Question

In children under 5 years, does a health visitor (compared to no health visitor) prevent accidental injury?

Are the results of this systematic review of therapy valid?

Did the review address a clearly focused question?
Yes.
How likely is it that the search strategy would have missed eligible trials?
Unlikely to have missed eligible published trials.
Are the inclusion criteria clearly stated?
Yes.
Are the inclusion criteria relating to population, intervention and comparison groups and outcome appropriate?
Yes. However, criteria includes both intentional and unintentional injury as outcomes, and there is broad mix of populations and interventions.
How likely is it that the conclusions are valid (ie. Are the included studies good quality randomised controlled trials?)?
Likely. Conclusions not altered by limiting analysis to trials with high quality scores.
If a meta-analysis was performed, were the included studies sufficiently homogeneous to make it appropriate to pool data?
Yes for unintentional injury. Heterogeneity noted in studies of intentional injury.

Are the valid results of this systematic review important?

Translating odds ratios to NNTs. The numbers in the body of the table are the NNTs for the corresponding odds ratios at that particular patient’s expected event rate (PEER).
Odds Ratios
0.9 0.85 0.8 0.75 0.7 0.65 0.6 0.55 0.5
Patient’s Expected Event Rate (PEER) 0.05 2091 139 104 83 69 59 52 46 412
0.10 110 73 54 43 36 31 27 24 21
0.20 61 40 30 24 20 17 14 13 11
0.30 46 30 22 18 14 12 10 9 8
0.40 40 26 19 15 12 10 9 8 7
0.50 * 3 25 18 14 11 9 8 7 6
0.70 44 28 20 16 13 10 9 7 6
0.90 1014 64 46 34 27 22 18 15 125
Odds Ratios
1.1 1.2 1.3 1.4 1.5
Patient’s Expected Event Rate (PEER) 0.05 212 106 71 54 43
0.10 112 57 38 29 23
0.20 64 33 22 17 14
0.30 49 25 17 13 11
0.40 43 23 16 12 10
0.50 42 22 15 12 10
0.70 51 27 19 15 13
0.90 121 66 47 38 32

The numbers in the body of the table are the NNTs for the corresponding Odds Ratios at that particular patient’s expected event rate (PEER). This table applies both when a good outcome is increased by therapy and when a side-effect is caused by therapy.

Results: For unintentional injuries, pooled odds ratio is < 1 which favours intervention; ie. there are less unintentional injuries in children whose mother was visited at home than in those not visited

Pooled OR = 0.74 with 95% CI of 0.6 to 0.92. As CI does not include 1, result is statistically significant.

Can you apply this valid, important evidence from a systematic review in caring for your patient?

Do these results apply to your patient?
Are my patients so different from those in the review that there are likely to be important differences in treatment effect?
Maybe. Studies mainly involved ‘high risk’ families.
Is the intervention in the studies in the review sufficiently similar to the treatment that I am considering?
Maybe. Most studies involved non-professional visitors.
Are the outcome measures documented an adequate reflection of the outcomes of importance to my patients?
Yes.

How great would the potential benefit of therapy actually be for your individual patient?

  1. Method 1: In the table on page 1, find the intersection of the closest odds ratio from the overview and the CER that is closest to your patient’s expected event rate if they received the control treatment (PEER).

    OR = 0.74
    Assume PEER for injuries is only 5%
    NNT = 83

  2. Method 2: To calculate the NNT for any OR and PEER:

    NNT = [1 - (PEER x (1 - OR))]/[(1 - PEER) x PEER x (1 - OR)]
    Your PEER is 0.05, so with OR of 0.74, NNT = 80

Are your patient’s values and preferences satisfied by the regimen and its consequences?
Do your patient and you have a clear assessment of their values and preferences?
Yes
Are they met by this regimen and its consequences?
Not entirely. Need information on intensity of visiting and effect and acceptability of professional compared with non-professional visitors.
Should you believe apparent qualitative differences in the efficacy of therapy in some subgroups of patients? Only if you can say “yes” to all of the following:
Do they really make biologic and clinical sense?
Yes, unintentional injuries are different from intentional injuries.
Is the qualitative difference both clinically (beneficial for some but useless or harmful for others) and statistically significant?
Yes
Was this difference hypothesised before the study began (rather than the product of dredging the data), and has it been confirmed in other, independent studies?
Yes
Was this one of just a few subgroup analyses carried out in this study?
Yes

Additional Notes

Despite the ill-defined intervention (i.e. professional and non-professional visitors) and the crude outcome measures (some parent recall, some hospital attendance, there is a statistically significant result which suggests that home visiting has a real impact on children’s unintentional injuries.

Childhood Injuries – Reduced by home visit

Clinical Bottom Line

Home visiting can reduce the number of unintentional.

Citation

Roberts, I., Kramer, M.S., and Suissa, S. Does home visiting prevent childhood injury? BMJ 312:29-33, 1996

Clinical Question

In children, can home visiting reduce the injury rate?

Search Terms

‘home visiting’ and ‘child’ and ‘injury’ and ‘randomised-controlled-trial’

The Study

Systematic review of 11 RCTs including 3433 participants.

The Evidence

Pooled OR of home visiting Vs ‘control’ for unintentional injuries is 0.74 (95% CI 0.6 to 0.92).

If baseline risk for unintentional injuries is 5%, NNT is 80.

If baseline risk for unintentional injuries is 15%, NNT is 29.

Comments

  • The populations in all trials were disadvantaged.
  • Interventions varied: visits were by non-professionals and occurred during and/or after pregnancy and at different intensities.
  • Outcomes varied from hospital attendance to parent recall of injuries.
  • The review on intentional injuries is difficult to interpret as reporting of injuries was confounded by increased detection in the home visiting groups.
  1. The relative risk reduction(RRR) here is 10% 
  2. The RRR here is49% 
  3. For any OR, NNT is lowest when PEER =.50 
  4. The RRR here is 1% 
  5. The RRR here is9%