Systematic Review Scenario

You are supervising the purchase of equipment for the Accident and Emergency Department in a large, inner city hospital. You hear from one of the clinicians that crystalloid fluids (salt water) may be as effective as colloid (protein based) fluids, and perhaps safer for resuscitating patients in shock – something that happens a lot in A&E. You formulate the question: “In critically ill patients is fluid resuscitation with crystalloid solutions as effective as that with colloid solutions?”

You wonder whether there is any validity in this claim, particularly as colloid fluids cost nearly 8 times as much as crystalloid fluids, and the Department badly needs money for more nurses. You do a brief search on the Cochrane Library within the Systematic Review database for “colloid” AND “crystalloid“. Up comes a systematic review that looks useful (Schierhout G, Roberts I, Alderson P. Colloids compared to crystalloids in fluid resuscitation of critically ill patients (Cochrane Review). In: The Cochrane Library, 4, 1998. Oxford: Update Software.) You later note that this review is also available from the electronic BMJ website under the slightly different title, Fluid resuscitation with colloid or crystalloid solutions in critically ill patients: a systematic review of randomised trials, BMJ 1998;316:961-4.

Read the article and decide:

  • Is the evidence from this systematic review valid?
  • Is this valid evidence from this systematic review important?
  • Can you apply this valid and important evidence from this systematic review in caring for your patient or population?

Completed Systematic Reviews Worksheet for Evidence-Based Purchasing


Schierhout G, Roberts I, Alderson P. Colloids compared to crystalloids in fluid resuscitation of critically ill patients (Cochrane Review). In: The Cochrane Library, 4, 1998. Oxford: Update Software.

Are the results of this systematic review of therapy valid?

Is it a systematic review of randomised trials of the treatment you are interested in?
Yes. It compared the overall effects of colloid versus crystalloid fluids in a range of typical resuscitation situations. (NB it may be helpful to point out that “crystalloid” fluid is usually salt water [also known as normal saline]!)
Does it include a methods section that describes finding and including all the relevant trials?
Yes, they captured 48 apparently randomised trials from a wide search of the Cochrane Controlled Trials Register, MEDLINE, EMBASE and BIDS ISTP; by handsearching 29 journals and proceedings of international meetings; by checking reference lists and by contacting authors for information on any other published or unpublished trials. 37 of the 48 identified met the inclusion criteria. 19 of these were used to assess the outcome of death.
Does it include a methods section that describes assessing their individual validity?
Yes; inclusion criteria were randomised or quasi-randomised controlled trials; method of randomisation was checked with author and quality of concealment rated.
Were the results consistent from study to study?
Looking at the odds ratio table (see Summary of Analyses metaview in the Cochrane Library version, or page 964 in the BMJ), the results look fairly similar, in that there are not statistically significant differences between or within patient groups.

Are the valid results of this systematic review important?

Translating odds ratios to NNTs. The numbers in the body of the table are the NNTs for the corresponding odds ratios (or relative risks) at that patient’s expected event rate (PEER).
Odds Ratios (OR)
0.9 0.85 0.8 0.75 0.7 0.65 0.6 0.55 0.5
Patient’s Expected Event Rate (PEER) 0.05 2091 139 104 83 69 59 52 46 412
0.10 110 73 54 43 36 31 27 24 21
0.20 61 40 30 24 20 17 14 13 11
0.30 46 30 22 18 14 12 10 9 8
0.40 40 26 19 15 12 10 9 8 7
0.503 38 25 18 14 11 9 8 7 6
0.70 44 28 20 16 13 10 9 7 6
0.90 1014 64 46 34 27 22 18 15 125
Relative Ratios (RR)
0.9 0.85 0.8 0.75 0.7 0.65 0.6 0.55 0.5
Control Event Rate (CER) .05 200 133 100 80 67 57 50 44 40
.10 100 67 50 40 33 29 25 22 20
.20 50 33 25 20 17 14 13 11 10
.30 33 22 17 13 11 10 8 7 7
.40 25 17 13 10 8 7 6 6 5
0.5 20 13 10 8 7 6 5 4 4
.70 14 10 7 6 5 4 4 3 3
.90 11 7 6 4 4 3 3 2 2

Can you apply this valid, important evidence from a systematic review in caring for your patient?

Do these results apply to your patients?
Yes – the results suggest that crystalloids are probably at least as good as colloids for general use.
Is your patient so different from those in the systematic review that its results can’t help you?
You could argue from these results that surgical patients might do worse on crystalloids (although the results were not significant, with the total number of patients small [191 in 7 trials] and number of deaths even smaller [9])
How great would the potential benefit of therapy actually be for your individual patient?
Method I: In the tables on page 1, find the intersection of the closest odds ratio or relative risk from the overview and the CER that is closest to your patient’s expected event rate if they received the control treatment (PEER):
It is hard to calculate the weighted NNH from the numbers presented. Still, if you take the summary statistics of increased risk of death from colloids of 4%, then the NNH is 1/0.04 = 25. That is, if 25 people use colloids instead of crystalloids, one extra person dies. Alternatively, to turn this into an NNT, you could say that to avoid one death, you need 25 people to take crystalloids instead of colloids.
Method II: To calculate the NNT for any RR and CER:
$$mathit{NNT} = frac{1}{mathit{CER}(mathit{RR}-1)}$$
  • Note that it is possible to do these calculations on any of the EER/CER presented in the Summary of Analysis (the RR table in the BMJ version).
Method II: To calculate the NNT for any OR and PEER:
$$mathit{NNT} = frac{1-[mathit{PEER}times(1-mathit{OR})]}{(1-mathit{PEER}) times mathit{PEER}times(1-mathit{OR})}$$
Are your patient’s values and preferences satisfied by the regimen and its consequences?
Do your patient and you have a clear assessment of their values and preferences?
Would need to check with patients/clients/population, but it is unlikely in this case that people – and their families – would be opposed to surviving resuscitation efforts!
Are they met by this regimen and its consequences?
Should you believe apparent qualitative differences in the efficacy of therapy in some subgroups of patients?

Only if you can say “yes” to all of the following:

Do they really make biologic and clinical sense?
Not really (surgery patients).
Is the qualitative difference both clinically (beneficial for some but useless or harmful for others) and statistically significant?
Was this difference hypothesised before the study began (rather than the product of dredging the data), and has it been confirmed in other, independent studies?
Was this one of just a few subgroup analyses carried out in this study?

Additional Notes

Critically ill patients – Crystalloids are safer than colloids

Purchasing Bottom Line

Use of colloids was associated with an increase of absolute risk of mortality of 4%.


Schierhout G, Roberts I, Alderson P. Colloids compared to crystalloids in fluid resuscitation of critically ill patients (Cochrane Review). In: The Cochrane Library, 4, 1998. Oxford: Update Software.

Purchasing Question

In routine resuscitation of critically ill patients, are colloids or crystalloids more effective?

Search Terms

“colloids” AND “crystalloids” and “resuscitation” in The Cochrane Library, MEDLINE.

The Study

Systematic review of 37 RCTs with a total of 1622 patients. Mortality data derived from 19 of the 37 RCTs with 1315 patients.

The Evidence

AR(colloids) AR(crystalloids) RR
ARR NNH with colloids (or treat with crystalloids to avoid death)
Total deaths (all patients) 168/685
= 0.25
= 0.19
1.31 0.058 17
Deaths from trauma 82/335
= 0.24
= 0.15
1.60 0.09 11
Deaths from surgery* 3/105
= 0.029
= 0.07
0.41 -0.04 25 (NNH with crystalloids)
Deaths from burns 60/207
= 0.29
= 0.24
1.20 0.05 20
The evidence (from weighted data in relative risk table and results section)
Total deaths (all patients) 1.19 (0.98-1.45) 0.04 (0.0-0.08) 25 (13 to infinity)


  • On first glance, it appears that surgical patients do better with colloids, but it should be noted that this result is not significant, and based on a small number of patients (117) and deaths (9).
  • It might be noted that the overall relative risk of mortality from colloids vs. crystalloids is not statistically significant (the confidence intervals cross the 1.0 line). However, this probably does not invalidate the main message from the study that crystalloids are at least as safe if not safer than colloids, and an awful lot cheaper for general use in Accident and Emergency!

Appraised By

Anna Donald and Sam Vincent.


  1. The relative risk reduction (RRR) here is 10% 
  2. The RRR here is 49% 
  3. For any OR, NNT is lowest when PEER = .50 
  4. The RRR here is 1% 
  5. The RRR here is 9%