Meta‐analyses of the published literature are increasingly being used, allowing similar clinical trials to be combined quantitatively, thereby increasing the precision of the estimation of treatment effect. Four meta‐analyses were recently published comparing the impact of dialysis membranes or dialysis modality on clinical outcomes of patients with acute renal failure (ARF) requiring renal replacement therapy. Two studies compared dialysis membranes in intermittent hemodialysis (IHD) and two studies compared continuous renal replacement therapy (CRRT) to IHD. The findings between each pair of meta‐analyses were discordant. This was due in part to the differences among meta‐analyses in inclusion and exclusion criteria, the paucity of randomized controlled trials, variation in individual study quality, and heterogeneity in the study populations and settings. Understanding these issues is important to properly interpret results from these meta‐analyses.