When I first embarked on my journey towards a doctorate, I was way out of my depth to read most mainstream scientific articles. A lot of your run-of-the-mill papers would often employ scientific jargon unique to their field, use complex statistical methods well beyond anything I knew, and discuss theories that seem inaccessible to mortals. I was a noob back then.

In those days, I would turn to systematic reviews to get up to speed on the latest findings in the field. For those who don’t know, a systematic review is a kind of scientific article that seeks to summarize and synthesize the findings from a field of study, in hopes of generating new ideas or insights built on previous works. Unlike your run-of-the-mill article, a review typically has no analyses or experimental design section, making it easier for the average reader to enter the field by taking in the broad findings from past works.

That was nearly a decade ago.

Fast forward till today, I have my doctorate behind me and can engage with my area of research much more competently (or at least, I would like to believe so). These days, I have the opposite problem from my undergraduate times – I hate reading and writing systematic reviews. A good review often requires one to engage with the field for years, often decades to garner that level of experience and wisdom. Too often, reviews these days are often viewed as a cheap ticket for entry-level researchers to score their first publication, subsequently resulting in a surge of low-quality, redundant reviews that often rehash old ideas but contribute little else.

Are there really more systematic reviews floating around these days?

Don’t take my word for it. This phenomenon has been a growing plaque in the field of medicine, where researchers face more reviews of clinical trials than trials itself. In the best-case scenario, reviews and meta-analyses can be valuable studies by themselves – often serving to find a broad consensus or a hidden trend where there is one yet to be discovered. In the worst-case scenario, a poorly conducted review is a waste of hours of effort sieving through the literature, as well as the mental effort of all readers and reviewers involved. 

In my own experience on ResearchGate and other academic platforms these days, I have noticed a substantial increase in the number of meta-analyses, reviews and studies proposing new “frameworks” recommended to me recently, much to my chagrin. I’ve complained in a previous post about how our obsession with novelty is forcing researchers to put out increasing volumes of studies containing untested ideas and “novel” frameworks that lead to nowhere. If I had to pick a data paper versus another sexy-sounding review, you can bet I’ll pick the former any day of the week, no matter how simple or boring that data may be.

This leads me to today’s topic – do we have a review crisis in ecology these days? Or is it just me and my algorithm?

A review of reviews in ecology

To objectively answer whether the share of review papers in the scientific literature have increased these days, I consulted the WebOfScience database for publication statistics in the top ecology journals with IF =>3.0 in 2024 (see this list). Subsequently, I collated the number of total publications and published reviews of each journal per year into a spreadsheet manually and used R to compute the share of published literature taken by reviews. Here’s what I found:

Figure 1: Proportion of publications in top ecology journals that are labelled as reviews. Red lines in the top graph represent journals with an increasing share of publications as reviews (slope > 0.5 by simple linear regression), and vice versa for blue lines (slope < -0.5). Number of publications and reviews have been aggregated into a single value per year and reported in the bottom figure. Each number above a point represents the number of review articles published across all the top ecology journals for that year.

Right away, I noticed that several journals have quietly shifted their publishing habits — putting out a noticeably higher share of review articles relative to their total annual output. In contrast, many journals have stuck to a fairly consistent share of review articles annually, with no clearly discernible trend across the past 11 years. However, after summing all the articles and reviews produced by the 53 journals represented in the above graph (the exceptions being Current Opinions in Insect Science and Annual Review of Ecology, Evolution and Systematics, both of which produce exclusively reviews), the pattern becomes much clearer. Journals, on average, are indeed producing a greater share of review articles in recent years. In particular, two journals have >30% of their annual output labelled as reviews (Trends in Ecology and Evolution and Frontiers in Ecology and the Environment), though this does not negate the fact that over half of all journals in this list (34 journals) have increased their share of reviews in their annual output, albeit slightly. Very broadly, the share of reviews relative to the total publication output across these top journals have increased by 15-20% across the past decade or so.

5% of all publications may not seem to amount to much. Yet, when one focuses on highly cited publications alone, reviews are vastly overrepresented in the cited literature (~25%) given their modest share in the total ecological literature (Figure 2). This isn’t a surprise given that it is well-known that reviews command far more attention from academics compared to run-of-the-mill studies and articles. Yet, I wasn’t prepared to see a five-fold increase in review share. No wonder I keep getting reviews recommended to me these days…  

Figure 2: Share of reviews among highly cited publications aggregated from top ecology journals. Numbers represent the number of reviews produced that year that have been highly cited.

Surprisingly, the big 4 journals in academic publishing (Nature, Science, PNAS and Cell) do not seem to obey this trend (see Figure 3). In contrast, the share of reviews in these journals has remained consistent (or slightly decreased) across the past 11 years. What this means is that the growing share of reviews in ecology seen in Figure 1 is driven not by an influx of reviews published in prestigious journals, but in more mid-to-lower tier ecology journals instead.

Figure 3: Share of reviews among publications from the Big 4 journals – Nature, Science, PNAS and Cell. No significant trends found here.

Finally, I got curious and wondered: perhaps the reason why I have been seeing so many reviews recommended to me is because the journals I frequented most have been pushing out more reviews? Subsequently, I compiled the same statistics for the top 20 journals I frequented most (based on the list in my Mendeley software). This time, journals seemed to publish reviews at roughly the same rate every year (the exception being Mycorrhiza, which has increasing its proportion of reviews published, and vice versa for Frontiers in Plant Science). Overall, there has been a net drop in the share of reviews published by the journals I frequented most (Figure 4). Maybe it is just me…

Figure 4: Among the top 20 journals I frequented most, I did not find a significant trend in share of reviews published over the past decade. When taken together, the proportion of reviews relative to the overall publication output of these journals seem to be decreasing instead.

Finally, I narrowed down my search to focus solely on highly cited publications. If you thought the above overrepresentation of reviews was bad, try 45%.

Figure 5: Around 35-50% of the most cited publications from the journals I frequented most were reviews. No wonder I got recommended so many reviews these days…

Conclusion

Despite all the data I’ve shown so far, I believe the true number of reviews out there is much greater than what was compiled here. For one, journals often have different policies regarding what counts as a review article and what doesn’t, and this can easily skew the numbers either way. It is not hard to imagine that there exist far more framework/perspective/recommendation pieces out there listed as articles but are essentially glorified reviews of past work that add little to the literature.

Is the slow but growing share of reviews in the literature a problem? I think so. After all, good systematic reviews should follow only after years of extensive experience in a field, and thus by definition they should be rare. Yet, because reviews require no funding for experiments and often command greater citation rates than most mainline studies, they are often an attractive “low-hanging fruit” for new researchers to pursue. Furthermore, AI has further expedited this process, making it easier than ever to summarize findings from a large number of publications at once to produce a systematic review of questionable value.

Years from now, this could mean we may face a bombardment of systematic reviews, framework pieces, recommendation letters and other similar articles, similar to what the medical field has been experiencing now. In a field like ecology where primary data is already so scarce in certain fields, ecologists may spend more time talking about past results rather than generating new ones instead. When taken to an extreme, our science will never progress beyond reframing old ideas and rehashing existing biases.

What do you think? Am I being a little too cynical towards systematic reviews? Let me know if you have a different perspective!