On 2014 Sep 13, Hilda Bastian commented:

The authors make an important point: just because a systematic review has not assessed publication bias (PB), it does not mean that there is none.

However, in this study, there were only 36 reviews that did not assess publication bias, and nearly half of those were in the minority subset of reviews that didn't have a comprehensive search strategy. For many, a comprehensive search strategy is a defining characteristic of a systematic review (e.g. in DARE, the Database of Reviews of Effects). Those reviews may not be able to provide an adequate overview of published studies, either.

The authors point out that a limitation of their study is that there were many (planned) subgroup analysis - and it's on a small number of reviews. Especially as the number of adequately systematic reviews was small, the exclusion of the Cochrane Database of Systematic Reviews - a journal that publishes systematic reviews and was eligible for their study - is disappointing. The reason given for the exclusion was because the results of the (Moher D, 2007) study showed that publication bias "is regularly performed in articles published in this database." However the authors of that study concluded the assessment of publication bias was disappointing overall. For Cochrane reviews in that study, publication bias was assessed (or intended to be assessed) in only 32% of those reviews (and it was considered in another 39%).

(Disclosure: I work on projects related to systematic reviews at the NCBI (National Center for Biotechnology Information, U.S. National Library of Medicine.)

On 2014 Sep 13, Hilda Bastian commented:

The authors make an important point: just because a systematic review has not assessed publication bias (PB), it does not mean that there is none.

However, in this study, there were only 36 reviews that did not assess publication bias, and nearly half of those were in the minority subset of reviews that didn't have a comprehensive search strategy. For many, a comprehensive search strategy is a defining characteristic of a systematic review (e.g. in DARE, the Database of Reviews of Effects). Those reviews may not be able to provide an adequate overview of published studies, either.

The authors point out that a limitation of their study is that there were many (planned) subgroup analysis - and it's on a small number of reviews. Especially as the number of adequately systematic reviews was small, the exclusion of the Cochrane Database of Systematic Reviews - a journal that publishes systematic reviews and was eligible for their study - is disappointing. The reason given for the exclusion was because the results of the (Moher D, 2007) study showed that publication bias "is regularly performed in articles published in this database." However the authors of that study concluded the assessment of publication bias was disappointing overall. For Cochrane reviews in that study, publication bias was assessed (or intended to be assessed) in only 32% of those reviews (and it was considered in another 39%).

(Disclosure: I work on projects related to systematic reviews at the NCBI (National Center for Biotechnology Information, U.S. National Library of Medicine.)