File-drawer analysis is a method used to measure the likelihood that published studies are only a subset of many studies conducted on the same topic. It is based on the premise that studies with statistically significant results are more likely to be published than those with statistically insignificant results, creating a file-drawer effect. This phenomenon has been studied extensively since the 1950s and is a key concern in meta-analysis, an analysis of multiple studies. The file-drawer analysis is used to estimate the number of unpublished studies to make the results of a meta-analysis more accurate and reliable.
The file-drawer analysis is based on the method of Rosenthal (1979), which uses the number of published studies and the magnitude of their effects to estimate the number of unpublished studies. The formula is:
N = K/R2
where N is the estimated number of unpublished studies, K is the number of published studies, and R2 is the combined effect size. The effect size can be calculated by squaring the difference between the observed proportion of statistically significant results and the expected proportion of statistically significant results. For example, if the observed proportion of statistically significant results is 0.90 and the expected proportion of statistically significant results is 0.50, the effect size would be 0.40.
When conducting a file-drawer analysis, it is important to be aware of potential biases. For example, studies may be more likely to be published if they have larger sample sizes or more impressive results. Additionally, studies may be more likely to be published if they are conducted in a certain language or by a certain researcher. To account for these biases, it is important to consider the context of the studies being analyzed.
In conclusion, file-drawer analysis is an important tool to consider when conducting a meta-analysis. It can help to estimate the number of unpublished studies, reducing potential bias and making the results of a meta-analysis more reliable and accurate.
References
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638-641.
Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). (2009). The handbook of research synthesis and meta-analysis (2nd ed.). New York, NY: Russell Sage Foundation.