Re: New Metabolomics Society Journal - Your Input Needed!
Reply #13 –
Just a quick comment on Jan's wishlist - why double blind review? Why not consider the other way and have open review - to get back to #transparency :-) As a reviewer I appreciate knowing who the work comes from, it is often obvious (but not always and guesswork is not a great idea). If I want to discuss the work with authors beyond the review, I would like the opportunity to do so; I have also considered identifying myself in some reviews, my husband always does and receives very positive feedback. I feel this way is more constructive than double blind, but that's my personal opinion.
Because I think it is important to avoid bias also in the review process.
Of course I cannot know if there is a real problem with that but I am personally convinced that high profile researchers often get away with things other researchers might not (and that they shouldn't); lack of openness about methods, "trust me" kind of replies to review points, borderline appropriate papers for the journal etc. At least I have seen what I considered such kind of biases.
There might also be a bias from the reviewer being more likely to accept unclear methods and speculative conclusions if they already trust the authors. Also gender bias could be an issue.
Metabolomics is a small community so I would find it prudent to avoid favoritism, unconscious or not. This could also make it easier to find reviewers without conflicts in a narrow field.
I can see that generally people are moving to open reviews but it is my personal gut feeling, without much evidence, that it does not lead to better reviews. With open reviews you might tend to make more thorough reviews, but perhaps less likely to shoot down papers that really is not worth the time of a thorough review in the state it was submitted.
I am not sure "positive feedback" to reviews is really a metric worth aiming for...
I do accept that I could be completely wrong about this idea.
I just found this old paper on the subject that suggest there is some effect of blind reviews: http://www.unifr.ch/wipol/assets/files/PhD%20Course/Blank1991.pdf