Editorial Note: How Do We Evaluate Your Manuscript?
Jaya, E.S. (2020). Editorial Note: How Do We Evaluate Your Manuscript?. Makara Human Behavior Studies In Asia, 24(2), 99-100. DOI:10.7454/hubs.asia.1311220
|Edo S. Jaya||- Faculty of Psychology, Universitas Indonesia, Depok, Indonesia
The criteria for evaluating manuscript submissions differ between journals. Different journals have different aims and scopes. Here we describe our criteria for the initial selection of your manuscript which are sent to reviewers. Rejection of a manuscript is uncomfortable for the authors and also, for us. Unfortunately, now we have to reject approximately 50% of manuscript submissions because of our precious reviewers’ time. Furthermore, to help you understand why rejection happens and why we have these criteria, we provide an overview of our situation.
2. Reasons for Accepting your Manuscript
The problems of increasingly metric-based evaluation of science are widely recognized (Onie, 2020). This has also influenced Makara Human Behavioral Studies in Asia. Currently, this journal is open-access, and there is no fee incurred for submission and reading. We receive funding from Universitas Indonesia’s internal grant for our operation year to year. However, this is public fund and understandably, our editorial team needs to provide a strong basis for why we should receive that funding to our funding body. This is where metric evaluation comes into play.
In Indonesia and also many other developing nations, science is increasingly metric-based. Researchers receive different salary bonus for publishing in certain journals. This increases pressure for our journal to select manuscripts that satisfies the requirement of the indexes.
There are quantitative and qualitative criteria related to these indexing programs. If the authors keep these criteria in mind while submitting to our journal, there could be a greater likelihood of acceptance.
First, the authors should think about the citability of the manuscript. Many indexing metrics depend on citations, and thus editors are pressured to publish manuscripts that are likely to gain citations in the near future. Beside the quality of the manuscript, diversity of authors coming from different countries has been shown to be a good predictor of citability of manuscripts (Larivière et al., 2015). Second, the authors should reflect on the potential of media or social media interaction with the manuscript. Sometimes people forget that science is a conversation, and that a scientific journal as its instrument is a form of mass media. Similar to other mass media, we do look for engagement too. Third, now qualitatively, the manuscript should be of importance to the discipline. Assessing importance is difficult. We look at conceptual novelty, methodological novelty, and/or policy-related importance. We also look at whether the data reported is difficult to obtain or not. Conceptual novelty can be observed in the form of new theoretical approach or an interesting discovery in the discipline. Methodological novelty can be identified in the form of advanced statistical analysis, advanced instruments (e.g. using Magnetic Resonance Imaging), or advanced methods (e.g. strict experiment, longitudinal data, experience-sampling).
However, we are aware that not all good science can be measured in such a way. For example, a paper published in 1993 showing the relationship between intelligence and listening to Mozart (Rauscher et al., 1993) gained substantial media attention and citations, which created the term “Mozart effect”. After the publication of the study and media interest, there have been many attempts to replicate the results, but no evidence of the Mozart effect was found in nearly 40 studies with over 3000 participants (Pietschnig et al., 2010).
3. Reasons for Rejecting your Manuscript
Before deciding to send a manuscript to reviewers, we also screen for potential problems. Here are some of the main reasons of rejection.
First, the manuscript is not related to our aims and scope. We publish empirical studies on human behavior. Sometimes we have to reject a submission because it is not an empirical study, such as a literature review or text analysis. We also have rejected submissions because they are not related to human behavior and are perhaps better published in other specialized journals. For example, we had to reject a quality paper on a text or literature analysis of a narration, which is not considered human behavior.
Second, we identify problems related to methods in the manuscript. The statistical analyses could be inappropriate for the aim of the study, effect sizes may not be reported and interpreted, and/or data bias such as outliers may not be considered. The design of the study may sometimes be too weak for the conclusion. Some examples include using the term “effectiveness” of an intervention in a randomized-controlled efficacy trial or using the term “efficacy” with a design that is not a randomized-controlled trial. More importantly, we also look for potential copyright issues. Authors who use questionnaires or scales in their study may not always be aware of copyright issues and unintentionally infringe on the copyright of the owner of scales.
Third, we often reject a manuscript because of a lack of novelty. For example, authors may conduct a study on correlates of psychological well-being in a university student sample from a given city. There have been numerous studies on the same topic. While we are aware of the WEIRD (White, Educated, Industrialized, Rich, Democratic) bias of human behavior literature (Henrich et al., 2010), the authors are required to explain why the findings could be the same or different from previous studies given the context of their sample. More often than we like, the authors do not provide explanation.
To summarize, here are some questions that the authors can ask themselves before submitting to Makara Human Behavior Studies in Asia: 1) Who will cite this manuscript? 2) Will this manuscript gain media or social media attention? 3) Is there conceptual novelty? 4) Is there methodological novelty? 5) Is it of relevance policy-wise? In addition, here are some points that should be avoided when submitting the manuscript. If the answer is no to any of the following questions, the submission will most likely be rejected: 1) Is the variable of the study on human behavior? 2) Is it an empirical study? 3) If there is statistical analysis, is it rigorous? 4) Is the method strict? 5) Does the conclusion match the method? 6) Do you have permission to use the questionnaire or scale for your study? 7) Is there novelty? You can use the Research in Context section to argue for novelty, citability, and general importance of your submission.
Dr. phil. Edo Sebastian Jaya, M. Psi.
Makara Human Behavior Studies in Asia