Submissions

Author Guidelines

Foreword

This page provides guidance on how to prepare a submission for ReScience X, including the types of submissions we distinguish, key aspects and material that must be included in submissions, and the structure we expect submissions and supplementary material to follow.

ReScience X considers original empirical research from all disciplines, spanning natural sciences, life sciences, social sciences, humanities and engineering. Authors working on reproducing results from published computational work are invited to submit their work to our sister journal, ReScience C.

The aim of the review process is to provide actionable feedback to authors, and contribute to the improvement of the work, until it meets ReScience quality standards. We focus on quality, and reviewers are instructed not to base their evaluation on trends, novelty, or on the perceived importance of the results submitted or that of the work reproduced.

Authors planning a submission are strongly advised to contact the ReScience X editorial team, and to liaise with the original authors, if possible, before starting work on their project. We encourage collegiate and fruitful collaboration, and hope these early interactions can strengthen the proposed work. If the original authors are not involved in the project, ReScience X editors will invite them to comment on the finalised article, upon acceptance for publication.

ReScience X is organised in Sections, and our submission portal follows the OJS submission workflow. Authors are advised to go through the OJS documentation, and can even take a short online training module, if needed.

We follow an opt-out Open Peer-Review scheme, where reviewers are identified and reviews are made public, unless agreed with the editor. Should authors feel the need to submit their work anonymously, they are invited to contact the ReScience X editorial team as soon as possible. Upon acceptance for publication, editors can append a commentary, or a summary of the topic, and the authors of the original work may be invited to comment.

Submissions are made by creating an account on the submission portal, and must include a Cover Letter and an original Research Article, using either the ReScience docx or the TeX/overleaf templates, and submitted as both source (Open Office, Microsoft Word, LaTeX) and PDF files.

Submissions cannot be under review in another academic venue at the same time, but authors can disclose their work as they see fit, e.g. on social media, as a preprint, in a blog post, or at a conference. Authors retain copyright on any work submitted, and a license must be assigned prior to acceptance for publication. CC-BY 4 is preferred. Authors grant full rights to ReScience X to publish, distribute and promote their work.

Upon acceptance, submissions will be made available on the ReScience X website, indexed in relevant search engines, and advertised on social media.

Submissions

Cover Letter

A cover letter must accompany a submission, using the ReScience X Template, and submitted as a PDF on the submission portal.

The cover letter must include the following information:

• the full names and affiliations of all authors, who must all have agreed to the submission;

• the email address of the corresponding author; this information is displayed on every submission, and all communication between the corresponding author and ReScience X Editorial Board is done through the submission portal.
• the full reference to the original paper, including DOI if available;
• a brief summary (500w max) of the submission;
• a URL to an OSF project, and to its registration if it has been performed (preferred). This OSF project should be used to centralise access to all files, material, documents, and links to online repositories used in the work submitted;
• a list of at least 3 potential reviewers, with contact information and a statement , and if applicable a list of at most 3 people who should not be contacted for review.

Original Research Article, and supplementary material

An original Research Article must be using the ReScience X Template, and submitted as a PDF file on the submission portal. It should follow the guidelines and conventions in the academic field of the original paper.

The associated OSF project should centralise all supplementary material, raw txt/tex/docx/odf/etc files, and rendered PDF files, as well as links to online repositories hosting the material not on OSF; e.g. source code, curated repositories. Once accepted for publication, a submission will be copy-edited and relevant links added to the finalised PDF article, before indexation.

A README document should contain a description of the structure of the OSF project.

All supplementary material should be consolidated and curated in the accompanying OSF project, according to FAIR principles (Wilkinson et al., 2016), and include:

• the raw data, clearly labelled, for every single experiment;
• the final, processed (normalised) data, clearly annotated, for every single experiment;
• the complete list of material, hardware and software used, from collection to analysis, including version numbers and URLs to vendors and manufacturers, as well as calibration scheme and data;
• if possible, the actual stimuli/input and scripts used to run the experiments;
• the complete list of essential laboratory and data processing methods and protocols;
• all the scripts used to analyse the data and produce the figures included in the Research Article, and in the supplementary material.

Types of submissions

Reproductions of methods -- "The method is the same, results are, hopefully, consistent"

We refer to the definition of reproducibility proposed by Peng (2011): the purpose of a reproduction is to assert the outcome of a particular experimental setup, to assert that a given method achieves consistent results.

In other words, a reproduction is an account of the faithful application of the original work; if at all possible, that means the same setup, the same scripts, the same software, the same stimuli and apparatus, the same kind of participants or subjects, the same chemical compounds, the same kind of equipment and facilities, the same analyses, etc. This may be more easily done in some disciplines than others. This would include, for instance, the measurement of chemical reactions, with raw material that will be of the exact same composition as in the original experiment.

The Research Article should include, and will be assessed on the following items. Any discrepancy with the original paper should be stated explicitly:

• a brief summary of the original paper, and contextual information;
• a description of the aim of the reproduction, clearly specifying what aspect of the method is being reproduced and to what end;
• a detailed description of the method under scrutiny: authors must share the material used, scripts, blueprints, calibration scheme and data, etc, in the accompanying OSF project;
• a statement of the expected outcome as per original paper, and the authors's hypotheses;
• annotated experimental factors, coding and randomization schemes;
• details of how samples were constituted (e.g. a priori method to determine sample sizes, and if no power analysis was performed then a description of stopping criteria, recruitment procedures, incentives, etc);
• details of the experimental designs, including explicit references to files hosted in the supplementary materials;
• inclusion and exclusion criteria;
• whether experimenters were blind to group assignment, and outcome assessment;
• a detailed account of the analyses performed, and any discrepancy from the original paper;
• a review and discussion of the results;
• a conclusion.

Replications of results -- "The method can be different, results are, hopefully, exactly the same"

While reproducibility is considered a minimal standard to evaluate a scientific claim, replicability (of results) is often considered the ultimate standard by which a result can be trusted.

The purpose of a replication is to assert a trait or a property that may be considered universal, within a particular context. A replication is an account of the attempt to perform a similar experiment as presented in the original paper; unlike a reproduction, a replication needs not be as faithful, as it is understood that some aspects of the original work cannot be reproduced, or actually need to be tested differently. For instance, an experiment in linguistics on English participants, may not qualify as a reproduction if it is performed on German participants. Similarly, an experiment that relies on the use of particular piece of equipment does not qualify as a reproduction, if it is performed on a similar piece of equipment but produced by a competing manufacturer, which may be performing measurements in very different way.

Submissions should include

• a brief summary of the original paper, and contextual information;
• a description of the aim of replication, clearly specifying what aspect of the results is replicated and to what end;
• a detailed description of the method: if at all possible, authors must share the raw material used, scripts, blueprints, etc;
• a statement of the expected outcome as per original paper;
• a detailed account of the analyses performed, and any discrepancy from the original paper;
• a review and discussion of the results;
• a conclusion.

Re-analyses

The purpose of a re-analysis is to assert the knowledge gained from a given dataset, either reproducing the same analyses, by means of a different methodology, or pooling different sources of data together. Not all science needs be producing new data, and we strongly encourage the recycling of data, with a view to formulating new insights.

Reproducibility toolkit

...

Open letters / Editorials

The purpose of an open letter / editorial is to provide authors with the room to share their experience of a given field. The replication of research is a sensitive topic, because of the way scientific research is evaluated by funding agencies, academic institutions and the general public.

Reviews

The purpose of a review is to assimilate the knowledge of a large body of work, to assist the formulation of a coherent, comprehensive and stable building block in the edifice of science. Reviews can take several forms, dictated by shared practices in academic communities. All reviews must include a one-page summary accessible to a lay audience.

Peer review policy

ReScience X follows a strict open peer review scheme, whereby reviewers of papers submitted are disclosed to the authors. The workflow is based on implementation by the Centre for Open Science. A submission “unit” is a Project in the Open Science Framework, containing links and hooks to all the material necessary to the submission. Pre-registrations are immutable, timestamped snapshots of OSF Projects, and the entire review process is logged and made public upon acceptance for publication. Authors retain control of the Project after publication, but published papers are themselves immutable, timestamped snapshots of the OSF Project, which are publicly available, searchable and indexed. Open comments to registered users are moderated.

Whenever possible, we strongly encourage authors of replications to engage with the authors of the replicated work, even before they invest efforts to produce new data. We also recognise that direct collaboration is not always possible, and therefore seek to foster collaborative, open, dispassionate and purposeful dialogue.

Submission and review guidelines

A step-by-step guide to submitting work to ReScience X is available. A submission must include a cover letter and a manuscript, and may include data, scripts, source code, software and images/VMs, amongst other things. Wherever applicable, authors are encouraged to provide as much detail and material to allow for the reproduction of their work—that is the faithful reproduction of the work and data.

Open peer review

References

Peng, R. R. (2011), Reproducible research in computational science. Science, 334(6060), 1226–1227. DOI:10.1126/science.1213847.

Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., … Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1), 1–9. DOI:10.1038/sdata.2016.18

Upon acceptance, submissions will be made available on the ReScience X website, indexed in relevant search engines, and advertised on social media.