Reproducible research and reformed publishing infrastructures
You can find this talk at jscaux.org/talks/Are you ready for
publishing reproducible research?
Delft, 15 May 2019
Jean-Sébastien Caux
Institute of Physics
University of Amsterdam
Plan of the talk
When do you have to worry about reproducibility?
When you...
- try to reproduce your own results from # years ago
- try to check/reuse somebody else's result
- referee somebody's paper
- act as an evaluator for a grant/job search/review panel
When reproducing, what do you have to worry about?
Do you...
- have access to all required material (info; derivations; data; plot scripts...)
- have the right software to view/rerun?
- have the rights to view, modify and reuse?
Typical reproducibility mistakes by researchers
Many (most? all?) of you are guilty of:
- not properly archiving your own work
- using closed-source (deprecation timebomb) software instead of free/open alternatives
- not reproducing results before reusing them
- not publishing reproducibibility-enabled results
"Fine, I see the point of reproducibility, but I don't have convincing reasons to do it"
- It costs too much time Think longer-term!
- Nobody reads/reproduces my stuff anyway Most probably, someone eventually will
-
Proprietary software works better than Free/Open alternatives
Most cases: bull...; and laziness costs €€€ -
I can get away with not doing it
It should/will soon be mandated
To achieve reproducibility, we need renewed infrastructure
- personal-level workflows
- collaborative toolboxes
- reliable "UNIX philosophy" Free software
- publishing systems
- evaluation systems
What is it? |
A complete publishing portal
(from preprint to metadata curation)
|
Who runs it? |
Professional scientists
(is and will remain grassroots)
|
What does it offer? |
|
Summary? |
|
What does it aim to achieve?
A complete reform of publishing at all levels
-
Implement Genuine Open Access
two-way: free for readers, free for authors -
Implement a healthier business model
- strict editorial/financial decoupling
- no profit-making on the back of scientists
- consortial model; not based on APCs
-
Modernize the refereeing procedure
- open; more credit to referees
- streamline post-publication feedback
- Reform impact assessment
Genuine Open Access Principles
Journals
302 publications
|
|
Roadmap
- Physics upscaling
- Selections
- Codebases
- Metrics
- Meta services
- Chemistry
- Maths
- Comp. Sci. ?
- [others?]
SciPost's infrastructure
Based exclusively on Free Software
Tech stack:
SciPost's main software systems are all built from scratch, in-house
- Django [Python web framework]
- Postgresql [database engine]
- Node.js [server-side javascript]
- Bootstrap [frontend]
- MathJax [LaTeX javascript engine]
- Gitea [git repo hub]
- Haystack, Celery, RabbitMQ, Flower...
Let's now talk business...
I love markets!
(heck, I'm from Canada, I even trained as a stock broker)Markets work well... except when they don't
Publishing: prime example of capitalism propped up by socialism
Classifying publishing business models
SciPost's business model
- no APCs, no subscriptions [Platinum]
- fully not-for-profit operations [Palladium]
- consortial model: donations
(inspirations: arXiv, OLH, ...)
Organizations benefitting from our activities
The PubFractions System
- Upon publication, authors are asked to apportion a unit of "support credit" between organizations
- These fractions are then linked to the relevant Organizations
- Every year, SciPost computes an average full infrastructure cost per publication
- Organizations can use this to determine their level of support
The PubFractions System
Organizations which support us are on our
Sponsors board
- (Inter)national funding agencies
- Universities & libraries
- Governments
- Foundations
- Benefactors
PubFractions: visible by sponsors
(they are given a special account)
Sponsors
Austrian Consortium
Plan S
What about Plan S?
Personally, I am very much in favour
How does SciPost relate to Plan S?
SciPost is (over)compliant
… but has not yet received official support from cOAlitionS (from some members: yes it did)
C.f. Point 3 of Plan S; we are hopeful!
Robert-Jan Smits @ KNAW
Dec. 2018
What about reactions to Plan S?
Personally, here, I am partly concerned
Genuine concerns exist, but…
lots of incorrect info is being propagated
I’d prefer to think scientists as being immune to fake news and hidden agendas, but…
My hope remains that scientists will embrace the needed reform rather than undermine it
End message:
Scientists: you hold the keys to the future.
With you, full OA with reproducible research will inevitably happen.
Without you, it won't.
So take your responsibilities and just do it.
Thanks!
Reproducible research and reformed publishing infrastructures
You can find this talk at jscaux.org/talks/
Are you ready forpublishing reproducible research?
Delft, 15 May 2019
Jean-Sébastien Caux
Institute of Physics
University of Amsterdam