Read the latest Issue
How to crowdsource a paper
Having taken an unconventional approach to writing his latest scientific articles, Aidan Budd shares tips on what to do – and what to avoid.
Aidan Budd has spent a considerable amount of time over the past year spearheading activities that have just culminated in two papers in PLoS Computational Biology. This would seem normal for an EMBL scientist – in Budd’s case, in the Gibson group at EMBL Heidelberg – were it not for the fact that the resulting papers are not bioinformatics analyses of gene regulation networks or metabolic interactions, but rather tips and rules for building a bioinformatics community and organising an unconference. Another particularity is that in writing the papers, Budd and collaborators took the collaborative nature of science a step further: both papers made use of crowdsourcing.
Whether it is getting video-gamers to help predict the 3D structure of proteins, drawing on people’s ability to identify shapes to understand how galaxies form or using a screensaver to search for signs of extraterrestrial life, the idea behind crowdsourcing is to pool resources and contributions from many people to generate data and knowledge.
Budd is keen to encourage others to try employing this approach while writing certain kinds of scientific papers, as it offers an opportunity for more and different voices and opinions to be heard on the topic of the paper, and is, he believes, an extremely effective community-building activity in itself.
Do…
…Have one person who really owns the project
“Someone who really wants to make it happen and will take responsibility, and at the same time has the confidence to believe that they can make it happen. If you’re that person, you have to be comfortable asserting yourself and pushing others and ultimately making decisions. As much as possible you’d like everyone involved to decide with you, and agree with you on the process, but also things need to stop at certain points.”
…Commit to open, transparent communication at all times
“If you’re open and transparent about what you want, what you want to do, how much time you have, and so on, this builds trust – it makes people more willing to work with you. When that openness is not there, I’ve only ever seen it cause resentment and frustration. And even though I know this, I catch myself also being non-transparent sometimes. You just go ‘oh, I know that person, I’ll just ask them if they’re happy to do that’, because it’s quicker at the time. But that carries a huge risk that other people will resent not being asked – so you have to really make a conscious effort. Typically in such situations I’ve gone ‘d’oh! Right, what can I do about this so that I can actually open it to everyone and genuinely give everyone an opportunity?’ And that can be difficult at times, not least because it often means giving up things that I want to do. But I accept that, by doing so, I’m helping the community to move forward.”
…Check your tone is always respectful
“It’s really easy to slip on this – especially in moments of stress or hurry. Amazing people who I know and like can be – without realising it – really quite disrespectful at times, and do things that I think ‘ah, I wish they hadn’t done that because I expect it’ll cause others to be less likely to contribute.’ So, remember that the way you talk to people has an impact on their likelihood of wanting to be involved.”
… Ensure you can get critical mass
“Make sure you have access to reasonably large groups of people who could be interested in getting involved. People are much more likely to get involved in something like this if they know the person who’s asking, and it’s someone who they think will actually deliver – a generic request from a stranger is much less likely to succeed. And in connection with that, look for people who are hubs of connections with similar interest, and who could know of other contexts or communities to tap into and help mobilise others.”
…Do some recon
“It’s also useful to contact potential publications in advance to see if they’d be interested in publishing something like what you have in mind – to see if there’s any point in doing it, and if so, if they have guidelines you should be aware of.”
…Consider organising related events
“For the community article we organised a workshop at ISMB [the International Society for Computational Biology’s annual conference] and invited all speakers and organisers of that workshop to contribute to the paper – the more crowdsourced part being that we collected feedback from all 100-plus participants and used this as a resource when writing the article. For the unconference paper, we held a ‘birds of a feather’ session at ISMB and invited everyone there to contribute. This kind of thing can help bring new people onboard, including people you might not know beforehand. But perhaps most importantly, organising something like this raises your profile and provides evidence of your dedication to the project and your ability to deliver, which can make people that bit more likely to join. You never know when you’re going to meet these people again; I just introduced myself to a new bioinformatician here at EMBL, asking if he’d like to work with us on the Bio-IT project, and he said ‘You don’t remember me, but I know you; I was at the unconference session at ISMB in Berlin two years ago.’”
…Realise that things have to end at some point
“These papers actually aren’t as crowdsourced as I would have liked. I had hoped to open the community article up further by asking people generically on Twitter and other social media, and incorporating those contributions too, but in the end I just couldn’t face the large amounts of work and delay that that would bring. At some point, I wanted to get something out there that was useful for people, rather than waiting to make it even more crowdsourced.”
Don’t…
…Underestimate the time and sustained effort required from the lead person
“I’ve found most people can make a valuable contribution without it taking too much of their time and effort, but you need at least one person – and, ideally, just one person – who makes a considerable amount of effort to make things happen.”
…Do everything by email
“Actually talk to people. Decisions can be made so much quicker sometimes by discussing in person or at least on Skype – or teleconferences, group chats, whatever – than in endless looping email rounds. You can clear up misunderstandings so much faster, and get a sense of the majority’s feeling – and just generally decide together.”
…Have too high a standard
“There are things I’d change in both articles, but I’m alright with them. Actually, I’m proud of them! For the paper to actually get out, you have to be a non-perfectionist. I often ask the question ‘Is there anything in here that is so bad you would not be prepared to put your name to it?’ – this is different from ‘Is there anything you think should be changed?’. Obviously you want something good, but you can’t enter endless cycles of change after change after change. People need to appreciate the power they have to say ‘No’, and the amount of work and effort and delay that comes from making changes. You want revisions, but not too many: the right number to get something that has enough quality for everyone to be prepared to go with it. Assuming you have smart, clued-in people involved, this level of quality will be high enough for the resulting document to be useful for readers. So, if everyone’s prepared to put their name to it, then we’re OK. People have studied these things; there’s a whole framework for reaching consensus – if you’re thinking about doing something like this, read about it.”
…Crowdsource research papers
“Crowdsourcing is great at bringing diverse, valid input to an article when the resources needed to contribute are low; it’s easy for more people to contribute. This makes it good for papers like opinion, training or review articles, which present broad ranges of ideas and opinions on a topic. But when you crowdsource a paper, the whole point is to open it to contributions from many people, and this makes it hard to limit who has access to the text. So crowdsourcing doesn’t work well in contexts where the article cannot be safely shared with a large group of people without compromising intellectual property (IP) or scientific precedence, for example. This, combined with the fact that research articles require more resources – experiments take time and money to plan and execute, and so does data analysis – means that they’re not suited to crowdsourcing. Having said that, valid, important research articles can be published from data collected via crowdsourcing. This is the basis of ‘citizen science’ – a topic on which we’re trying to crowdsource another article! However in these projects, the resulting articles aren’t usually crowdsourced: the scientists designing and controlling the study are typically the paper’s authors.”
The two papers just published in PLoS Computational Biology brought together people – mainly bioinformaticians, but also teachers and journalists – from three continents, and Budd would like to extend their reach even further. “Even the ‘building a bioinformatics community’ paper has many tips that could be widely applicable in other fields. In general, these are things that people don’t often think about, so we are happy just to raise the profile of thoughtful community building – that, rather than diving straight in, it’s something that’s worth deliberating, and reflecting on the success, or not, of others.
“It’s the same with ‘unconferences’: it’s about spreading ideas and things that we’ve enjoyed a lot and consider have been very successful, and would like to offer other people the opportunity to do the same. It’s about letting people know that these things exist, and maybe spiking their interest and giving them the confidence and inspiration to try them themselves. All of that goes beyond bioinformatics. We’d be very happy if people from other areas read the articles and shared their thoughts and experiences. That’s part of the reason why we made sure these articles were published open access, so that everyone – bioinformaticians or not, scientists or not – can read them.”
And if you have – or are planning to – crowdsource a paper, Budd would love to hear from you, either in the comments below, on twitter, or via email.
This story has been updated since it was first published to clarify how crowdsourcing was used to a lesser and greater extent for both papers. For more information please contact us.
