Publish AND perish: Medical researchers under pressure

VOICEOVER

This is Up Close, the research talk show from the University of Melbourne, Australia.


ANDI HORVATH
I'm Doctor Andi Horvath, thanks for joining us. Today we bring you Up Close to research on researchers. We're putting academics and scientists under the microscope and on the psychiatric couch. While scientists serve society well, we are increasingly concerned about misconduct in research results that don't hold up under scrutiny. We're now realising that many research findings aren't in fact reproducible, a key factor in research credibility.

The reasons can range from fraud, to poor understanding of methodology, to simply careless work. So what's driving researchers to undermine their own science? As with people in other professions, scientists have job related stress and occupational burn out, they're under pressure to perform, after all it's a “publish or perish” world.

But does the pressure on scientists help explain compromised research? What else could be going on between the ears of the people whose work can have a real impact on the rest of us? To explore the question of how scientists and other researchers view the pressures they're under, and how their work is affected, we're joined by researcher and psychiatrist, Dr Joeri Tijdink. Joeri has written a number of papers picking apart the culture around research publications and what it does to the people who write them.

Dr Tijdink works as a psychiatrist at Waikato Hospital in Hamilton, New Zealand, and he's in the final stages of finishing his PhD at VU Medical Center in Amsterdam. He's here as a guest of the University of Melbourne's Office for Research, Ethics, and Integrity. Welcome to Up Close Joeri.


JOERI TIJDINK

Thank you very much for having me here.


ANDI HORVATH

Now what prompted you to start researching the researchers?


JOERI TIJDINK
Well I was a junior medical student and I was at the psychiatry department at my/ MI University and I saw professors during a research meeting stressing out about authorships. It was a discussion whether one should be on the third or the fourth place on an author's list. At that time I thought what is going on here? I mean they are fighting for a spot on an author's list and it's an article, not in a very high impact journal and it's their third or the fourth spot, why is it so important? So that's where the actual seed started to grow.

I mean why is the culture like that that professors are fighting, are stressed about their publications? I thought that medicine was about treating patients and getting patients better. I was naive at that point I guess but they didn't stress about patients or about patient care or about education or duties or about management, they stressed about publications and about getting their names on papers.


ANDI HORVATH

Why is that important to them?


JOERI TIJDINK

Well for their careers it's essential to publish, preferably in high impact factor journals, like the BMJ (British Medical Journal) or The Lancet. The other thing is that - and it comes out of my research as well - researchers, to quote one of the participants in my study he said, you just get high when your article is accepted in the high impact factor journal, you just get high. I even can acknowledge that because I had the privilege to publish in the BMJ as well and the consequence is it can make or break your career. I mean high impact factor journal will always be on your record and it will help you get funded, get the positions you want.


ANDI HORVATH

So how do we define pressure in the world of research?


JOERI TIJDINK

Yes, the present culture is predominately built on citation and publication records of scientists, so publishing is very important. It's the same with getting funded, it's also based on your scientific output, so that's why the pressure to publish is that high. If we measure scientists on a different level, like say educational skills or management skills, then probably the publication pressure we feel would be less.


ANDI HORVATH

Is some of the pressure also brought on by journals wanting to publish certain things?


JOERI TIJDINK

Well every journal wants to be novel, innovative, or creative, or have unique findings and the scientific world wants to publish what the journals want. This will influence the researcher in a sense as well because they will have to publish positive results, negative results won't get published, and that's another very important factor in the current debate on publication culture, that the negative results they end up in a drawer.


ANDI HORVATH

What about even the choice of research projects. For instance, if I know that I can get more papers out of a certain research project am I going to go down that line?


JOERI TIJDINK

Yes, if you know for sure that you get high impact factor journal publication out of a certain line of reasoning, a certain line of research, you will tend to go for that line of research. Well I think that we should emphasise the quality of science, also the innovative and the creative part but also the quality of science -- that should come first.


ANDI HORVATH

So does the old chestnut, publish or perish, actually drive scientists to the dark side? Tell us about definitions of misconduct in the scientific world.


JOERI TIJDINK

Well it's very hard to put that as a causal relation, of course, but what is exactly research misbehaviour? We have to go back to what is right and what is wrong and that's not something dichotomous, it's a very grey area. Of course, on the right side is responsible conduct of research and on the wrong side it's fabrication of data, that's essential, but between that it's like fifty shades of grey. Is it actually that bad to add an author to your author's list without significant contribution?

Well it's bad, but you should not go to jail for that. On the other hand, is it bad to fabricate data or to delete outliers? You can question that but fabricating data is definitely a really bad thing. This is exactly why it's so difficult to actually define research misbehaviour.


ANDI HORVATH

All right, let's get a picture of the prevalence of misconduct amongst researchers. How did you explore this and how do we know how rife it is and I'm talking about those shades of grey that you've referred to?


JOERI TIJDINK

Well to put it bluntly, you just ask and you try to guarantee anonymity among your respondents. So you send out an electronic email which is based on 24 items regarding research misbehaviours and you ask them, did you conduct these kinds of research in the past three years? It's open to speculation whether someone would actually answer this question honestly. Let's say if I did some questionable research practises I would be very hesitant to answer this honestly. I would be a little bit paranoid about the answer and about what's going to happen with my answers of course. But still, a lot of scientists were pretty honest in their answers.


ANDI HORVATH

That's good to hear, were they also asked to speculate on their colleagues' levels of misconduct?


JOERI TIJDINK

Yes, we put these questions in to entice them to answer them honestly, because if you say no to a question for yourself, if you observed it in yourself, maybe you've observed it in colleagues. Then you get an estimate on whether this research misbehaviour is actually present in his research group or in his close colleagues and this will help you to get an estimate.


ANDI HORVATH

So what did your findings actually show?


JOERI TIJDINK

The findings showed that there is severe misconduct in terms of fraud, in terms of fabrication or falsification of data, plagiarism, that was pretty common, more than 10 per cent of the sample I asked in Flanders responded positively. They admitted it, they have committed these types of scientific misconduct and there were less severe cases. We divided them in three categories, one is outright fraud, a falsification, or plagiarism, then we had the severe misconduct and the moderate misconduct and moderate misconduct can be adding an author to the author's list without significant contribution.

We found that that was extremely common, that a lot of Belgian scientists put someone on the author's list without signification contribution, about 60 or 70 per cent of researchers did that which is extremely high in my opinion. Over 10 per cent of researchers admitted that they have conducted fraud, fabrication, falsification or plagiarism of data.


ANDI HORVATH

Did you assess different stages of a scientific research career right from PhD students to professors?


JOERI TIJDINK

Yes, and interestingly the number of young scientists that admitted that they have committed severe research misconduct was higher [than] among professors. Of course, it's a small sample, over 300 people participated in the questionnaire and there are heaps of limitations on the research, not only in terms of causality but also in terms of incidents of reporting bias or selection bias, that's essential for the scientific process that we take that into account. But yes, in this sample the younger scientists admitted more frequently that they committed these severe research misconducts.


ANDI HORVATH

I'm Andi Horvath and you're listening to Up Close. In this episode we're talking about the pressures of publication performance in science careers with psychiatrist and researcher of researchers Dr Joeri Tijdink. In the past few decades there's been considerable research in the business sector on the predominant personality types of the captains of industry, the CEOs, and high level managers. Many have been found to lack empathy and there are even sociopaths among them. Joeri, it's astonishing to hear the proportion of scientists who are so pressured to publish that they're not conforming to the ideals of science and research. Is there a particular personality type that is predominant among the high level researchers and scientists?


JOERI TIJDINK

It's an interesting comment. I do have to tell that these are still preliminary data where we're talking about - but we did some research on whether personality profiles relate to scientific misbehaviour and also relate to academic position as a secondary outcome. What we do find is that narcissism and psychopathy are more prevalent among the higher academic ranks.


ANDI HORVATH

Scientists and researchers don't often work nine to five because their work doesn't lend to that, tell me more about scientists and their immersion into their work and do they suffer burn out at all?


JOERI TIJDINK

Yes, that's actually the first study we performed on full professors in the Netherlands which was the base for further research. In that study we found that the full professors, 24 per cent of them had extreme levels of burn out characterised by emotional exhaustion and that's extremely high I must say. Burn out is not a psychiatric condition, it's more like a psychological condition, very related to your work. But coming from that same research sample we saw that these professors were highly engaged with their work.


ANDI HORVATH

So they could be engaged but also burnt out.


JOERI TIJDINK

Yes, maybe because of their extreme engagement, the dedication and vigour regarding their work and probably in your surroundings you know a professor, they are workaholics but they love their work as well and they love to conduct science and they love a lot of things in their work.


ANDI HORVATH

Sure.


JOERI TIJDINK

But they also feel emotionally exhausted and they can be cynical towards science.


ANDI HORVATH

Given that human behaviour is driven by emotions then this would have to have a biasing effect on research quality and other parameters as well in the world of research.


JOERI TIJDINK

Well that's very difficult to research actually but it is a logical line of reasoning. If you're stressed, if you're emotionally that attached to your work it influences objectivity. I mean science is about being as objective as possible, certainly in methodology. We didn't study that but I really believe that there is something going on that we have to take emotional perspective into account. I am biased towards my research, I really believe in my research, and readers that read my publications they don't see that but it is there and it's very hard to address.


ANDI HORVATH

It might seem like the issues of misconduct and burn out in the research sector are only a problem for that sector but it's actually of much greater importance and wider impact because the rest of the world relies on and invests trust in scientists and their findings. So how can we ease the pain for researchers?


JOERI TIJDINK

Oh, that's a very hard question and I'm not sure if we can ease the pain for researchers or for the public. The problem lies in so many different factors and aspects in the publication culture that it's not so easy to address. But from a psychiatric perspective - and I always like to use my profession to deal with a problem - we should focus on diagnosing first, not only to measure publication pressure and not only to measure research misbehaviour but also to measure publication bias in journals, to measure how we cope with authorship. It's also to measure stress levels in scientists of course. That's just the diagnostics.


ANDI HORVATH

What about treatments?


JOERI TIJDINK

Exactly, treatment and prevention, I would say we have to educate our future professors. Only five to 10 years ago the first widespread research integrity training programs were actually implemented in educational systems. We only started a year or two years ago with compulsory research ethics training before you can actually graduate or before you can do your PhD. These are things that are essential because you have to learn what's good and bad behaviour and not only by education, also by good mentoring or by clear guidance or by signing honour codes, there are so many different ways.


ANDI HORVATH

How about the publishers, is there a way forward for them and I want you to tell me about negative results as well as positive results or whether or not we should be fighting impact factor?


JOERI TIJDINK

Of course, I totally agree that if negative results end up in a drawer why should we conduct that research at all, and then you come with the actual question of research waste, this problem was raised in The Lancet a couple of years ago. The two step submission design, for example, can be a very good way to cope with the negative results problem. The two step research design is that you send in to a journal the introduction and the methods of a scientific paper, and based on this information, a journal is going to accept the publication, whether or not the outcome would be positive or negative.

This can partially help prevent publication bias and it also helps us to determine before we actually conduct an experiment what our primary outcomes, our secondary outcomes, our confirmatory research, and our exploratory research, essential ingredients for responsible conduct of research which right now is totally blurred for a lot of researchers. Still I really want to make this statement, the vast majority of researchers are very honest. But there's this small proportion of outliers that is intentionally misbehaving and a large proportion of the whole group is unintentionally misbehaving.


ANDI HORVATH

I know one of the concerns for universities is that industry and commerce come to universities to be collaborators in various research projects, what pressures do they bring to bear on the research?


JOERI TIJDINK

Well let's do an experiment of thought, let's say I'm a researcher and I'm completely funded by a pharmaceutical company that is developing a new antidepressant. I conduct the research and of course the industry will tell me, well you're completely free whatever you write down but I - in the back of my mind I know they are paying me maybe tens of thousands of Euros to get the research done. Do you think that I want to give them a negative result regarding their development of the antidepressant, no, of course not?

If there's a negative result I'm not very likely that I'm going to publish it because it's going to be a negative influence on the company. So I think this is a clear example that it's so hard to conduct proper research for these companies. They pay you for your living so it's so hard to get very honest research on that.


ANDI HORVATH

Yes, do we need a journal of negative results?


JOERI TIJDINK

Oh, there is already one.


ANDI HORVATH

Oh, there is.


JOERI TIJDINK

Yes, but the problem is that the impact factor is so low that no one wants to publish there.


ANDI HORVATH

We certainly have our challenges ahead of us.


JOERI TIJDINK

Yes.


ANDI HORVATH

Dr Joeri Tijdink, thank you so much for being our guest today on Up Close.


JOERI TIJDINK

Thank you very much for having me here. I especially want to acknowledge two of my very close colleagues, Professor Doctor Ivo Smulders and Professor Doctor Lex Bouter. They have been great for me in guiding me in the right direction in my research.


ANDI HORVATH

Thanks Joeri, it's time for better science and the time starts now.


JOERI TIJDINK

Exactly.


ANDI HORVATH

We've been speaking with Joeri Tijdink, a psychiatrist at Waikato Hospital in Hamilton, New Zealand. He's also in the final stages of his PhD at VU Medical Center in Amsterdam. You'll find details of Joeri's publications on the Up Close website together with a full transcript of this and all our programs. Up Close is a production of the University of Melbourne, Australia. This episode was recorded on 16 March 2016. Producer was Eric van Bemmel, Audio Engineering by Gavin Nebauer and I'm Doctor Andi Horvath, cheers.


VOICEOVER

You've been listening to Up Close, for more information visit upclose.unimelb.edu.au. You can also find us on Twitter and Facebook. Copyright 2016, the University of Melbourne.