Sloppy Science is Bad Science

Generally when I am criticizing sloppy or bad science, my criticism is aimed at creationists (the Intelligent Deception lobby) or global warming deniers (the Denial Lobby). Sadly, today I have to criticize my fellow research biologists for some really sloppy science.

There is an alarming, if a bit overstated, article on BBC news today. It seems sloppy science may invalidate a chunk of cancer research, setting back treatments for patients with particular types of cancer. To quote the article:

...it comes as a shock to learn that millions of pounds in charitable donations and from taxpayers are being wasted on "worthless" research for lack of good housekeeping practice in the lab.

...many scientists fail to carry out simple and inexpensive checks to ensure that they are working with the right experimental materials - particular forms of human cancer cells.

As a result, thousands of studies are invalid.

First off, let me say that this is slightly overstated. In many cases the basics of these studies will still be valid. But it is true that in many cases the results would need to be re-evaluated and may mean the results do not necessarily apply to the particular kinds of cancer they purported to study. That alone is devastating to science...but it doesn't mean these studies are all completely invalidated.

For more than 20 years, researchers in many countries have used a particular experimental culture of cancer cells known as TE7 to study one type of oesophageal cancer.

This 'cell-line' has been sold worldwide and formed the basis of hundreds of published studies.

But Dr Tselepis and an international group of colleagues discovered that TE7 is the wrong type of cancer, and the implications for two decades of study are grave.

"If there's a study which is claiming that a particular drug has a particular effect on a cell type and potentially you could use this drug for the treatment of patients that clearly is going to be misfounded," he explained.

In my previous job I encountered a situation like this. We were not doing research on any particular type of cell. Rather we were looking at some very basic processes that apply across cell types and our goal was to study a variety of cells. One of the cell lines we picked to study turned out to not be what it was supposed to be, but rather a contaminant representing a different cell type. The mistake was not ours, it was the mistake of the people who originally isolated the cell line. Fortunately for us, all it meant was that we had a redundant result and one less extra cell type we looked at, but it in no way invalidated the study. (For anyone interested, the published paper can be found here). In the case of the misidentified cell line we were working with, the cell line itself was misleading to the people who first isolated it. They identified it as being one cell type based on the presence of a particular protein, a technique that is fairly common and usually reliable. However, later work using a far more sensitive technique looking at something called "microsatellite DNA" showed the cell line was NOT what the protein marker indicated, but that it was a varient of another, previously isolated cell line that must have been a contaminant.

Needless to say, this kind of thing is sloppy and embarassing. Problem is, in studies where the exact kind of cell type being used is important, this kind of thing can greatly detract from the value of the research. When potential clinical treatments are based on such studies, then the potential clincal treatment may be called into question.

I should emphasize that clinical treatments I am talking about are ones that are in the early stages of study. By the time a treatment goes through clinical trials and becomes accepted, it has already been tested for the particular use in question. That means that anyone currently receiving such treatments don't have to worry. The treatment, once it has passed clinical trials, will do what it claims to do. The problem is with so-called experimental treatments that are undergoing clinical trials. A clinical trial may wind up testing a drug for the wrong type of cancer because of the sloppy science the clinical trial is based on. So what is slowed is the pace of discovering new drugs. The efficacy of drugs already in use is not affected by this sloppiness.

But in cases where mistakes are discovered, the proper thing to do is report that mistake, reevaluate any studies based on that mistake, and stop making that mistake. It seems some scientists, either through ignorance of the misidentified cell lines or deliberately (in which case they move from sloppy into potentially fraudulent) continue pretending that the misidentified cell lines are what they were mistakenly thought to be. In other words, some science continues to make the same mistake.

And even after a false cell-line has been unmasked, scientists carry on using it as if it were the real thing while other studies are utterly invalidated by the error.

Professor Gertrude Buehring, found a third of studies involving a false cell-line in publications from 1969 to 2004 used it in an invalid way as if it were the cell type of origin and obviously having no idea it was a contaminant.

There is talk of requiring this kind of research to authenticate the cell lines they are using if they want to publish a paper using that cell line. This is a very reasonable and very appropriate way to deal with this situation. We are expected to prove that anything we use is what we claim it is. That is critical for genuine scientific research. I know I would have been dismayed at such a requirement when I was using cell lines, but the truth is the authentication is fairly easily done and probably many institutes could start a core facility to carry it out for its researchers. Or some companies could provide the service like. Core facilities and companies now do the DNA sequencing for most labs, a technique that I had to do myself as a grad student and which I became quite expert at...a skill that is now long obsolete. Authentication of cell lines could easily be done in a similar way and is comparable since a great deal of DNA sequencing done today is to confirm that something made in the lab is accurate. Hopefully such a requirement will be put in place by at least the top journals and we can reduce the sloppiness in this kind of research.

Bookmark/Search this post with:

| | | | |