CHOICE membership

Scientific research


#1

What if we made science accountable?
Science has never been more accessible. Anyone can research any topic in the comfort of their own home at all hours of the day.
At the moment you can search the internet and find a published study/scientific paper to back almost any side of the argument you like. Whilst I understand there is never one clear answer to a question, or a single solution to a problem, it seems to me that with the enormous amount of information out there today we could do with a second level of accountability.
What if every scientific study had to be reproduced by an independent group before it could be published? Or what if they had to have an independent peer group assess it for quality and give it a rating?
Do you think there is an issue?
How do you think we can put the reliability and credibility back into science?


#2

You seem ignorant of the scientific process, peer review, etc, that governs the vast majority of scientific journals and publications. It is very rigorous. Is your concern because you don’t agree with many findings? I would add that questionable independent scientific papers have always been scrutinised. The dodgy ones are routinely called out.

Unfortunately some vested interests make names to hide behind, that don’t reflect who or what they are, and fund “fringe science” to market and support their causes. One of our government ministers stated it is everyone’s right to be a bigot, and everyone has a right for equal time on the soap box, intelligent as well as not. Pretty special, eh?


#3

Hi pdtbaum, I can see how my post could suggest my ignorance to the process, I am however aware of the rigour involved.
It is however possible, that the current system is not holding up, as financial gain becomes more imbedded in the outcome, the research could be loosing objectivity.


The following paper in 2012 discusses the low rate of reproducibility of studies on cancer research http://www.nature.com/nature/journal/v483/n7391/full/483531a.html
while I recognise the irony in using scientific papers to questions science, these results do raise some questions around the topic.
Further to the involvement of money: Is there bias in article selection by the journals who make profits by publishing them? http://www.nature.com/news/open-access-the-true-cost-of-science-publishing-1.12676
More than that I wonder how the general public (layperson) is impacted by the scientific ‘evidence’ they now have access to, can they really decipher it, can they see the weakness in the studies, are they able to pick a strong result from a weak one? Or do we need an overlay of explanation in common language to increase its value to those outside the industry?
http://www.americanscientist.org/blog/pub/8-myths-about-public-understanding-of-science
kindly Sonia


#4

Thanks for that. Much clearer about your concerns. Well considered information in the links. While I am empathetic and supportive of your description of the issue, I would have to be on the side that thinks any attempt to do more than educate the uneducated would be either counter-productive because of unexpected outcomes of any attempt at regulation, or would water it down to the point of triviality and thus abuse rivalling the original. Regarding scientists using jargon and thus communicating poorly, I have been in many meetings where their conclusion was that by using “layman’s terminology” it would usually change the discussion by removing specificity and precision of what they are trying to state.

Then let’s use a royal commission as example - unbiased? predetermined based on who is appointed? political? Having worked with both academia and governments in scientific disciplines, getting independent corroboration (as compared to peer review) would be somewhat like a mine field that would probably do no more than add more confusion, interests, etc, into the equation while somehow funding the “independent experts” who are likely not going to be independent and unbiased, subject to who and how they are appointed. Ref my previous comment on our illustrious minister as how it could start. A dollar in someone’s pocket could be introduced as motivation in numerous ways.

Revealing sponsorship would be a major step, but that could and would be abused by vested interests with dodgy science making companies with nice names to protect “the guilty”.

Cheers,


#5

@soniagulwadi , one also has to remember that science is testing a hypothese, using the best/preferred methodology at that time.

As more ‘science’ is done, others may test the same hypothesis using a newer technique or preferred methodology. The underlying assumptions may also change as the science is tested and becomes clearer.

This can give similar or contrasting results. This is what science is about, questioning and testing.

A simple example is water. Science has proven that water is toxic at high consumption (can impact on salt balance resulting in death). It could be reported that water is toxic to humans, when in fact it is essential to life at low to moderate consumption. Neither outcome is wrong, just different hypotheses and methodologies giving different outcomes.

Having worked as a academic scientist and in the environmental science are for many years, it is important not to only understand the results, but the process which lead to them. A good scientist also questions both the results and methodology…something you appear to be doing.

Science can also take decades to resolve or gain consensus - climate change is a good example of this.

It is also important not to confuse populist scientific articles with journal articles. Populist articles usually are written to provide a particular outcome based on the authors own thoughts.


#6

In my humble view, the argument presented above are all valid either one or another. Having been in the academic glass-house for some years before entering the industry arena, I have learnt to take journal articles somewhat differently. Back in the university days, I have been taught one motto: “either you publish or your perish.” I guess this remains frightfully true?

Then in the real world, having worked for two companies manufacturing a similar range of products, I have also seen how data can be tweaked into one’s advantage or shift the reader’s focus without falsifying any result whatsoever. Therefore, I always go straight to the summary of results and ‘author’s declaration’. I usually apply a discount factor to those journals known to be without vigorous peer review or research commercially sponsored. I do pay attention into, for example, sample size (especially those based on meta-analyses), the experimental design. Sadly, I have seen semi-quantitative analytical results turned into ‘statistically significant data’.

I recently read an published article from a reputable institute (but sponsored by one of my ex-employers). As it turned out, the results did not support what the company had been purporting for the product studied; much to the advantage of her competitor. There was no influence or intervention whatsoever.

I would suggest that we should have an open mind and assess each paper on their own merit. In the meantime, keep promoting the importance of ‘integrity’. I know some sections of the scientific community have already been doing that.


#10

Apologies. I agree with what soniagulwadi said.


#11

This report sheds a new light on some peer reviews. It seems ‘China Inc’ did not set out to do this, but because of misguided approaches incubated a ‘fake research’ culture. Subtitle: ‘What is the value of peer reviews in any particular case?’ Diligence about the research and review process is more important than ever, not counting unsubstantiated, ideologically driven, and sometimes unchallenged claims re various science-based issues by an increasing number of politicians globally.


#12

I suppose this culture has existed for centuries, but possibly has become more prominent in the past century. An example is the harvest of animal parts from Africa to satisfy a demand within China for some hopeful medical cure/treatment…when such has no scientific basis. Just because an animal runs fast, a prolific breeder or is strong, doesn’t mean one of it body parts will cause the same in a human.

Having lived in China, it is interesting to see some of the more out there TCM practitioners diagnosing a wide range of ailments just by holding ones wrist