Science is not an ideology, religion or set of facts. Science is a process for trying to determine a more accurate Truth of what is Real. Even though science is a good process, it is a process that is done by people, so we do have to contend with the flawed human factor. While there are problems with how we implement the scientific process, it is still the best method we humans have developed for understanding the difference between an idea that is false and an idea that is accurate.
Truth, Science and People
Let us talk about truth for a moment.
Objective Reality is the Real universe with a big R, where we cannot truly know what is there because the scope of the universe is just to complex and beyond our real understanding. We can approximate what the universe is, and via the Scientific Process, we can move our model of the universe closer to Real. So Real with a capital is the objective actual universe, while real with a small letter is what we think is real. I will also talk about Truth versus truth, where Truth is Objective Reality True and truth is what we understand to be as close as we can get, which allows for us to learn new more accurate truths later.
If we call Objective Reality the Truth, with a big T, and recall that we can’t truly understand Objective Reality because it is beyond our human comprehension, then we can’t truly know Truth. Our goal, however, is to get as close to Truth as we can – that is, we can approach it, but never truly get there. Anyone who claims they have Truth knowledge is lying. We want experts to tell us the truth, with a small t, which is the closest to Truth they have got. The expert should be able to tell us why they think it is close to Truth, but also why it isn’t the Truth.
How do we find our close to Truth truth? This is the heart of the scientific process. Science is not Truth, it is a process for moving our understanding towards Truth. Essentially it works like this.
- We have an idea about a thing. All ideas that are useful are effectively helping us make predictions. For example:
- Is this thing an apple? If yes, it can be eaten and have certain properties. It it isn’t an apple, it may or may not be edible, check to see if it is in another “edible” category, or in an “inedible” category.
- There is a loud sound on the other side of those trees, which might be a large dangerous animal. If it is dangerous, I should do something to improve safety. If it isn’t, that action is a waste of energy.
- The idea will have rules, and those rules will be used to define what is in that set of rules to be the idea, or out and thus excluded. Those rules give us a prediction of the future. Based on that hypothesis, you predict the next part of the pattern. If what happens next matches your prediction, your hypothesis is an accurate enough model to use. If it doesn’t match, then your hypothesis is wrong, create a new one, or modify the one you have to be more accurate.
That, right there, is the basic idea of science. Trying to get more accurate ideas about the world by testing those ideas through what those ideas predict.
The takeaway message here is, the scientific process helps us to work out better ideas, hypotheses and models. That moves our truth closer to Truth. With these better models and predictions, which can make our lives better. We have got so good at these ideas, tests and predictions, that we can make the device that you are reading this on – a device made through science.
Essentially, or Models are False
George Box, a British statistician, working on quality control, time-series analysis, design of experiments, and Bayesian inference
“Essentially, all models are wrong, but some are useful”, George E.P. Box, Empirical Model-Building and Response Surfaces
Remember what I said about Object Reality is Real but beyond our understanding? All of our science and all of our personal understanding of the world are mere models of Objective Reality. They are all wrong, as they only model what is complex in a way that we can comprehend. So long as they model is useful, we can use it, but we need to keep in mind that it is not actually Objective Reality.
Isaac Newton created a model of physics back in the 1600’s that we still mostly use to describe how physical objects move. Albert Einstein created a more specifically accurate model with his General Relativity. We often don’t need Einstein’s physics for everyday calculations, because Newton’s is good enough for most things, even though Einstein’s is a bit better.
All of our models have limits of accuracy. So long as the model is useful in getting us mostly the right answers most of the time, it is good enough. Don’t discard the model just because it isn’t perfect, but do replace bad models with better ones.
Not Truth, “Alternative Facts” and Pseudo Science
We have got really good at figuring out what isn’t true. In the early days of the scientific process, we had many hypothesis that seemed sound until we tested them and found that either the idea had no predictive power, or was wrong. For example, phlogiston is an an idea that there was a chemical or element inherent in flammable things that enabled them to burn. We now know this idea is wrong, that things burn by combining with oxygen (generally not in the thing, but in the atmosphere), to make new chemicals. We knew things burned, a hypothesis for how that happened was proposed, it was shown to be wrong. In searching for what worked, we replaced the hypothesis with a different mechanism, and testing this showed it was accurate.
While the scientific process is pretty good, science is done by humans, who make errors and mistakes. This can lead to scientists having a pet theory that they refuse to let go of even when the evidence says it is wrong, or for movements within science taking a while to be corrected. We know of so many times that scientists have made mistakes because the scientific process works – without science correcting those mistakes, we’d still think they were true. The ideas were tested and found to be faulty, which increased our knowledge. Science is constantly checking itself with newer technology and greater understanding and fixing its own past errors, constantly moving human knowledge towards Truth. Some may see that as an indication that we can’t trust science because it makes errors. To err is human, to fix it takes a process, and we call that the scientific process, or science for short.
When an idea has been tested for a while in multiple ways, and found to not work, then you can trust that it is fiction. Acupuncture is a good example of this. Acupuncture is said to be an ancient Chinese medicine, dating back thousands of years, where piercing the skin in key locations will change the flow of ki energy in humans, which can either harm or heal a person.
While what we recognise to be acupuncture now is fairly new, as the technology to make thin needles is only a few hundred years old, that doesn’t automatically mean that what we call acupuncture in this day and age doesn’t work. To determine if a model works, you need to test it. Many scientific examinations of this have been performed. A few of these tests have shown that acupuncture works better than chance and better than the placebo effect. Acupuncturists state, therefore, that acupuncture is evidence based.
The problem with this claim of evidence based, is that most of the evidence says it doesn’t work. For a start, many leading practitioners disagree with each other about where these acupuncture points are, yet state that there version consistently works and is the only model that is true. If where you insert the needle doesn’t matter, then it isn’t acupuncture, it is just stabbing you. If it does matter, then how can there be many systems of “real”? When setting up a trial for acupuncture, trying to compare “real” versus “sham” puncturing is impossible when there is no agreed upon definition for “real”. Even so, scientists have tested most of the leading versions of “real” and found that they consistently give the same result as the sham, which is often no greater than the placebo effect (where you think you are better because you got treated, regardless of whether the treatment was real or not). [Read more: Acupuncture is Theatrical Placebo, Acupuncture on Science Based Medicine, Acupuncture: A History)]
If there is a great deal of uncertainty in the world’s view of what that thing is, then look to the experts. Often there will be a strong “we mostly think it is ‘This'” with some small groups saying “it could also be That”. It is rare for the experts to have a fairly equal division (50/50) on what the experts think is correct. A recent example of that was with SARS-CoV-2, which causes COVID in humans. During the first 6 months, there was a great deal of uncertainty about what this novel (new) virus did to humans and what was the best way to medically treat it, and socially combat it. Around 6 months later, strong groups of effective methodology had been found and relayed to the population of Earth. There were also isolated pockets of fringe claims that were clearly false, like ivermectin. By the end of a year, most of the world experts agreed on the majority of the management plan with some minor differences in some areas. Despite this consensus agreement born of masses of data and evidence, and despite the hundreds of millions who were saved, most of the world’s people surveyed are not confident that the scientific consensus is accurate, which only shows that people are easily confused by media chasing click bait headlines.
The ivermectin claim is a good example of pseudoscience. On scant evidence, the medication ivermectin, which is excellent for certain conditions, was supposed to significantly help in the treatment of Covid. Since we didn’t know what would or wouldn’t work, and a plausible mechanism for how ivermectin could work for this condition was offered, medical scientists worked quickly to test this hypothesis. Experiments fairly quickly demonstrated that ivermectin didn’t help in any reliable or consistent way, so it was dismissed. Fringe groups continued to try to push for this treatment, despite the massive lack of evidence that it worked (and the lack of probable fraud on the ones that said it did). Oddly, the people who most promoted ivermectin as a treatment stood to gain financially by selling it. As Dr Kyle Sheldrick states, they did not find “a single trial” claiming to show that ivermectin was an effective preventative or treatment, without “either obvious signs of fabrication or errors so critical they invalidate the study” [Ivermectin: How false science created a Covid ‘miracle’ drug]. Effectively, the claim was extraordinary, the evidence to support it was suspect rather than extraordinary, so the claim was discredited and abandoned.
Pseudo Science is the term given to people who claim that their method works because it was scientifically proven. They will either cherry pick the experiments that gave them a positive outcome and dismiss the ones that didn’t, or they will create poor quality studies that bias the results, like putting your thumb on the scale, invalidating the result. They will often throw random science words, or science sounding words, into their discourse like “quantum”, or “biofield”, to confuse people who don’t understand the scientific process.
Inherent Problems in the Scientific Process
While the Scientific Process is the best way we currently have to become more accurate in what we know and what we can do, there are some major problems with this process. This comes in two major flavours – human weakness and the limits of flawed systems.
Science is done by people. People need to earn money so that they can live. Money is paid to scientists who get results, which is measured by publishing or the creation of products. To publish means to submit your experimental results or theoretical frameworks in reputable journals, who may reject your paper because it is faulty, repetitive, or not interesting enough.
For scientists, it is often publish or die (career death).
To get your paper published often means ensuring that your paper is not just repeating what someone else has done and getting the same result, because that is considered boring and won’t help sell the publication. Yet the scientific process relies on a measured outcome being shown to be reliable through repetition. This has led to faulty papers not being properly tested because if your test shows the same results, you don’t get published, which risks your career.
Some humans will accidentally misrepresent their data via accidental P hacking. P is a symbol used to represent the probability that your experimental result was not just chance. Having a good P number (small) doesn’t mean that your experiment was right, just that it is unlikely to be a chance result (not random). That is, an experiment that has an interesting result, but could just be random, is not informative. An experiment that is unlikely to be a random result may be informative, but has yet to be shown to be likely accurate due to lack of other repeats agreeing with this result, is more likely to get published. Changing how we do the experiment can change the P value, accidentally making the result seam more plausible than it really was, and thus wasting our time thinking we have an interesting result when what we have is a circumstance that biases what we think is random versus not random.
An excellent source for learning about P Hacking is by Science Based Medicine [External LINK]
For example, let us say I have two teams competing in a sports ball competition. I can predict that my team is better, and if I stop the game as soon as my team is beating your team, then I will win that prediction. Clearly my team was shown, in this experiment, to be better than yours. Except that the chances of my team beating your team at some point is close to 100%, unless there is a huge disparity in the skills between the teams. That isn’t a good result, that is a bias of statistics. On the other hand, if we both agreed at the start of the experiments that an hour is long enough for the better skilled team to make themselves known through the score, then whoever has the higher score at the end of that hour is the superior team. I don’t get to change the time of the competition to suit my goal – we see what the teams can do.
There are many ways that scientists may accidentally change a criteria which invalidates the results.
There are also many ways that scientists may intentionally bias the results, or rather commit fraud, so that they can get paid, or praise, or further an ideological agenda. After all, scientists are human, and humans can have human weaknesses.
A classic example of this is when Andrew Wakefield published in The Lancet 1998 a fraudulent paper, now retracted, claiming that vaccines caused Autism. Wakefield’s results were poor at best, and investigation showed that they were fraudulent. The general consensus of scientists in the field at the time were that Wakefield was wrong, and within 1 year, Wakefield’s claims had been fully checked, tested and found to be false. Unfortunately, many people believed Wakefield’s fearful conclusion which has led to many people dying from preventable diseases. Wakefield is very responsible for those deaths, and the continued deaths and illness caused by people catching preventable diseases from refusing other vaccinations.
While science was able to quickly correct the erroneous results created by Wakefield, the public zeitgeist errors and life harm are still being felt by millions of people from this fraud.
How to Use Scientific Knowledge Properly
Scientific Consensus
Science has got big and complex. As such, there are many fields of science, and scientists will specialise in a field for most of their career. They will get very good at that particular field and have a passing knowledge of other fields. Based on what the experiments in that field have shown, a general consensus of what Scientific Truth is for that field will be formed – the best truth we have right now that is considered to be the closest to Real Truth.
On average, if you are going to bet on something being right or wrong, if you bet with the Scientific Consensus, you’ll almost always win.
As science progresses, that consensus opinion becomes more refined and closer to Truth, and only rarely does it say “we were very wrong” and do a major course correction.
Often we view media that says “scientist claims that all scientists are wrong about X”.
- The Wrong Expert: Some of the time, the media has a scientists giving an opinion that is outside of their area of expertise. In these cases, ignore that opinion. It isn’t an expert opinion. For example, I’m an expert on mental health, so my expert opinion should be about mental health. If I give you advice about your car, it isn’t an expert opinion. It is a general opinion and shouldn’t be taken too seriously. We often think because this person is a scientist, they are an expert in all of science, and that just isn’t the case at all.
- Untested New Research: It takes time for new research to be properly tested by other scientists to confirm or deny that the new claim is accurate. Consider how often you have seen this claim, only for that claim to disappear, slinking off into the night and hoping that no one paid attention? That is because follow up research showed that there was an error in that paper. Pro tip: wait a few years before investing in the claim.
- Click Bait: Often what the title and general article claim the experiment showed is not at all what the experiment showed. The mainstream article, paper title and paper abstract are rarely written by the authors of the paper. Only around 1/3 of the time are the paper title and abstract in line with the paper.
Scientific Consensus changes slowly, because good science takes time to do properly.
When Scientific Consensus is Wrong
While betting on the consensus of expert scientists is generally the right call will win most bets, Scientific Consensus can be wrong. All that it takes to show that an idea, hypothesis and Theory are wrong is good evidence that shows that it is wrong.
It is rare that such evidence shows that the entire basis for the current scientific knowledge in that field is wrong. Humans have been doing good science in most scientific fields for around 400 years, and excellent science for the last 100 years. In the very early days of good science, it did not take much evidence to topple a long held superstition that was considered to be good science. These days, so much testing in many fields has occurred that it is highly unlikely that scientific understanding on a broad area has been fundamentally wrong.
What does happen is specific subfields of scientific research show that our understanding is incomplete. For example, Isaac Newton published his physics of motion in 1687, and this was tested over and over again without fail for centuries. An anomaly in the timing of the orbits of Jupiter’s inner moons compared to what we expected based on Newton’s physics indicated that something was wrong. This was solved by Albert Einstein when he published his Postulate of Relativity in 1905 (known as General Relativity). This did not show that Newton’s physics was wrong, just limited. Newton’s physics works well when gravity is weak and speed is slow, while Einstein’s physics regardless of mass and speed, but are difficult to use for normal human environments.
Scientific Consensus can be wrong. When Einstein published his relativity, showing that time was not invariant, many people were upset. A letter with 100 scientists signatures was written against Einstein’s paper. Einstein retorted “to defeat relativity one did not need the word of 100 scientists, just one fact.” Or as Richard Feynman pointed out, “It doesn’t matter how beautiful your theory is, it doesn’t matter how smart you are. If it doesn’t agree with experiment, it’s wrong.”
This needs to be remembered in the context of Carl Sagan’s quote: “Extraordinary Claims require Extraordinary Evidence.” For you to show that the Scientific Consensus is wrong, you need to be able to show your evidence for it, and it has to actually work reliably and consistently. Often the claims that try to overturn scientific knowledge and consensus falter under scrutiny because the evidence was fiction or mistaken.
One of the beautiful features of science is that when someone makes a claim, so long as that claim has some semblance of substance, it will be checked and rechecked to see if it is true. When it is found to be false, the claim is shown to be an error or the claimant fraudulent, and if the claim turns out to be true, then scientific knowledge is updated. That is, on average, Science corrects itself towards greater accuracy and Truth most of the time.
What is odd is that the few times that science becomes less accurate, or realises it was wrong and corrects itself is used by grifters to try to undermine the process rather than highlight that this is evidence that the Scientific Process is working.
Reading a Paper
First tip: don’t read a paper. To get a good idea of what is going on, you mostly need to read several papers. If you aren’t familiar with the topic, read a few meta analyses of the question you have.
Know the difference between an test study, which will have a small sample size and a limit to how much follow up they did, and a more extensive series of studies, which will have a collective large sample size and consistent results. Know the difference between a controlled experiment, where confounding variables are well controlled for, and a retrospective analysis, where we are looking only to see if there is a pattern that might connect two or more elements.
When you read the paper, you want to consider the sample size, what was included and excluded in the paper, what variables were controlled, if the paper actually tested what they thought they did, and if the conclusions matches the results. You’d be shocked at how often this doesn’t happen, especially in psychology papers. Even a poorly written paper can have some interesting ideas for you to consider and do some research on. Some papers are too faulty to take seriously. A good measure of reliability is the citation count, but watch out for controversial papers being over cited.