Pseudoscience is the term given for any belief or practice that is mistakenly based on the scientific method. The scientific method is an iterative process that includes systematic observations, measurements and experiments to test an idea / hypothesis for its accuracy. When an idea is considered accurate enough, it is considered to be scientific. People who attempt to pass ideas and practices as scientific which have yet to show rigorous evidence of accuracy via the scientific method are pushing pseudoscience. These people may be naïve, true believers or charlatans. Pseudoscience is harmful as at the least it parts people from their money, and at worst can lead to people’s untimely death. Unfortunately, in the health industry, people are often looking for simple solutions to complex problems and so are prone to falling for pseudoscience. This can be both the person seeking to resolve their own health issue, or practitioners offering solutions.

Pseudoscience List

In this section are the links and brief descriptions for the therapies that I classify as Pseudoscience.

Following this list is the explanation about what Pseudoscience is, and why that is different to real science.

Scientific Method Versus Pseudoscience Claims

Understanding the Scientific Method

Science is not an ideology, religion or set of facts.

Science is a process for trying to determine a more accurate Truth of what is Real.

I have capitalised the T in Truth as a reference to an ideal philosophical objective Reality Truth.

We have zero evidence that such a TRUTH exists, however, for the scientific method, we are aiming to get as close to that philosophically objective Reality Truth.

That is, a consistent set of fundamental rules to the entire universe for all time. When I use a small t truth, then I am referring to an approximation of that fundamental Truth. Science aims for Truth, but cannot prove it has uncovered it. Science moves from one truth, to the next more accurate truth, aiming for the Truth.

The capitalisation of Real is similar, referring to that ideal philosophical objective Reality Truth.

What matters most is what is the Truth, and as we hunt for that, separating out False ideas for more true ideas, always approaching the ideal Truth.

Science is Done by Scientists, who are people, thus it isn’t perfect.

Even though science is a good process, it is a process that is done by people, so we do have to contend with the flawed human factor. While there are problems with how we implement the scientific process, it is still the best method we humans have developed for understanding the difference between an idea that is false / inaccurate and an idea that is more accurate.

Science is Making Models (Hypotheses) of the World

As part of the scientific method, we construct a model for how we think something in the real world works. This will, by necessity, be a simplification of how it truly works, as we are trying to construct a rule that explains a range of similar phenomena in such a way as to predict what is going to happen next. In science, we are looking for the most accurate model that is the most useful (accurately predictive).

The model is trying to create a generalisable scientific principle that is testable, repeatable, and predictive (accurate/useful). If the model is not generalisable (can be applied in more than one situation), note testable (we can’t prove it works or doesn’t work), repeatable (works all the time – one could be a coincidence), or predictive, then it is not a good scientific model.

“Essentially, all models are wrong, but some are useful”,
George E.P. Box, Empirical Model-Building and Response Surfaces.

George Box, a British statistician, working on quality control, time-series analysis, design of experiments, and Bayesian inference. Box helpfully points out that what we want is a useful model, even though we know it isn’t 100% correct. Box’s statement also points out that some models aren’t useful, and thus shouldn’t be used (eg Pseudoscience).

Don’t settle for models that are useless, and don’t be fooled by models that have yet to be properly tested.

Good Science: Formulating a ‘Hypothesis’ to ‘Explain’ ‘Phenomena’

A good hypothesis (model) will explain a mechanism for what is happening such that you can make predictions for what should happen next, and what would happen if you changed a certain something in that process.

This leads to testable predictions. The scientist (person who tests these ideas using the scientific method) will then perform experiments to test the predictions.

If the hypothesis is not accurate enough, modifications may be made to try to improve the accuracy of the hypothesis. If the hypothesis is inaccurate enough, then it will be discarded. If the old hypothesis is inaccurate, it is often discarded, but like our gravity example above, sometimes both are kept, but we know which is the superior one to use when things get complicated.

Science is Repetitive and Adversarial

Generally, there are several models (ideas / hypotheses) that are being pitted against each other, and the scientific method is attempting to reveal which of the models are more accurate, more useful, and thus preferable.

The scientific method is also often showing which ideas are based on flawed mechanisms and we should discard those.

Adversarial Models

Let us say we have two hypotheses (models) for the observed phenomena, Model A and Model B. We want to test those against each other and see which is more reliable at predicting the phenomena.

Just because an experiment shows that Model A is more accurate than Model B does not mean that Model A wins. That is merely 1 data point.

Good science relies on replication, ideally by someone else, somewhere else, using a similar method and equipment. If their experiment also concludes that Model A is more accurate, we now add a level of credence that Model A is more accurate, and that the experimental result was not a fluke (not a coincidence, rather it was a causal chain [cause and effect]).

Adversarial Peers

While each scientists is motivated to be “right” and publish their results, they are also very motivated to point to any scientists who has made a mistake and helpfully (and or gleefully) point out the error(s).

You have to prove to your peers that you are right, or they will tell you why you are wrong.

Your peers can’t just say “you are wrong” because they don’t like your hypothesis though, they have to point out your actual errors and give evidence – such as the results of their experiment for testing the same thing.

“When I was doing Professor Albert Einstein’s bust he had many a jibe at the Nazi professors, one hundred of whom had condemned his theory of relativity in a book. ‘Were I wrong,’ he said, ’one professor would have been enough.’”

Jacob Epstein, Let There Be Sculpture, 1940.

None of the professors gave good reasons why Professor Einstein’s theory was wrong, they just didn’t like it [Wikipedia].

Peer Review & Plausible Mechanisms

Various experts will also look at the methods that were used to test the models to determine if these were fair tests, a plausible model, and that the results are not flawed. The results could be flawed if the method measures the wrong thing.

This is called Peer Review.

It is very important that the proposed mechanism is realistic (using what we know to be real) and plausible (seems to explain the process and results). This is where a great deal of pseudoscience immediately fails the sniff test.

For example:

Hypothesis: “People wearing red shirts will run faster”.
Mechanism: Red cars go faster, so red shirts should mean runners go faster.
Test: A running race is held with various runners wearing various coloured shirts. If the winner is wearing a red shirt, then the hypothesis is correct.
Result: The winner of the race wore a red shirt, so therefore the hypothesis is correct.

Probable errors:

• Mechanism:
  • Is there any mechanism proposed for why a red shirt would change the speed of the runner?
  • The idea behind “red cars go faster” is that a few manufacturers of fast cars painted their cars red. The speed of the car was about engineering of the machine, not the colour of the paint. There is no good / plausible reason why wearing a red shirt would mean the runner would run faster.
• Test:
  • If there was only 1 runner, then of course they won, the above doesn’t define how many runners there were.
  • What was the ratio of red shirts to not red shirts?
    • If they all wore red shirts, or 50% of the runners wore red shirts, this is not really testing the red shirt.
  • Was the expected fastest runner always assigned a red shirt prior to the race?

While this is a farcical example, many terrible studies that got past peer review have literally been this bad.

Science is Iterative

Science continues to iterate (test, modify, test, modify, then test again), becoming more accurate and useful.

Even if a model may be shown to be a better predictor, with a reasonable proposed mechanism and it may be replicated, there is room for even better science and more accuracy.

Good Science Convergese

When different fields of science come to a similar conclusion, then we have a convergence of hypothesis pointing to a more fundamental layer of Truth. When different streams or branches of Science converge on the some conclusion, these ideas are considered to be far more accurate to Truth than hypotheses that don’t yet.

For example:

Einstein’s General Theory of Relativity has been verified from many different lines of enquiry, showing that it is not just a good model for certain predictions, but that it is an excellent model for how the universe seems to work. Even so, we know that it is not complete as it does not describe quantum events. Even though we know it is incomplete, we don’t know what needs to change to make it even better.

Flawed Models: Not losing the baby with the bathwater

A model may be flawed or even wrong, yet it may point to another model that is useful, or the fault in the model can be modified to remove a discovered error.

For example:

Data: Blood tests from people diagnosed with Depression found a consistently low presence of Vitamin D.
Hypothesis: Low Vitamin D causes Depression.
Predictions: Two predictions can be formed from this model.

1. One is that raising the level of Vitamin D will decrease measured Depression,
2. All people with similarly low levels of Vitamin D will be noticeably Depressed.

Experiment:

1. Supplementing Vitamin D such that the test subjects tested “normal” levels of Vitamin D in their blood, test for depression.
2. Test more people and see if all people with low Vitamin D were depressed.

Results:

1. No change in depression upon supplements.
2. People with low Vitamin D did not necessarily test as “depressed” using the usual metrics.

Conclusion: This effectively destroyed the hypothesis that low Vitamin D causes Depression.

These experiments were replicated (repeated), and the logic of the outcome is sound. Low Vitamin D is a marker for possible depression and their may be a connection that is not direct like the hypothesis first predicted.

A hypothesis that I propose:

Since Vitamin D is made out of LDL cholesterol, and LDL cholesterol is also used to regulate the production of Dopaminergic Neurotransmitters (Dopamine, Noradrenaline and Adrenaline) via the hormone Oestradiol (Oestradiol is an end product of LDL cholesterol), then problems in the processing of LDL cholesterol can lead to both low Vitamin D and problems in synthesising sufficient Dopaminergic Neurotransmitters, which is a leading contender for Depression.

At this point, this is just a hypothesis since to the best of my knowledge, there is no study to investigate this.

I need to ensure that this hypothesis is just presented as an idea until the science is done to show that it is fact or fiction.

Since low Vitamin D does seem to be a common marker, perhaps there is a mechanism that shows why this correlation is high. The hypothesis that I have proposed makes logical sense, and it may be right. However, the idea that low Vitamin D caused depression also seemed logical, but was wrong.

“It doesn’t matter how beautiful your theory is, it doesn’t matter how smart you are. If it doesn’t agree with experiment, it’s wrong.”

by Richard Feynman.

Pseudoscience skips the Science

Pseudoscience skips the Science and pretends the claim is true:

• Often with inplausible mechanisms,
• Often with no testing of the hypothesis (no peer reviewed published papers),
  • Or very poorly constructed or run tests that show strong bias in the results (that got past the peer reviewers),
• Faulty math on the results,
  • Or falsified data,
• Conclusions that don’t match the results.

These seem to be almost always motivated for money or power.

Scientists are also motivated by money. However, good scientists recognise that they get more money by discovering things that are true, because at some point their lies will be uncovered by other scientists and they’ll be discredited and lose too much if they lie. Sometimes, this does happen, but it is rare.

If I were to use my hypothesis (about Vitamin D), an untested theory, as a statement of scientific truth, especially if I use that to promote a medical treatment for depression, then I would be engaging in Pseudoscience.

Just because it makes medical sense does not mean it is correct, nor does it mean it is wrong. I just means it isn’t scientifically tested, and any claim to say “this is how it works” that misses the fact of “no evidence yet” shows it to be pseudoscience. A more accurate statement is “this is how it may work”. Unfortunately, accurate statements like “this is how it may work” don’t sell many products, and people often buy false certainty over cautious accurate “may”, “probably” and “in these circumstances it does”.

Pseudoscience Belief and Practices.

For a belief or practice to be considered Pseudoscience, it has either yet to be tested scientifically, or it has been shown to not be scientifically accurate (proven not to work). I’ll add in a third option – it has been shown to be less effective or accurate than already available methods.

“By definition, alternative medicine has either not been proved to work, or been proved not to work.
You know what they call alternative medicine that’s been proved to work?
Medicine.”

Tim Minchin, from Storm.

Put another way, ideas (hypotheses) that have withstood the scientific method are considered to be in the large space of provisionally passed ideas called ‘scientific‘ or ‘science based‘, while those that have no reliable evidence showing that they are correct; have been discredited, yet are still being promoted as scientific, or are less accurate or effective than existing ‘scientific‘ versions; are called Pseudoscience.

That leaves the dubious term of ‘Evidence Based‘, which we will discuss next.

For a more comprehensive breakdown on good science and its limits, check out Understanding Science.

The Obfuscation of Evidence Based

We just talked about how ‘good science‘ has ‘good experiments‘ to test a reasonable mechanism led hypothesis / model, and that we ‘don’t consider it evidence without replication‘. We also noted that scientists are human, and sometimes humans make mistakes – sometimes genuine, sometimes self serving.

It unreasonable for regular people looking for health therapies to do the research themselves and figure out if a particular therapy is legitimate or pseudoscience. That burden should be taken up by the experts that the regular person is seeing. After all, part of the education for university degrees in applicable fields is supposed to include formal education on both how to write and execute your research properly, and thus how to spot when a study you read has.

In my opinion, far too few experts retain and exercise this critical thinking. Far too few experts do the work themselves, spending the time and energy to read the current up to date research, to figure out what is legitimate and what is not. To be fair, most of us are busy and rely on someone trusted to do that for us – such as Peer Review, Meta Studies and things that are labeled as ‘Evidence Based‘.

To try to distinguish between misleading claims of efficacy and scientific claims of efficacy, the statement of ‘Evidence Based‘ was proposed. It was supposed to mean ‘scientifically tested and shown to be useful/right/accurate‘. ‘Evidence Based Therapy‘ is used to show that there is valid evidence behind the claim that ‘this treatment should be expected to work for that condition‘.

Good researchers learn how to consider the quality of research and differentiate between a ‘good study‘ versus a ‘flawed study that snuck through‘. Not all research is created equal, and too many charlatans have created false or flawed studies to fabricate ‘evidence‘ that their therapy works, or scientists have conned themselves that their idea is good and accidentally create flawed studies that ‘provide evidence‘ they are ‘right‘. Studies that have been published in a reputable journal should have been ‘peer reviewed‘, where experts in the field try to weed out fraudulent or faulty studies. Unfortunately, far too many poor quality studies sneak in.

The best lies have a kernel of truth wrapped in a smothering of lies. That has led to be erroneously thinking that if any study is ‘Peer Reviewed‘ and shows that the ‘therapy works‘, then that means it is ‘Evidence Based‘. This has effectively turned the phrase ‘Evidence Based‘ to become more or less meaningless.

An argument is rising to create a new standard, called ‘Science Based’ medicine or therapy [source]. This acknowledges that to fix ‘evidence based’ is now too hard and we need to effectively start again. While I think this will work for a little while, it won’t be long before those who broke the ‘evidence based’ model will do the same thing to ‘science based’. I anticipate that the AI revolution is going to make this a lot worse.

To save you some time, I have compiled a list of common therapies that are pseudoscience, and why they are regarded as such. Reading through the why will help you figure out how to tell if a treatment falls more into ‘scientific’ or ‘pseudoscience’.

Quick Red Flag Tests for Pseudoscience Treatment

• Anecdotes and testimonials instead of quality research evidence.
• Appeal to nature eg: ‘it’s natural’.
• Buzzwords:
  • Ancient / Traditional / Native / Folk Treatment / Alternative/ Complementary / CAM / Holistic.
  • Energy (without a description of type).
  • Quantum, Quantum Properties, Quantum Effects, Entanglement, Superposition.
• ‘Burden of Proof’ is misplaced.
  • It is on the claimant to prove it works, not on you to prove that it doesn’t.
• Invocation of the mystical:
  • Magic / magik / Rune etc.
  • Spirit / ki / chi / qi.
  • Life energy / Life force.
  • Meridians.
• Mainstream large organisations don’t use it (other than patented).
• Panacea:
  • Cures / fixes / prevents everything.
  • Explains everything.
• Problematic Research:
  • Mostly poor quality.
  • Published quality research shows weak results.
  • Significant disagreement about the results of the research.

Anecdotes and Testimonials

The claims of efficacy rely mostly on anecdotes and testimonials that this treatment really works, rather than testimonials at the professionalism of the therapist.

The plural of anecdote is not data. It can be a beginning point for why a research study should be done for that claim, but it is not evidence that the claim is real without the research to back it up.

Appeal to Nature, Natural

Appeal to Nature is a logical fallacy, where the arrangement is that something is good, valid, or justified simply because it is ‘natural’, or bad because it is ‘unnatural’ / ‘human made’.

The false premise is that ‘natural’ things are inherently safe or morally superior.

This leaves us with the question ‘what does a supernatural thing look like?’, as if it is not supernatural, then it is natural, even if it is man made.

Buzzwords

Ancient / Traditional / Native / Folk Treatment / Alternative/ Complementary / CAM / Holistic

If the words Ancient or Traditional are used to describe the treatment, it is probably pseudoscience.

Medical treatments are updated all of the time as newer and better ways to provide the therapy are discovered. The fact that this treatment is ancient and unchanging is a big red flag. Good scientific medicine progresses and evolves.

Energy

Energy is either

• Atomic (inside an atom)
• Chemical (between atoms)
• Thermal (kinetic of a group of molecules)
• Potential (kinetic energy that can be released)
• Kinetic (the movement energy that has been released)

Energy with no type affixed to it is not a scientific concept. If I have spend most of the paragraph referring to kinetic energy, then I might skip the kinetic for that paragraph, since the type was already specified. It is bad practice to refer to energy without specifying what kind.

If someone is trying to talk about how you get energy, or manipulate energy, but doesn’t specific which kind, then active your critical thinking.

Psycho Babble

Words that sound like they mean things, but either:

• Don’t actually have any useful definition.
• Ultimately describe a thing that has no proof of existing, without any distinction from that original meaning.

Quantum, Quantum Properties, Quantum Effects, Entanglement, Superposition

Burden of Proof is Misplaced

If I make a truth claim, it is my burden of proof to show that it is true or acknowledge that I cannot.

Pseudoscience will often shift the burden of proof to show that their claims are false, rather than showing evidence that their claims are true.

Not all evidence is weighted equally. See Understanding Science for more.

“Extraordinary Claims require Extraordinary Evidence.”

Carl Sagan

Invocation of the Mystical

Magic Energy, Ki / Spirit / Chi / Qi / Life Energy / Life Force / Meridian

No scientific investigation has found this energy (the name changes often).

Scientific tools can measure the ripple of gravitational waves that chance space time as little as 1/10,000 of the width of a proton, can measure the movement of water inside of a body, can look at bones without surgery and so on, can detect electromagnetic signals that are vast in range, but can’t seem to find the spirit energy that many pseudoscience treatments claim to treat with magnets, bone manipulation and needle points.

Mainstream Large Organisations Don’t Use it

Western Hospitals provide the best recommended treatments for health conditions.

While a new therapy may not be available yet in a specific hospital, that same treatment is not likely to be available in a small private hospital either.

• If a health expert is claiming that their new treatment works but hospitals aren’t providing it, be careful and check for further signs of pseudoscience.
• If a health expert is claiming that this ancient or old treatment works, but hospitals aren’t providing it, it’s a scam.

There are treatments that exist that are not suitable for large hospitals for certain conditions. When this is the case, these treatments provided in clinics are considered to be ‘The Standard of Care‘ for that condition, or is listed as a common treatment for that condition.

Unfortunately, many pseudoscience treatments have manage to weasel their way in to ‘Standards of Care’. Check our list below to see if the treatment or type of practitioner is listed in Pseudoscience Therapies.

Panacea: Cures / Treats / Prevents / Explains Everything

No single treatment treats everything. If it did, it would be a part of everyone’s required daily existence actions – like breathing both in and out, eating, having your heart beat.

When a treatment is claimed to treat many different unconnected or poorly connected conditions, it is time to get suspicious. A good test question is “does this treatment also treat …” and list something not connected to any of the conditions listed on the promotional material. Better acceptable answers are “no”, or “I don’t know, let me go and check”.

If there answer is a quick “yes”, then get very, very suspicious and activate your critical thinking.

Problematic Research

For the layperson, the terms “research, studies, trial, experiment, case study, field study, evaluation and white paper” are interchangeable. These all have specific meanings which approximately align in various fields of scientific enquiry (different fields have slightly different expectations of what the term means, but it is generally close enough).

What we are referring to is an intentional formal study into whether a claim is accurate or not, generally with an experiment that adheres to certain levels of quality.

Rating single and multiple papers for quality guide.

The basic criteria for good research is:

Rating an individual research Paper;

• Quality:
  • Is the paper listed in a good (scientifically respected, peer reviewed) journal?
  • Is the paper relatively recent
    • Last 5-10 years preferred
    • Additional scepticism should be applied for every decade added.
• Introduction:
  • Is the hypothesis clearly stated:
    • What is the idea that this research is testing?
    • Why is this idea important.?
    • What should we see:
      • If the idea is true?
      • If the idea is false?
    • What does the experimenter expect the outcome to be?
  • Inclusions / Exclusion:
    • Clear exclusion criteria, which defines the boundary of the hypothesis
    • Inclusion numbers are sufficient for the type of research (which can vary from a few to thousdands+)
• Method of the experiment:
  • How will the hypothesis be tested?
    • If it is true.
    • If it is false.
  • How other possible confounding variables will be mitigated or addressed.
  • What confounding variables couldn’t be mitigated or addressed.
  • Negative credit:
    • Not double blinded (if testing people) without a good explanation for how they excluded participant bias.
    • Test group isn’t compared to a placebo.
    • The study is performed by those who benefit from a positive outcome.
    • Extra credit:
      • The method is published prior to the study being done (this is a new standard, studies older than 2020 aren’t held to this standard).
      • Minimal assumptions are made to prior to the study (avoiding begging the question logical fallacy).
• Results:
  • What was the raw data.
  • What statistics were applied to the data, and why?
  • What does the statistics show? (raw data is meaningless on its own.)
  • Is this result significant?
    • What is the chance of getting these results by chance?
    • If this result is unlikely a fluke, then does it support or contradict the hypothesis?
    • Is the results clear and strong, or ambiguous, or weak?
• Conclusion (and optional Discussion):
  • What do the authors think may have confounded their results? What would they suggest needs specific further research?
  • What does this study mean?
    • If the results are significant and meaningful, what might this result affect?
    • Does the speculation match the hypothesis and results? Does it seem reasonable to conclude that?
  • Does this match the results and the abstract?

Comparing Papers / Multiple papers;

• Manual or Experts:
  • This can be done manually, by separating quality and poor papers. See the next bit to get an idea of how to do that.
  • This can be a meta review / meta analysis or systematic review (they do the assessment of quality and will explain what ‘inclusion’ looks like).
• Good signs:
  • If it has been released in the last 5-10 years, then you can likely trust it.
  • Excluded papers are due to not meeting the specific criteria questions rather than remarked consistent poor quality.
  • It agrees with other recent meta reviews, or has a good reason why it doesn’t.
  • Extra Credit:
    • Large sample size:
      • Either in each experiment, or overall combined numbers recalculated from the raw data.
      • Large number of ‘good papers’ to compare.
    • Intelligent design avoiding confounding factors.
    • Clear signal.
• Concerning:
  • The ratio of papers rejected for inclusion was over 50% due to issues with quality, yet the meta analysis finds the proposed ideas plausible.
  • The number of included papers is lower than 40%, yet the meta analysis is glowing of the outcome.
  • Number of participants is low.
  • Lots of self report.
  • No discussion about excluding confounding factors and steps to avoid including or factoring in those.
  • Method does not seem to lead to the conclusion.
• Assessing the papers:
  • Do the papers agree with each other (approximately)?
  • If they disagree with each other, is it 2-3 factions (camps of view), or more than 3?
    • If 2-3 factions, are the authors being polite and fair to each other, or unfair?
    • More than 3 factions is a warning that this field is in flux, and so there are no solid leads to truth yet.
  • If they agree, does that agreement include:
    • A plausible explanation?
    • A plausible mechanism?
    • Consistent outcomes?
  • Comparing ‘good’ and ‘poor’ papers:
    • What is the ratio of ‘good papers’ to ‘poor papers’?
    • Do the ‘good papers’ report a strong effect, a weak effect size, or a inconsistent (between papers) effect size?
    • Do the ‘poor papers’ report a strong effect, despite the average result of the ‘good papers’?

For More: See ‘How to read a paper‘ in Understanding Science.

Pseudoscience Therapy Types