“Be kind, for everyone you meet is fighting a hard battle” - Often attributed to Plato but likely from Ian McLaren (pseudonym of Reverend John Watson)

Sunday, July 03, 2011

Hierarchy of qualifications to evaluate research-like information

I habitually listen to Radio Paradise when working (or playing) at the computer. The station's web site is equipped with a comment facility for the music Bill Goldsmith plays, and with listener fora where all manner of topics are discussed. There's a denizen of the fora with screen name "nuggler" with whom I've exchanged barbs in the song comment areas. In the fora, nuggler is about as rabidly anti (Israel, "big Pharma," American military, Republican, "big oil," etc.) as anyone with whom I've ever interacted.

Nuggler is also fairly tin foil hat conspiracist with respect to HIV-AIDS, GMO crops, "allopathic" (i.e., done by actual doctors) or "Western" medicine as opposed to CAM (complimentary/alternative medicine), etc. He believes the strain of E Coli recently determined to be responsible for multiple deaths in Germany was genetically engineered. He had a long rant about one Dr. Stanislaw Burzynski who has supposedly developed an effective cancer cure involving "antineoplastons" that big Pharma and the FDA have brutally squelched.

So, I read a little bit about Dr. Burzynski and some of the material regarding him from his own site and from sites like Quackwatch and Science Based Medicine. I'm obviously ill-equipped to digest material from the primary medical literature both by background and by opportunity cost of time. So again, I have to decide whom to believe. I went into this a bit in a previous post regarding Anthropogenic Climate Disruption. Thus, it's back to a "who seems most credible" situation which is not at all easy (or at least it hadn't ought to be).

So I decided to propose a little hierarchy of how much confidence to have in one's ability to trust one's interpretation of research conclusions (especially conflicting conclusions) based on background and proximity to that research. This is what I came up with, I welcome suggestions and criticisms.

  1. I did the research.
  2. I am an expert in the specific subject area of the research and, though I didn't do the research, I have fully read and understood it.
  3. I have some specific knowledge in the particular area and deep knowledge in the general area of the research.
  4. I have a reasonable background in the general area of the research.
  5. I have done formal research, but not in any area related to that being evaluated.
  6. I have studied this area in depth, but not in any formal venue.
  7. I have completed a formal program of study though not one related to the topic at hand (this shows, at least, some ability to absorb and synthesize complex information).
  8. I have evaluated the coherence of the arguments propounded with respect to agreeing and disagreeing viewpoints.

These levels are not necessarily mutually exclusive. But, if I'm at any level less than this, the fact is that I simply should not allow myself to have an opinion. "It fits the way I'd like the world to work and believe it to be" simply will not suffice.

No comments: