Autism Biomedical Information Network



What is "good" research and "good" science?


Overview

Why is research done?
Anecdotal reports are not good enough
What is an unproven treatment?
The process of research
The plausibility test
Research as hypothesis-testing
Treatment must be evidence-based
"Aconceptual" versus conceptual-based research

Why is research done?

Since research is at the center of the process of learning about our world, it is important that parents understand how "good"research is done. We depend on the accumulated store of knowledge, experience, and wisdom of our civilization to guide our daily lives. Knowledge provides structure and predictability as we make sense of the cacophony of images that assault our senses. It enables us to "reality-test" so that we can distinguish that which is real and factual from the chimera of our imagination. Research is the process our civilization uses to build further on our storehouse of knowledge. Research is the process of discovering new information based on evidence collected in ways that minimize observer bias. Research is imperative if there is to be any hope of more effective treatments and behavioral interventions for autism.

Return to Overview

Anecdotal reports are not good enough

Parents, especially ones who have just learned of the diagnosis of autism/PDD in their child, are often exposed to a steady barrage of unproven treatments. Claims are made for treatments that are based solely on anecdotes by parents. There is inherent bias in collecting impressions of parents as to a particular intervention. Parents are not gene rally in a position to be truly objective observers of their own child's progress. There is a natural tendency to read more than is warranted into a minor change in behavior, especially if it occurs coincidentally with the introduction of a new intervention. Moreover, there are changes in behavior that occur naturally over the longitudinal time- line just as they occur during the course of development in typical children. The universally accepted standard of good research is to have observations made by truly impartial observers.

Return to Overview

What is an unproven treatment?

The tools now available for research enable us to move beyond mere anecdote. What is an "uproven treatment?" One for which there are no data supporting a claim of efficacy. This usually means that appropriate data have not been published in a journal using an editorial process of peer-review of manuscripts by disinterested referees (referees are persons specifically designated as impartial reviewers of manuscripts and are at peer level in the profession). It may also mean that there are conflicting data in the published biomedical literature such that definitive conclusions are not possible.

Return to Overview

The process of research

A research project begins with the seed of an idea, often in the mind of a person (a basic scientist or clinician) who has done other studies in the field. The idea may have germinated from a seed planted by research done by others. Publications, scientific meetings, and personal communications among scientists and clinicians contribute to the cross-fertilization of ideas. These ideas occur to research-minded persons with a background of experience in a particular field by dint of a little-understood process of intuitive creative intelligence. Sometimes it is a "eureka!" realization. At other times, it is one small step in the slow, plodding process of building knowledge, layer by layer. Researchers are innately curious about how things "work."

Return to Overview

The plausibility test

The exciting research currently unfolding in autism is being done by individuals with a solid background in neurobiology, cognitive neuroscience, genetics, or psychobiological aspects of behavior. This background gives them the ability to do the first "acid test" of a proposal for research even before actual laboratory or clinical research begins: the test of plausibility. For example, a neuroscientist has an intuitive sense about the plausibility of ideas relating to how the brain "works." There is only a certain range of possibilities. Not everything is possible. On the other hand, the average lay person is not able to bring the same depth of background knowledge of the understanding of basic brain mechanisms to a consideration of plausibility. In the lay person's view the range of possibilities may seem limitless.

Return to Overview

Research as hypothesis-testing

Once an idea "passes" the initial plausibility test, a researcher then sets out to formulate a plan for research. The first step is to state the hypothesis to be tested. Next, the methods to be used to make observations and gather data are described in detail. These must be in accord with a well-designed research plan that takes into account possible bias of the observer. The research plan includes a way for controlling for bias (such as random assignment of individuals to treatment and nontreatment groups; tests showing that individuals in either group are otherwise comparable; double-blinded, so that those who know the allocation scheme of subjects to the test or treatment groups are not the same persons as those who make the observations and measurements). Data collected from such observations, if appropriate, need to undergo statistical tests to determine if there are real differences (i.e., that differences between the groups are not likely to have occurred by chance alone). If significant differences occur, then the investigator is able to conclude that the hypothesis being tested is supported by the data. In effect, a new bit of knowledge has been created.

Ethical scientists submit a report of their research to a peer-review jounal. The journal then sends the article to one or more impartial referees, who have knowledge of the field, to critically review the manuscript as to suitability for publication or the need for revision. It is also the mark of a good investigator, before submission of a manuscript, to report findings to a meeting of an appropriate professional society. This also provides a forum for peer-review and can often contribute further ideas for study or discover flaws in the research design and the need to "go back to the bench."

Return to Overview

Treatment must be evidence-based

The above is an outline of the usual process of research by an ethical investigator as an idea wends its way from an initial "seed " to final flowering as new information that contributes to the overall body of knowledge. This process has resulted in the current standing of the U.S. biomedical establishment as pre-eminent in the world. It is a result of insistence on the highest standards of research. It demands that a proposed treatment for a disorder be subjected to this rigorous process before it is then possible to conclude that the treatment is either evidence-based or quot;not proven" to be effective; or that further study, in accord with good research design, is needed.

Return to Overview

"Aconceptual" versus conceptual-based research

"Aconceptual research" in biology is research that does not fit a pre-existing framework of knowledge and understanding of how biological systems work. It may be nothing more than a "fishing expedition." However, it is more likely than not that such research will not yield useful data. Conceptual-based research, on the other hand, is research based on a theory that flows logically out of antecedent studies and an accepted body of knowledge.

As Louis Pasteur said: "Chance favors the prepared mind."

|Return to list of commentaries| |Links to other autism resources|

Commentary by Ronald J. Kallen, M.D., ©1999
This page last updated on 2/10/99