The Scientific Method#

The Scientific Method

Overview

An example of oversimplification that confounds many students of biology (particularly early in their studies) is the use of language that hides the experimental process used to build knowledge. For the sake of expediency we often tell stories about biological systems as if we are presenting unquestionable facts. However, while we often write and speak about topics in biology with a conviction that gives the appearance of "factual" knowledge, reality is often more nuanced and filled with significant uncertainties. The "factual" presentation of material (usually lacking discussion of evidence or confidence in the evidence) plays to our natural tendency to feel good about "knowing" things but it tends to create a false sense of security in the state of knowledge and does little to encourage the use of imagination or the development of critical thinking.

A better way to describe our knowledge about the natural world would be to explicitly qualify that the knowledge presented represents our current best understanding that has not yet been refuted by experiment. Unfortunately, repeated qualification becomes rather cumbersome. The important thing to remember is that while we may not say so explicitly, all of the knowledge we discuss in class represents only the best of our current understanding. Some ideas have withstood repeated and varied experimentation while other topics have yet to be tested as thoroughly. So if we're not as certain about things as we'd like to believe sometimes, how do we know what to put confidence in and what to be skeptical of? The complete answer is non-trivial but it begins with developing an understanding of the process we use in science to build new knowledge. The scientific method is the process by which new knowledge is developed. While the process can be described with long lists of "steps" (often seen in textbooks) its core elements can be described more succinctly.

Succinct Description of Scientific Method (adapted from Feynman)

1. Make an observation about the world.
2. Propose a possible explanation for the observation.
3. Test the explanation by experiment.
4. If the explanation disagrees with experiment, the explanation is wrong.

At its core, that's it! In science there may be multiple simultaneously proposed explanations or ideas that are tested by experiment. The ideas that fail experimentation are left behind. The ideas that survive experimentation move forward and are often retested by alternative experiments until they too either fail or continue to be retained.

Making an observation and asking a question

The ability to make useful observations and/or ask meaningful questions requires curiosity, creativity and imagination – this cannot be over-stated. Indeed, historically, it is first and foremost the application of these skills, perhaps more than technical ability, which has led to big advances in science. Many people think that making meaningful observations and asking useful questions is the easiest part of the scientific method. This is not always the case. Why? Seeing what others have not and being creative takes work and thoughtful reflection! In addition, our senses of observation are often biased by life-experience, prior knowledge, or even our own biology. These underlying biases influence how we see the world, how we interpret what we see, and what we are ultimately curious about. This means that when we look at the world, we can miss a lot of things that are actually right under our noses. Douglas Adams, who is best known for his book entitled The Hitchhiker’s Guide to the Galaxy, once expanded on this point by writing:

“The most misleading assumptions are the ones you don't even know you're making.”

Scientists, therefore, need to be aware of any underlying biases and any assumptions that may influence how they internalize and interpret observations. This includes approaching our bias that the variety of places we get our knowledge (i.e. textbooks, instructors, the Internet) are representing the absolute truth with a healthy dose of skepticism. We need to learn to examine the evidence underling the “facts” we supposedly know and make critical judgments about how much we trust that knowledge. More generally, taking the time to make careful observations and to uncover any assumptions and biases that could influence how they are interpreted is, therefore, time well spent. This skill, like all others, needs to be developed and takes practice and we’ll try to start you on this in BIS2A.

For fun, and to test your observation skills, Google “observation tests”. Many of the search results will take you to interesting psychological tests and/or videos that illustrate how difficult accurate observation can be.

Generating a testable hypothesis

The "possible explanation" referred to in step 3 above has a formal name; it is called a hypothesis. A hypothesis is not a random guess. A hypothesis is an educated (based on prior knowledge or a new viewpoint) explanation for an event or observation. It is typically most useful if a scientific hypothesis can be tested. This requires that the tools to make informative measurements on the system exist and that the experimenter has sufficient control over the system in question to make the necessary observations.

Most of the time, behaviors of the system that the experimenter wants to test can be influenced by many factors. We call the behaviors and factors, dependent and independent variables, respectively. The dependent variable is the behavior that needs explaining while the independent variables are all of the other things that can change and influence the behavior of the dependent variable. For example, an experimenter that has developed a new drug to control blood pressure may want to test whether her new drug actually influences blood pressure. In this example, the system is the human body, the dependent variable might be blood pressure, and the independent variables might be other factors that change and influence blood pressure like age, sex, and levels of various soluble factors in the blood stream.

Note: For more on dependent and independent variables

on Wikipedia
on Khan

Note

In BIS2A, and beyond, we prefer to avoid using language like “the experiment proved her hypothesis” when referring to a case like the blood pressure example above. Rather we would say, “the experiment is consistent with her hypothesis.” Note that for convenience (one of the language shortcuts we discussed earlier) we referred to the alternative hypothesis simply as “her hypothesis”! It would be more correct to state, “the experiment falsified her null hypothesis and is consistent with her alternative hypothesis.” Why take this shortcut since doing so adds confusion when a student is trying to learn? In this case, it was done to illustrate the point above about language shortcuts and hence the lengthy explanation. However, be aware of this commonly used shortcut and learn to make sure you can read in the correct meaning yourself.

Note: Possible discussion

What does the statement about falsifying hypotheses mean in your own words? Why is falsification critical to the scientific method?

Controls

In an ideal case, an experiment will include what are known as control groups. Control groups are experimental conditions in which the values of the independent variables (there may be more than one) are maintained as close to those in the experimental group with the exception of the independent variable being tested. In the blood pressure example, an ideal scenario would be to have one identical group of people taking the drug and another group of people identical to those in the experimental group taking a pill containing something known to not influence blood pressure. In this oversimplified example, all independent variables are identical in the control and experimental groups with the exception of the presence or absence of the new drug. Under these circumstances, if the value of the dependent variable (blood pressure) of the experimental group differs from that of the control group, one can reasonably conclude that the difference must be due to the difference in independent variable (the presence/absence of the drug). This is, of course, the ideal. In real life it is impossible to conduct the proposed drug dosage experiment; the sheer number of possible independent variables in a group of potential patients would be high. Fortunately, while statisticians have come to the rescue in real life, you won’t need to understand the nuances of these statistical issues in BIS2A.

Accuracy in Measurement, Uncertainty, and Replication

Finally, we mention the intuitive notion that the tools used to make the measurements in an experiment must be reasonably accurate. How accurate? They must be accurate enough to make measurements with sufficient certainty to draw conclusions about whether changes in independent variables actually influence the value of a dependent variable. If we take, yet again, the blood pressure example above. In that experiment we made the important assumption that the experimenter had tools that allowed her to make accurate measurements of the changes in blood pressure associated with the effects of the drug. For instance if the changes associated with the drug ranged between 0 and 3 mmHg and her meter capably measured changes in blood pressure with a certainty of +/- 5 mmHg she could not have made the necessary measurements to test her hypothesis or would have missed seeing the effect of the drug. For the sake of example, we assume that she had a better instrument and that she could be confident that any changes she measured were indeed differences due to the drug treatment and that they were not due to measurement error, sample-to-sample variability, or other sources of variation that lower the confidence of the conclusions that are drawn from the experiment.

The topic of measurement error leads us to mention that there are numerous other possible sources of uncertainty in experimental data that you as students will ultimately need to learn about. These sources of error have a lot to do with determining how certain we are that experiments have disproven hypothesis, how much we should trust the interpretation of the experimental results and by extension our current state of knowledge. Even at this stage, you will recognize some experimental strategies used to deal with these sources of uncertainty (i.e. making measurements on multiple samples; creating replicate experiments). You will learn more about this in your statistics courses later on.

For now, you should, however, be aware that experiments carry a certain degree of confidence in the results and that the degree of confidence in the results can be influenced by many factors. Developing healthy skepticism involves, among other things, learning to assess the quality of an experiment and the interpretation of the findings and learning to ask questions about things like this.

Note: Possible discussion

After moving to California to attend UC Davis, you have fallen in love with fresh tomatoes. You decide that the tomatoes in the stores just don’t taste right and resolve to grow your own.

You plant tomato plants all over your back yard; every free space now has a freshly planted tomato seedling of the same variety. You have planted tomatoes in the ground in full sunlight and next to your house in full shade.

Observation: After the first year of harvest, you make the observation that the plants growing in full shade almost always seem shorter than those in the full sun. You think that you have a reasonable explanation (hypothesis) for this observation.

Based on the information above, you create the following hypothesis to explain the differences in height you noticed in your tomatoes:

Hypothesis: The height that my tomato plants reach is positively correlated to the amount of sunlight they are exposed to (e.g. the more sun the plant gets the taller it will be).

This hypothesis is testable and falsifiable. So, the next summer you decide to test your hypothesis.

This hypothesis also allows you to make a prediction. In this case you might predict that IF you were to shade a set of tomatoes in the sunny part of the yard, THEN those plants would be shorter than their full-sun neighbors.

You design an experiment to test your hypothesis by buying the same variety of tomato that you planted the previous year and plant your whole yard again. This year, however, you decide to do two different things:

(a) You create a shade structure that you place over a small subset of plants in the sunny part of your yard.

(b) You build a contraption with mirrors that redirects some sunlight onto a small subset of plants that are in the shady part of the yard.

Question 1: We used a shortcut above. Can you create statements for both the null and alternative hypothesis? Work with your classmates to do this.

Question 2: Why do you create a shade structure? What is this testing? Based on your hypothesis what do you predict will happen to the plants under the shade structure?

Question 3: Why do you create the mirror contraption? Why do you potentially need this contraption if you already have the shade structure?

New Data: At the end of the summer you measure the height of your tomato plants and you find once again that the plants in the sunny part of the yard are indeed taller than those in the shady part of the yard. However, you notice that there is no difference in height between the plants under your shade structure and those right next to the structure in full sun. In addition, you notice that the plants in the shady part of the yard are all about the same height, including those that had extra light shined on them via your mirror contraption.

Question 4: What does this experiment lead you to conclude? What would you try to do next?

Question 5: Imagine an alternative scenario in which you discovered, as before, that the plants in the sunny part of the yard were all the same height (even those under your shade structure) but that the plants in the shady part of the yard that got “extra” light from your mirror contraption grew taller than their immediate neighbors. What would this say about your alternate hypothesis? Null hypothesis? What would you do next?

Question 6: What assumptions are you making about the ability to make measurements in this experiment? What influence might these assumptions have on your interpretation of the results?

In this class you will occasionally be asked to create a hypotheses, to interpret data, and to design experiments with proper controls. All of these skills take practice to master, we can start to practice them in BIS2A. Again, while we don’t expect you to be masters after reading this text, we will assume that you have read this text during the first week and that the associated concepts are not completely new to you. You can always return to this text as a resource to refresh yourself.

Disclaimer

While the preceding treatment of the experimental method is very basic - you will undoubtedly add numerous layer of sophistication to these basic ideas as you continue in your studies – it should serve as a sufficient introduction to the topic for BIS2A. The most important point to remember from this section is that the knowledge represented in this course, while sometimes inadvertently represented as irrefutable fact, is really just the most current hypothesis about how certain things happen in biology that has yet to be falsified via experiment.