F2023_Bis2A_Singer_Scientific_Method

The scientific method overview

An example of oversimplification that confounds many students of biology (particularly early in their studies) is the use of language that hides the experimental process used to build knowledge. For the sake of expediency, we often tell stories about biological systems as if we are presenting unquestionable facts. However, while we often write and speak about topics in biology with a conviction that gives the appearance of "factual" knowledge, reality is often more nuanced and filled with significant uncertainties. The "factual" presentation of material (usually lacking discussion of evidence or confidence in the evidence) plays to our natural tendency to feel good about "knowing" things, but it tends to create a false sense of security in the state of knowledge and does little to encourage the use of imagination or the development of critical thinking.

A better way to describe our knowledge about the natural world would be to explicitly qualify that what we know to be "true" in science represents only our current best understanding of a topic; an understanding that has not yet been refuted by experiment. Unfortunately, repeated qualification becomes rather cumbersome. The important thing to remember is that while we may not say so explicitly, all of the knowledge we discuss in class represents only the best of our current understanding. Some ideas have withstood repeated and varied experimentation while other topics have yet to be tested as thoroughly. So if we're not as certain about things as we'd like to believe sometimes, how do we know what to put confidence in and what to be skeptical of? The complete answer is non-trivial but it begins with developing an understanding of the process we use in science to build new knowledge. The scientific method is the process by which new knowledge is developed. While the process can be described with long lists of "steps" (often seen in textbooks), its core elements can be described more succinctly.

Succinct description of scientific method (adapted from Feynman)

1. Make an observation about the world.
2. Propose a possible explanation for the observation.
3. Test the explanation by experiment.
4. If the explanation disagrees with experiment, the explanation is wrong.

At its core, that's it! In science there may be multiple, simultaneously proposed explanations or ideas that are tested by experiment. The ideas that fail experimentation are left behind. The ideas that survive experimentation move forward and are often retested by alternative experiments until they too either fail or continue to be retained.

Making an observation and asking a question

The ability to make useful observations and/or ask meaningful questions requires curiosity, creativity, and imagination—this cannot be overstated. Indeed, historically, it is first and foremost the application of these skills, perhaps more than technical ability, which has led to big advances in science. Many people think that making meaningful observations and asking useful questions is the easiest part of the scientific method. This is not always the case. Why? Seeing what others have not yet asked and creativity both take work and thoughtful reflection! In addition, our senses of observation are often biased by life experience, prior knowledge, or even our own biology. These underlying biases influence how we see the world, how we interpret what we see, and what we are ultimately curious about. This means that when we look at the world, we can miss a lot of things that are actually right under our noses. Douglas Adams, who is best known for his book entitled The Hitchhiker’s Guide to the Galaxy, once expanded on this point by writing:

“The most misleading assumptions are the ones you don't even know you're making.”

Scientists, therefore, need to be aware of any underlying biases and any assumptions that may influence how they internalize and interpret observations. This includes approaching our bias that the variety of places we get our knowledge (i.e., textbooks, instructors, the Internet) are representing the absolute truth with a healthy dose of skepticism. We need to learn to examine the evidence underling the “facts” we supposedly know and make critical judgments about how much we trust that knowledge. More generally, taking the time to make careful observations and to uncover any assumptions and biases that could influence how they are interpreted is, therefore, time well spent. This skill, like all others, needs to be developed and takes practice and we’ll try to start you on this in BIS2A.

For fun, and to test your observation skills, Google “observation tests”. Many of the search results will take you to interesting psychological tests and/or videos that illustrate how difficult accurate observation can be.

Generating a testable hypothesis

The "possible explanation" referred to in step three above has a formal name; it is called a hypothesis. A hypothesis is not a random guess. A hypothesis is an educated (based on prior knowledge or a new viewpoint) explanation for an event or observation. It is typically most useful if a scientific hypothesis can be tested. This requires that the tools to make informative measurements on the system exist and that the experimenter has sufficient control over the system in question to make the necessary observations.

Most of the time, behaviors of the system that the experimenter wants to test can be influenced by many factors. We call the behaviors and factors dependent and independent variables, respectively. The dependent variable is the behavior that needs explaining while the independent variables are all of the other things that can change and influence the behavior of the dependent variable. For example, an experimenter that has developed a new drug to control blood pressure may want to test whether her new drug actually influences blood pressure. In this example, the system is the human body, the dependent variable might be blood pressure, and the independent variables might be other factors that change and influence blood pressure like age, sex, and levels of various soluble factors in the blood stream.

Note: for more on dependent and independent variables

on Wikipedia
on Khan

Note

In BIS2A, and beyond, we prefer to avoid using language like “the experiment proved her hypothesis” when referring to a case like the blood pressure example above. Rather we would say, “the experiment is consistent with her hypothesis.” Note that for convenience, we referred to the alternative hypothesis simply as “her hypothesis”! It would be more correct to state, “the experiment falsified her null hypothesis and is consistent with her alternative hypothesis.” Why take this shortcut since doing so adds confusion when a student is trying to learn? In this case, it was done to illustrate the point above about language shortcuts and hence the lengthy explanation. However, be aware of this commonly used shortcut and learn to make sure you can read in the correct meaning yourself.

 

---

Possible NB Discussion Point

What does the statement about falsifying hypotheses mean in your own words? Why is falsification critical to the scientific method?

---

 

Controls

In an ideal case, an experiment will include control groups. Control groups are experimental conditions in which the values of the independent variables (there may be more than one) are maintained as close to those in the experimental group with the exception of the independent variable being tested. In the blood pressure example, an ideal scenario would be to have one identical group of people taking the drug and another group of people identical to those in the experimental group taking a pill containing something known to not influence blood pressure. In this oversimplified example, all independent variables are identical in the control and experimental groups with the exception of the presence or absence of the new drug. Under these circumstances, if the value of the dependent variable (blood pressure) of the experimental group differs from that of the control group, one can reasonably conclude that the difference must be due to the difference in independent variable (the presence/absence of the drug). This is, of course, the ideal. In real life it is impossible to conduct the proposed drug dosage experiment; the sheer number of possible independent variables in a group of potential patients would be high. Fortunately, while statisticians have come to the rescue in real life, you won’t need to understand the nuances of these statistical issues in BIS2A.

Accuracy in measurement, uncertainty, and replication

Finally, we mention the intuitive notion that the tools used to make the measurements in an experiment must be reasonably accurate. How accurate? They must be accurate enough to make measurements with sufficient certainty to draw conclusions about whether changes in independent variables actually influence the value of a dependent variable. We take, yet again, the blood pressure example above. In that experiment, we made the important assumption that the experimenter had tools that allowed her to make accurate measurements of the changes in blood pressure associated with the effects of the drug. For instance, if the changes associated with the drug ranged between 0 and 3 mmHg and her meter capably measured changes in blood pressure with a certainty of +/- 5 mmHg, she could not have made the necessary measurements to test her hypothesis or would have missed seeing the effect of the drug. For the sake of the example, we assume that she had a better instrument and that she could be confident that any changes she measured were indeed differences due to the drug treatment and that they were not due to measurement error, sample-to-sample variability, or other sources of variation that lower the confidence of the conclusions that are drawn from the experiment.

The topic of measurement error leads us to mention that there are numerous other possible sources of uncertainty in experimental data that you as students will ultimately need to learn about. These sources of error have a lot to do with determining how certain we are that experiments have disproven a hypothesis, how much we should trust the interpretation of the experimental results and, by extension, our current state of knowledge. Even at this stage, you will recognize some experimental strategies used to deal with these sources of uncertainty (i.e., making measurements on multiple samples, creating replicate experiments). You will learn more about this in your statistics courses later on.

For now, you should, however, be aware that experiments carry a certain degree of confidence in the results and that the degree of confidence in the results can be influenced by many factors. Developing healthy skepticism involves, among other things, learning to assess the quality of an experiment and the interpretation of the findings and learning to ask questions about things like this.

 

---

Possible NB Discussion Point

After moving to California to attend UC Davis, you have fallen in love with fresh tomatoes. You decide that the tomatoes in the stores just don’t taste right and resolve to grow your own.

You plant tomato plants all over your back yard; every free space now has a freshly planted tomato seedling of the same variety. You have planted tomatoes in the ground in full sunlight and next to your house in full shade.

Observation: After the first year of harvest, you make the observation that the plants growing in full shade almost always seem shorter than those in the full sun. You think that you have a reasonable explanation (hypothesis) for this observation.

Based on the information above, you create the following hypothesis to explain the differences in height you noticed in your tomatoes:

Hypothesis: The height that my tomato plants reach is positively correlated to the amount of sunlight they are exposed to (e.g., the more sun the plant gets, the taller it will be).

This hypothesis is testable and falsifiable. So, the next summer you decide to test your hypothesis.

This hypothesis also allows you to make a prediction. In this case you might predict that IF you were to shade a set of tomatoes in the sunny part of the yard, THEN those plants would be shorter than their full-sun neighbors.

You design an experiment to test your hypothesis by buying the same variety of tomato that you planted the previous year and plant your whole yard again. This year, however, you decide to do two different things:

1. You create a shade structure that you place over a small subset of plants in the sunny part of your yard.
2. You build a contraption with mirrors that redirects some sunlight onto a small subset of plants that are in the shady part of the yard.

Question 1: We used a shortcut above. Can you create statements for both the null and alternative hypothesis? Work with your classmates to do this.

Question 2: Why do you create a shade structure? What is this testing? Based on your hypothesis what do you predict will happen to the plants under the shade structure?

Question 3: Why do you create the mirror contraption? Why do you potentially need this contraption if you already have the shade structure?

New data: At the end of the summer you measure the height of your tomato plants and you find, once again, that the plants in the sunny part of the yard are indeed taller than those in the shady part of the yard. However, you notice that there is no difference in height between the plants under your shade structure and those right next to the structure in full sun. In addition, you notice that the plants in the shady part of the yard are all about the same height, including those that had extra light shined on them via your mirror contraption.

Question 4: What does this experiment lead you to conclude? What would you try to do next?

Question 5: Imagine an alternative scenario in which you discovered, as before, that the plants in the sunny part of the yard were all the same height (even those under your shade structure) but that the plants in the shady part of the yard that got “extra” light from your mirror contraption grew taller than their immediate neighbors. What would this say about your alternate hypothesis? Null hypothesis? What would you do next?

Question 6: What assumptions are you making about the ability to make measurements in this experiment? What influence might these assumptions have on your interpretation of the results?

In this class, you will occasionally be asked to create a hypotheses, to interpret data, and to design experiments with proper controls. All of these skills take practice to master—we can start to practice them in BIS2A. Again, while we don’t expect you to be masters after reading this text, we will assume that you have read this text during the first week and that the associated concepts are not completely new to you. You can always return to this text as a resource to refresh yourself.

---

 

Disclaimer

While the preceding treatment of the experimental method is very basic—you will undoubtedly add numerous layer of sophistication to these basic ideas as you continue in your studies—it should serve as a sufficient introduction to the topic for BIS2A. The most important point to remember from this section is that the knowledge represented in this course, while sometimes inadvertently represented as irrefutable fact, is really just the most current hypothesis about how certain things happen in biology that has yet to be falsified via experiment.

 

The Design Challenge

Your BIS2A instructors have devised something that we call “The Design Challenge” to help us approach the topics we cover in the course from a problem solving and/or design perspective. This teaching tool helps us:

• develop a frame of mind or way of approaching the material and
• design a set of sequential steps that help structure thinking about course topics in a problem-solving context.

How is it intended to work? When we encounter a topic in class, “The Design Challenge” encourages us to think about it in the following problem-solving centric way:

1. Identify the problem(s) - this may include identifying "big" problems and also decomposing them into "smaller" nested sub-problems
2. Determine criteria for successful solutions
3. Identify and/or imagine possible solutions
4. Evaluate the proposed solutions against the criteria for success
5. Choose a solution

By using the structure of the design challenge, topics that are typically presented as lists of facts and stories are transformed into puzzles or problems that need solving. For instance the discussion about the topic of cell division is motivated by a problem. The problem statement can be: "The cell needs to divide". Some of the criteria for success can include needing to have a near identical copy of DNA in each daughter cell, distributing organelles between the daughter cells so that each remains viable etc. These would be considered sub-problems to the larger “the cell needs to divide” problem. One can then go on to explore what the challenges are and try to use existing knowledge and imagination to propose some solutions for each of those problems. Different solutions can be evaluated and then compared to what Nature seems to have done (at least in the cases that are well studied).

This exercise requires us to use imagination and critical thinking. It also encourages the student and instructor to think critically about why the particular topic is important to study. The Design Challenge approach to teaching biology attempts to make the student and instructor focus on the important core questions that drove the development of the knowledge in the first place! It also encourages students to dream up new ideas and to interact with the material in a manner that is question/problem-centered rather than “fact”-centered. The question/problem-centered approach is different from what most people are used to, but it is ultimately more useful for developing skills, mental frameworks and knowledge that will transfer to other problems that they will encounter during their studies and beyond.

Example

For example, the guiding problem in BIS2A is to understand “How to Build a Cell”. This rather complex problem will be broken down into several smaller sub-problems that include:

• acquiring the building blocks to construct cellular parts from the environment
• acquiring the energy to build cellular parts from the environment
• transforming the building blocks of the cell between different forms
• transferring energy between different storage forms
• creating a new cell from an old cell
• problems we identify in class

As we explore these sub-problems, we will at times explore some of the different ways in which biology has addressed each issue. As we get into details, let us however make sure to stay focused on and not forget the importance of always staying linked to the questions/problems that motivated us to talk about the specifics in the first plac

Scientific Method vs. The Design Challenge

At this point you might be thinking: "What is the difference between the scientific method and the design challenge rubric and why do I need both?" It's not an uncommon question so let's see if we can clarify this now.

The design challenge and the scientific method are both processes that share similar qualities. The critical distinguishing feature, however, is the purpose behind each of the processes. The scientific method is a process used for eliminating possible answers to questions. A typical scenario where one might use the scientific method would involve someone making an observation, proposing multiple explanations, designing an experiment that might help eliminate one or more of the explanations, and reflecting on the result. By contrast the design process is used for creating solutions to problems. A typical scenario for the design challenge would start with a problem that needs solving, defining criteria for a successful resolution, devising multiple possible solutions that would meet the success criteria and either selecting a solution or reflecting on changes that might be made to the designs to meet success criteria. A key operational difference is that the design challenge requires that criteria for success be defined while the scientific method does not.

While both are similar, the differences are still real and we need to practice both processes. We'll assert that we use both processes in "real life" all of the time. A physician, for instance, will use both processes interactively as she forms hypotheses that try to determine what might be causing her patient's ailments. She will turn around and use the design process to build a course of treatment that meets certain success criteria. A scientist may be deep into hypothesis generation but he will eventually need to use a design process for building an experiment that will, within certain definable success criteria, help him answer a question.

Both processes, while similar, are important to use in different situations and we want to begin getting better at both.

An important note:

Your instructors will propose some functional hypotheses for you to consider that address these broader points. Our hypotheses may sometimes come in the form of statements like, "Thing A exists because of rationale B." To be completely honest, however, in many cases, we don't actually know all of the selective pressures that led to the creation or maintenance of certain cellular structures, and the likelihood that one explanation will fit all cases is slim in biology. The causal linkage/relationship implied by the use of terms like "because" should be treated as good hypotheses rather than objective, concrete, undisputed, factual knowledge. We want you to understand these hypotheses and to be able to discuss the ideas presented in class, but we also want you to indulge your own curiosity and to begin thinking critically about these ideas yourself. Try using the Design Challenge rubric to explore some of your ideas. In the following, we will try to seed questions to encourage this activity.