Planning and Executing Credible Experiments. Robert J. Moffat
(Panel 4.1).
With the help of (and some resistance from) my students, I discovered the problem: I was not communicating well enough. The students and I did not always share the same picture of what should be done next. I decided that this was because I was trying to tell them what to do, whereas I should have been telling them where they were trying to go. I began to address the central issue: how could the student and I come to a meeting of the minds such that they would be able to solve all of their problems without coming to me?
Panel 4.1 There's a Hole in My Bucket
| (American folk song, origin unknown) A dialog between Henry and Liza over a chore with never‐ending quandaries! | |
| Henry | Liza |
| There's a hole in my bucket! Dear Liza, Dear Liza. There's a hole in my bucket, Dear Liza, A hole! | Well, mend it! Dear Henry, Dear Henry, Dear Henry. Well, mend it! Dear Henry, Dear Henry, Mend it! |
| With what shall I mend it? Dear Liza, Dear Liza. With what shall I mend it? Dear Liza, With what? | With a straw! Dear Henry, Dear Henry, Dear Henry. With a straw! Dear Henry, Dear Henry, A straw! |
| But the straw is too long! Dear Liza, Dear Liza. But the straw is too long! Dear Liza, Too long! | Then cut it! Dear Henry, Dear Henry, Dear Henry. Then cut it! Dear Henry, Dear Henry, Cut it! |
| With what shall I cut it? Dear Liza, Dear Liza. With what shall I cut it? Dear Liza, Cut it? | With a knife! Dear Henry, Dear Henry, Dear Henry. With a knife! Dear Henry, Dear Henry, A knife! |
| But the knife is too dull! Dear Liza, Dear Liza. But the knife is too dull! Dear Liza, Too dull! | Then whet it! Dear Henry, Dear Henry, Dear Henry. Then whet it! Dear Henry, Dear Henry, Whet it! (Whet means “to sharpen.”) |
| With what shall I whet it? Dear Liza, Dear Liza. With what shall I whet it? Dear Liza, With what? | With a stone! Dear Henry, Dear Henry, Dear Henry. With a stone! Dear Henry, Dear Henry, A stone! |
| But the stone is too dry! Dear Liza, Dear Liza. But the stone is too dry! Dear Liza, Too dry! | Then wet it! Dear Henry, Dear Henry, Dear Henry. Then wet it! Dear Henry, Dear Henry, Wet it! |
| With what shall I wet it? Dear Liza, Dear Liza. With what shall I wet it? Dear Liza, Wet it? | With water! Dear Henry, Dear Henry, Dear Henry. With water! Dear Henry, Dear Henry, Water! |
| With what shall I fetch it? Dear Liza, Dear Liza. With what shall I fetch it? Dear Liza, With what? | With a bucket! Dear Henry, Dear Henry, Dear Henry. With a bucket! Dear Henry, Dear Henry, A bucket! |
| There's a hole in my bucket! | |
| Virginia Maier, of Pittsburgh, PA, gave me a tape of this song from her collection. | |
Ever been through one of these discussions? I have, many of them – with grad students who didn't have a clear picture of what needed to be done!
This led to the practice of formulating the lab programs in terms of the questions we were trying to answer.
4.2 An Anchor and a Sieve
I view the motivating question as a sort of intellectual nail driven into the wall of the world and to which the student is tied by some sort of intellectual “rope.” Once the motivating question has been really accepted (that is, thoroughly understood and accepted), the student is “tied” to that nail by her/his own rope of intent to finish. When students encounter an obstacle, they can pull themselves past it, always choosing a solution that moves them closer to the objective. This is a far different dynamic than the student trying to respond to my suggestions of what to do next. Once this approach was in place, I no longer felt I was trying to push ropes.1
I strongly recommend figuring out the “motivating question” for each experiment as a way of focusing attention on what must really be done. There are other advantages that accrue from working to answer a question as opposed to working to take data or study something. I hope they will become clear in this chapter.
4.3 Identifying the Motivating Question Clarifies Thinking
How can one identify the motivating question that drives an experiment, and who has the right to make this identification? Not the experimenter! Rather, it is the client who has this right because this is the person who is paying the bill for the experiment!
Identification of the motivating question is rarely easy. The urge to learn something may be strongly felt, but expressing exactly what you want to learn is very difficult. It is bad enough trying to talk about the subject when you have an hour or more to try to make clear your intentions. It is incredibly difficult to write down a concise description in such language that it cannot be misunderstood – and that is what we are trying to do. How often has the exercise of writing exposed our own sloppy thinking? How often have we felt despair at “trying to get it right” in writing? Take courage! The end result, your motivating question, is worth your effort to identify it early!
This chapter addresses how to formulate the motivating question and the advantage of working to a question you are trying to answer. If you experiment without a motivating question, you will take a series of steps – but when will you arrive?
4.3.1 Getting Started
When the urge strikes to run a new experiment, don't fight it. If the urge has struck you, then talk to yourself – takes notes on what you think you should do, what you think the rig might look like, or what you think might happen. Let this process run until you feel you have really expressed yourself. Then stop and ask, “What question am I trying to answer?”
If the urge has struck someone else, and you are that person’s sounding board, listen actively and take notes. Let the person keep talking until he or she runs dry – then the individual knows you have really heard the idea, entirely. Then ask, “If you do this, and you are successful, what question will you have answered?”
I think you must honor the enthusiastic urge by letting it have center stage until it plays itself out. I simply ask that the speaker (myself or my student) get reasonably specific right away. This is not intended in a critical sense but simply to detail the plan. I take notes about the proposed apparatus, the test conditions, and the proposed data, trying to absorb the real intent of the idea. I keep at this until the speaker (myself or the student) runs out of things to say about what to do.
At this point everything that could be said has been said. The pressure is off. We can get to work.
4.3.2 Probe and Focus
Now I raise the following five questions.
If we do this experiment and get all the data we have asked for:
Q1. What question will we have answered?
Q2. Is this question worth the cost of answering it?
Q3. Has it already been answered?
Q4. Is the proposed experiment the best way to