Planning and Executing Credible Experiments. Robert J. Moffat
engineers in Japan focused on the cooking profile of rice (not the shape of the device). By experiment, they found the shape of the temperature versus time curve, T = f (t), as shown in Figure 2.1, which optimizes rice taste when the ingredient is washed white, short‐grain rice. They also found a different shape that is best for unwashed rice and yet another shape for brown rice. Although aspects of the shape can be measured, such as the time at each temperature, it is the shape of the cooking profile that enhances the taste of the rice. By offering more delicious rice, manufacturers gained market share over older cookers that merely boiled rice.3
Another example comes from the physics of fluid flows. In boundary layer studies, the shape of the velocity profile is of considerable interest and often needs to be recorded. One way to deal with this is to present u = f (y), a set of ordered pairs (u, y). This allows the viewer to draw the shape and look at it. Another way, conveying less information but sometimes enough, is to present the shape factor: the ratio of two integral measures of the boundary layer thickness, the displacement thickness divided by the momentum thickness. Turbulent boundary layers have larger values of the shape factor than laminar boundary layers. Researchers who know approximately what the velocity profile looks like (i.e. what family of shapes to which it belongs) can communicate quite a bit of information to one another by quoting shape factor values. Yet the value of the shape factor itself is not a measure of shape. Only if the boundary layer is known to be laminar or turbulent or somewhere in between does the shape factor convey information. If the family of possible shapes is not specified either explicitly or implicitly, then it takes a very large number of scalar pairs to describe shape – enough data points to plot the shape so it can be looked at. Presenting u(y) throughout the boundary layer allows the viewer to see the shape, but that does not constitute a measurement of shape any more than a photograph of a face is a measurement of the shape of that face. The derived scalar “measures” of shape, such as displacement thickness, momentum thickness, and shape factor, can convey significantly wrong impressions of the shape of a boundary layer velocity distribution when they are reported for a “pathological boundary layer,” i.e. one whose velocity distribution is significantly different than usual. They convey the right information only when they are applied to boundary layers with generally typical velocity distributions.
Figure 2.1 Rice cooker design trajectory.
2.2.3 Measurable by the Human Sensory System
How amazing is the human sensory system and what it enables us to observe! Trying to extract as much information using scalar measuring instruments is quite a challenge. The human sensory system is very complex, and its receptors are very well tuned to our environment. If our eyes were just a few decibels more sensitive, we could see single photons; if our ears were just a few decibels more sensitive, we could hear the Brownian motion of individual air molecules as they bounced off our eardrums.
Consider our sense of touch. Our machinists claim to easily detect surface roughness of 50 mils (1 μm) with work‐calloused hands. Can our body measure force? Can it measure temperature? We sense not temperature directly but heat‐transfer rate; if you've ever dipped a cold toe into a warm bath, did the water feel boiling hot even if was just warm? (As in an Onsen hot spring in Japan.) We don’t feel force directly but pressure and shear force. In contrast, in the lab we have simple instruments for measuring force and temperature but need sophisticated techniques to measure heat‐transfer rate. Even touch is a marvel. Prepublication we learned: in the journal Nature, it has been “experimentally established that humans have the capacity to perceive single photons of light” (Tinsley et al. 2016). Furthermore “human tactile discrimination extends to the nanoscale … within billionths of a meter” (Skedung et al. 2013). In part this explains how polished steel tactilely differs from smooth rubber.
2.2.4 Identifying and Selecting Measurable Factors
One of the first problems to be faced in exploratory research, and in development work, is identifying which scalars are significant to the issue at hand. Sometimes a single scalar is sufficient, such as a temperature, pressure, or velocity. Sometimes a compound scalar measure can be put together from a set of simple scalars. One example would be the Reynolds number, used for characterizing the state of the flow in a channel. Sometimes two or more scalars can be combined into one measure to reflect value judgments or “trade‐offs” in desirability. For example, consider the problem of selecting the optimum heat exchanger for an engine application. In even the simplest situation, ignoring such considerations as cost, size, and durability, a heat exchanger for engine service has at least two important scalar descriptors: its effectiveness and its pressure drop. Typically, high effectiveness is “good,” while high‐pressure drop is “bad.” Also typically, pressure drop goes up when effectiveness goes up. Neither by itself is a measure of goodness for the heat exchanger. A weighted sum of the two can be used as a “composite scalar” by finding two weight functions, one for effectiveness and one for pressure drop, which accounts for the effects of both on some single, important parameter – brake‐specific fuel consumption is another example. Such “goodness factors” can be used to account for many factors at a time and to convert a nonmeasurable situation to a measurable one.
The choice of what to measure sets the course of the entire experiment, and that choice should be made with considerable care.
2.2.5 Intrusive Measurements
Always remember as you plan: making a measurement intrudes on the system measured. Ask: How much does the measurement intrusion alter the system? How does measurement affect the accuracy and precision of its results? Furthermore, the intrusion affects the uncertainty of the measurement. A prime motivator for inventing new probes and experimental techniques is the aim to intrude less when taking a measurement. Classic measurement is the focus of this text.
In certain quantum physics experiments, however, mere observation alters the system. In these experiments, if an observation is made the system has one result; if no observation is made the system shows contrary results. These tests have been reproduced worldwide. If you are interested, we recommend Richard Muller's book (Muller 2016). Or search “theory of measurement in physics.”
2.3 Beware Measuring Without Understanding: Warnings from History
There is an unfortunate tendency, among engineers particularly, simply to measure everything which can be measured, report the results, and hope that someone, someday, will find the results useful. This is a deplorable state of affairs, even though it has a long and honored past. In many respects, it follows the scientific tradition which emerged from Europe in the nineteenth century, when the art of measurement expanded so rapidly in Western civilization.
William Thomson, Lord Kelvin, famously proclaimed: “When you can measure what you are talking about and express it in numbers then you have the beginning of knowledge.” That is still true today but with some limitations (Thomson 1883).
Thomson's enthusiasm for measurements should be interpreted in terms of the times in which he lived. The European scientists of that period were infatuated with measurement. Every new measurement technique developed was applied to every situation for which it seemed to fit. There was no storehouse of knowledge about the physical world. Each new series of measurements revealed order in another part of the physical world, and it appeared that every measurement answered some question, and every question could be answered if only enough measurements were made. That was true partly because there were so many unanswered questions and partly because most of the questions which were being asked at that time could be answered by scalars.
Since that era, experimental