Planning and Executing Credible Experiments. Robert J. Moffat
the uncertainty. In such a situation, to produce more data with the same uncertainty as the existing dataset will not advance the art, nor will it satisfy the client.
The allowable uncertainty should be described in a manner consistent with current conventions. For most engineering work, the uncertainty is specified at the 95% confidence level and presumes a symmetric probability distribution, centered around zero error. For example, if the uncertainty is quoted as ±10% at the 95% confidence level, this means that 19 of 20 repeated trials of the same measurement act will yield a value within ±10% of the quoted value. The possible error is, by agreement, equally likely to be positive or negative.
In some cases, the client may wish to specify a “one‐sided” tolerance, for example to demonstrate that a product is better than its competition. A one‐sided test must be selected before the experiment is run. The one‐sided test must be clearly indicated in the report so that the results can be interpreted accurately in terms of the one‐sided error allowance. Otherwise, the experiment will be reckoned as a two‐sided or “zero‐centered” test.
4.8 Final Closure
From my experience, a written memo of understanding should be prepared that clearly states the motivating question, the form of an allowable answer, and the acceptable uncertainty. This should become a part of the experimental record.
It is difficult for some project managers to sign off on such a statement as, “The answer to this question, with no other benefit, will justify the cost of this experiment.” Lurking in the back of most minds are hopes for “spin‐off data,” bits of knowledge they hope to get, or a general feeling that the experiment ought to be “kept loose, so we can steer it as we go.” These urges usually arise out of reluctance to come to grips with the specific problem: what is it that we really must know?
I recommend organizing a “signing party” at which the people with a stake in the issue sign off on the question, the form of its answer, and the uncertainty. Too often, higher management pays little or no attention to these “philosophical issues” until too late. There is something about deliberately signing off that makes people pay attention.
Unless there is explicit agreement, early, on the objective of the experiment, a host of serious issues may ensue:
Disagreements on “what to do next.”
Dissatisfaction with the outcome.
“Second guessing” after the final report.
A general whitewashing to cover up the lack of definite progress.
Too many experiments simply don’t answer any question – they only contribute some facts that may or may not be useful.
Reference
1 Platt, J.R. (1953). Address before the Division of Physical Chemistry. American Chemical Society.
Homework
1 4.1 Propose at least three potential motivating questions that your experiment can answer.
2 4.2 Discuss with your client each of the potential motivating questions. Which one is preferred?
3 4.3 What questions does your client raise in light of the preferred motivating question?
4 4.4 How can this motivating question be refined? Can it be made more concrete?
5 4.5 Can your motivating question be clearly achieved?
6 4.6 Rinse and repeat from Exercise 4.1.
Notes
1 1 Upon my (Moffat) retirement from active teaching, my recent graduates presented me with a memorial plaque bearing a gold‐plated piece of rope and the legend “You Can't Push a Rope.” This saying is one of the first major lessons every engineer learns in the course Statics, when she/he draws forces on an object.
2 2 On an exam in a graduate heat‐transfer course, I once asked: “When can equation 8‐16 be used?” One student responded, “On Thursday!” What could I say!
3 3 In one program, it took nearly a year to get the final form of the question worked out. The rig was nearly ready for shakedown runs before we were content. That question, written on a 3 × 5 card and thumbtacked to the wall in the office, guided our research through three PhD theses.
Конец ознакомительного фрагмента.
Текст предоставлен ООО «ЛитРес».
Прочитайте эту книгу целиком, купив полную легальную версию на ЛитРес.
Безопасно оплатить книгу можно банковской картой Visa, MasterCard, Maestro, со счета мобильного телефона, с платежного терминала, в салоне МТС или Связной, через PayPal, WebMoney, Яндекс.Деньги, QIWI Кошелек, бонусными картами или другим удобным Вам способом.