Hjem

About the Authors xxi Preface xxiii Acknowledgments xxvii About the Companion Website xxix 1 Choosing Credibility 1
1.1 The Responsibility of an Experimentalist 2
1.2 Losses of Credibility 2
1.3 Recovering Credibility 3
1.4 Starting with a Sharp Axe 3
1.5 A Systems View of Experimental Work 4
1.6 In Defense of Being a Generalist 5 Panel
1.1 The Bundt Cake Story 6 References 6 Homework 6 2 The Nature of Experimental Work 7
2.1 Tested Guide of Strategy and Tactics 7
2.2 What Can Be Measured and What Cannot? 8
2.2.1 Examples Not Measurable 8
2.2.2 Shapes 9
2.2.3 Measurable by the Human Sensory System 10
2.2.4 Identifying and Selecting Measurable Factors 11
2.2.5 Intrusive Measurements 11
2.3 Beware Measuring Without Understanding: Warnings from History 12
2.4 How Does Experimental Work Differ from Theory and Analysis? 13
2.4.1 Logical Mode 13
2.4.2 Persistence 13
2.4.3 Resolution 13
2.4.4 Dimensionality 15
2.4.5 Similarity and Dimensional Analysis 15
2.4.6 Listening to Our Theoretician Compatriots 16 Panel
2.1 Positive Consequences of the Reproducibility Crisis 17 Panel
2.2 Selected Invitations to Experimental Research, Insights from Theoreticians 18 Panel
2.3 Prepublishing Your Experiment Plan 21
2.4.7 Surveys and Polls 22
2.5 Uncertainty 23
2.6 Uncertainty Analysis 23 References 24 Homework 25 3 An Overview of Experiment Planning 27
3.1 Steps in an Experimental Plan 27
3.2 Iteration and Refinement 28
3.3 Risk Assessment/Risk Abatement 28
3.4 Questions to Guide Planning of an Experiment 29 Homework 30 4 Identifying the Motivating Question 31
4.1 The Prime Need 31 Panel
4.1 There''s a Hole in My Bucket 32
4.2 An Anchor and a Sieve 33
4.3 Identifying the Motivating Question Clarifies Thinking 33
4.3.1 Getting Started 33
4.3.2 Probe and Focus 34
4.4 Three Levels of Questions 35
4.5 Strong Inference 36
4.6 Agree on the Form of an Acceptable Answer 36
4.7 Specify the Allowable Uncertainty 37
4.8 Final Closure 37 Reference 38 Homework 38 5 Choosing the Approach 39
5.1 Laying Groundwork 39
5.2 Experiment Classifications 40
5.2.1 Exploratory 40
5.2.2 Identifying the Important Variables 40
5.2.3 Demonstration of System Performance 41
5.2.4 Testing a Hypothesis 41
5.2.5 Developing Constants for Predetermined Models 41
5.2.6 Custody Transfer and System Performance Certification Tests 42
5.2.7 Quality-Assurance Tests 42
5.2.8 Summary 43
5.3 Real or Simplified Conditions? 43
5.4 Single-Sample or Multiple-Sample? 43 Panel
5.1 A Brief Summary of "Dissertation upon Roast Pig" 44 Panel
5.2 Consider a Spherical Cow 44
5.5 Statistical or Parametric Experiment Design? 45
5.6 Supportive or Refutative? 47
5.7 The Bottom Line 47 References 48 Homework 48 6 Mapping for Safety, Operation, and Results 51
6.1 Construct Multiple Maps to Illustrate and Guide Experiment Plan 51
6.2 Mapping Prior Work and Proposed Work 51
6.3 Mapping the Operable Domain of an Apparatus 53
6.4 Mapping in Operator''s Coordinates 57
6.5 Mapping the Response Surface 59
6.5.1 Options for Organizing a Table 59
6.5.2 Options for Presenting the Response on a Scatter-Plot-Type Graph 61 Homework 64 7 Refreshing Statistics 65
7.1 Reviving Key Terms to Quantify Uncertainty 65
7.1.1 Population 65
7.1.2 Sample 66
7.1.3 Central Value 67
7.1.4 Mean, μ or 67
7.1.5 Residual 67
7.1.6 Variance, Ï?2 or S 2 68
7.1.7 Degrees of Freedom, Df 68
7.1.8 Standard Deviation, Ï? Y or SY 68
7.1.9 Uncertainty of the Mean, δμ 69
7.1.10 ChiSquared, Ï? 2 69
7.1.11 pValue 70
7.1.12 Null Hypothesis 70
7.1.13 Fvalue of Fisher Statistic 71
7.2 The Data Distribution Most Commonly Encountered The Normal Distribution for Samples of Infinite Size 71
7.3 Account for Small Samples: The tDistribution 72
7.4 Construct Simple Models by Computer to Explain the Data 73
7.4.1 Basic Statistical Analysis of Quantitative Data 73
7.4.2 Model Data Containing Categorical and Quantitative Factors 75
7.4.3 Display Data Fit to One Categorical Factor and One Quantitative Factor 76
7.4.4 Quantify How Each Factor Accounts for Variation in the Data 76
7.5 Gain Confidence and Skill at Statistical Modeling Via the R Language 77
7.5.1 Model and Plot Results of a Single Variable Using the Example Data diceshoe.csv 77
7.5.2 Evaluate Alternative Models of the Example Data hiloy.csv 78
7.5.2.1 Inspect the Data 78
7.5.3 Grand Mean 78
7.5.4 Model by Groups: GroupWise Mean 78
7.5.5 Model by a Quantitative Factor 78
7.5.6 Model by Multiple Quantitative Factors 78
7.5.7 Allow Factors to Interact (So Each Group Gets Its Own Slope) 79
7.5.8 Include Polynomial Factors (a Statistical Linear Model Can Be Curved) 80
7.6 Report Uncertainty 80
7.7 Decrease Uncertainty (Improve Credibility) by Isolating Distinct Groups 81
7.8 Original Data, Summary, and R 82 References 83 Homework 83 8 Exploring Statistical Design of Experiments 87
8.1 Always Seeking Wiser Strategies 87
8.2 Evolving from Novice Experiment Design 87
8.3 TwoLevel and ThreeLevel Factorial Experiment Plans 88
8.4 A ThreeLevel, ThreeFactor Design 89
8.5 The Plackett-Burman 12Run Screening Design 93
8.6 Details About Analysis of Statistically Designed Experiments 95
8.6.1 Model Main Factors to Original Raw Data 95
8.6.2 Model Main Factors to Original Data Around Center of Each Factor 96
8.6.3 Model Including All Interaction Terms 97
8.6.4 Model Including Only Dominant Interaction Terms 97
8.6.5 Model Including Dominant Interaction Term Plus Quadratic Term 98
8.6.6 Model All Factors of Example 2, Centering Each Quantitative Factor 99
8.6.7 Refine Model of Example 2 Including Only Dominant Terms 100
8.7 Retrospect of Statistical Design Examples 101
8.8 Philosophy of Statistical Design 101
8.9 Statistical Design for Conditions That Challenge Factorial Designs 102
8.10 A Highly Recommended Tool for Statistical Design of Experiments 103
8.11 More Tools for Statistical Design of Experiments 103
8.12 Conclusion 103 Further Reading 104 Homework 104 9 Selecting the Data Points 107
9.1 The Three Categories of Data 107
9.1.1 The Output Data 107
9.1.2 Peripheral Data 108
9.1.3 Backup Data 108
9.1.4 Other Data You May Wish to Acquire 108
9.2 Populating the Operating Volume 109
9.2.1 Locating the Data Points Within the Operating Volume 109
9.2.2 Estimating the Topography of the Response Surface 109
9.3 Example from Velocimetry 109
9.3.1 Sharpen Our Approach 110
9.3.2 Lessons Learned from Velocimetry Example 111
9.4 Organize the Data 112
9.4.1 Keep a Laboratory Notebook 112
9.4.2 Plan for Data Security 112
9.4.3 Decide Data Format 112
9.4.4 Overview Data Guidelines 112
9.4.5 Reasoning Through Data Guidelines 113
9.5 Strategies to Select Next Data Points 114
9.5.1 Overview of Option
1: Default Strategy with Intensive Experimenter Involvement 115
9.5.1.1 Choosing the Data Trajectory 115
9.5.1.2 The Default Strategy: Be Bold 115
9.5.1.3 Anticipate, Check, Course Correct 116
9.5.1.4 Other Aspects to Keep in Mind 116
9.5.1.5 Endpoints 117
9.5.2 Reintroducing Gosset 118
9.5.3 Practice Gosset Examples (from Gosset User Manual) 119
9.6 Demonstrate Gosset for Selecting Data 120
9.6.1 Status Quo of Experiment Planning and Execution (Prior to Selecting More Samples) 120
9.6.1.1 Specified Motivating Question 120
9.6.1.2 Identified Pertinent Candidate Factors 121
9.6.1.3 Selected Initial Sample Points Using Plackett-Burman 121
9.6.1.4 Executed the First 12 Runs at the PB Sample Conditions 122
9.6.1.5 Analyzed Results. Identified Dominant First-Order Factors. Estimated First-Order Uncertainties of Factors 123
9.6.1.6 Generated Draft Predictive Equation 124
9.6.2 Use Gosset to Select Additional Data Samples 125
9.6.2.1 Example Gosset Session: User Input to Select Next Points 125
9.6.2.2 Example Gosset Session: How We Chose User Input 126
9.6.2.3 Example Gosset Session: User Input Along with Gosset Output 128
9.6.2.4 Example Gosset Session: Convert the Gosset Design to Operator Values 131
9.6.2.5 Results of Example Gosset Session: Operator Plots of Total Experiment Plan 132
9.6.2.6 Execute Stage Two of the Experiment Plan: User Plus Gosset Sample Points 132
9.7 Use Gosset to Analyze Results 133
9.8 Other Options and Features of Gosset 133
9.9 Using Gosset to Find Local Extrema in a Function of Several Variables 134
9.10 Summary 137 Further Reading 137 Homework 137 10 Analyzing Measurement Uncertainty 143
10.1 Clarifying Uncertainty Analysis 143
10.1.1 Distinguish Error and Uncertainty 144
10.1.1.1 Single-Sample vs. Multiple-Sample 145
10.1.2