Print
Category: Design of Experiments (DOE) - Case Studies

Introduction

This article will demonstrate how QLazDOE software can manage simple comparative studies with matched pairs of experimental material, which is called paired t-test (one-sample t-test).

Example used for this demonstration is from the book "Design and Analysis of Experiments" by Douglas C. Montgomery, which you can purchase from: https://www.wiley.com/en-hr/Design+and+Analysis+of+Experiments,+10th+Edition-p-9781119492443

Problem Statement

There is a hardness testing machine that presses a rod with a pointed tip into a metal specimen with a known force. By measuring the depth of the depression caused by the tip, the hardness of the specimen is determined. It is suspected that one tip produces different mean hardness readings than the other. Our task is to determine whether two tips produce same results or not.

We will use 10 material specimens, dividing each specimen into two parts, then randomly assigning one tip to one-half of each specimen and the other tip to the remaining half. The order in which the tips are tested for a particular specimen would be randomly selected.

Strictly speaking, we should consider 10 material specimens as 10 treatments, but for the sake of easier definition of the problem in QLazDOE software, we will consider 10 material specimens as 10 replicates and will imagine that there is only one level for each tip. Although scientifically not correct, this will be a useful trick for easier design of experiment.

Therefore, for the purpose of designing experiment, we will refer to the two tips as two factors, each having only 1 level. Since there are 10 measurements for each tip/specimen combination, we will consider this experiment to be replicated, with 10 replicates.

Finally, we will compare the means using one-sided, paired t-test (a single-sample t-test), in order to examine whether two tips can be considered equal or different in respect to producing results for material hardness.

Definition of Factors and Levels

First we need to insert and define new DOE project, entering project name, description and problem statement.

Now we need to provide definition of factors and levels for this project. We will treat this problem as full-factorial experiment with two factors (two tips) having 1 level. 

Click on the button  or corresponding menu item "Create new Factors and Levels Definition File", for creating and opening new factors & levels definition file.

 

 This action will create and open file for definition of factors and their levels, where we can enter and save our definition of two factors with one level. We can enter any value for level value, let's enter zero (0).

 

We can also click button  or corresponding menu item "Determine Number of Factors and Levels", in order to get information about number of factors and levels into the project record.

  

Design of Experiments

First we need to create new solution for the current project.

There will be 10 measurements for each tip/material specimen combination, therefore we will define full factorial replicated experiment with 10 replicates.

Once we defined our solution and parameters for DOE generator, we can trigger generation of designed experiment. You can generate new experiment by clicking button  or choosing corresponding action "Create DOE in one step" from the menu.

This action will create both DOE matrix file and corresponding DOE results file, ready for entering experimental results.

 

Remember that we used trick with 10 replicates to actually represent 10 material specimens, for easier design of experiments. Let us now correct this and rename column "replicate_index" into "Specimen".

Experimental Results

Now we can enter experimental results. We will replace zeros in columns "Tip 1" and "Tip 2" with actual  response variable results (hardness measurements). Don't forget to save the spreadsheet!

Analysis of Experimental Results

 For analysis of experimental results we will use integrated LazStats statistical software to perform one-sided, one-sample, correlated (paired) t-test.

FIrst we need to open results in the LazStats software, by clicking button  or by choosing corresponding action "Analyze DOE Results in LazStats" from the menu.

This action will open LazStats with loaded results. Now we need to find t-test function in the LazStats menu. Go to Analyses/Comparisons/t-tests.

"Comparison of Two Sample Means" form opens.

Now, we have to choose options as follows:

- Data Entry By: "Values in the data grid file"

- Test Assumptions: "Correlated Scores"

- Test probability: "One-tailed"

For the "First Variable" value, select "Tip 1" response variable. For the "Second Variable" value, select "Tip 2" response variable.

Click the "Compute" button to get results from LazStats.

LazStats returned following computation for the t-test:

COMPARISON OF TWO MEANS

Variable Mean Variance Std.Dev. S.E.Mean N
Tip 1 4.80 5.73 2.39 0.76 10
Tip 2 4.90 4.99 2.23 0.71 10
Assuming dependent samples, t = -0.264 with probability = 0.7976 and 9 degrees of freedom
Correlation between Tip 1 and Tip 2 = 0.868
Difference = -0.10 and Standard Error of difference = 0.38
Confidence interval = ( -0.96, 0.76)
t for test of equal variances = 0.397 with probability = 0.7017

So, we got t-test statistics value t = -0.264 with probability = 0.7976 and 9 degrees of freedom.

Now we need to compare this value with the tabular critical values for t-test, in order to drive our conclusion.

In QLazDOE software go to the "Statistical Tables" tab and choose "T Distribution" sub-tab.

Null hypothesis (H0) is that the two means are equal. Alternative hypothesis is that one mean is larger than other. We will reject null-hypothesis H0 only if one mean is larger than the other. Thus, we specify a one-sided alternative hypothesis.
Our calculated t statistics value is t = -0,264. If we choose alpha = 0.05, then we need to compare calculated numerical value of the test statistic t with the value of t-test statistical table for t0.025 (alpha = 0.05/2=0.025) and 9 degrees of freedom. The tabular value t0.025, 9 = 2.262.
We can see that absolute value of our t-statistics (t = 0,264) is significantly lower than the tabular critical value (tabular value t0.025, 9 = 2.262). Also, high P-value (probability = 0,7976) implies that we cannot reject null hypothesis at any reasonable level of significance.
Therefore, we must conclude that two tips are equal.

Conclusion

Absolute value of our t-statistics (t = 0,264) is significantly lower than the tabular critical value (tabular value t0.025, 9 = 2.262). Also, high P-value (probability = 0,7976) implies that we cannot reject null hypothesis at any reasonable level of significance.
Therefore, we must conclude that the two tips are equal, i.e producing same results for material hardness measurements!

Further Reading

Introduction to QLazDOE software
DOE Case Studies
QLazDOE FREE Application