Example - Quality check on two laboratories

We wish to do a quality control check on two laboratories.

Six samples of water are selected, and divided into two parts. One half is randomly chosen and sent to Lab 1; the other half is sent to Lab 2. Each lab is supposed to measure the impurities in the water.

Here is the raw data:

     Sample Lab 1 Lab 2
     ------------------
       1    31.4   28.1
       2    37.0   37.1
       3    44.0   40.6
       4    28.8   27.3
       5    59.9   58.4
       6    37.6   38.9

Notice that the difference between the laboratories is small relative to variation among the samples. You are really interested in examining if the differences among the laboratories is zero.

Does this satisfy the conditions laid out earlier for a paired design? Yes, the measurements from each laboratory are related by the sample. There are two readings from each sample - one from each laboratory.

It would be quite inappropriate to analyze this design using the `two-independent samples t-test'as the readings for lab 1 are not independent of the readings for lab 2.

Let:

• $\mu _{1}$ and $\mu _{2}$ represent the population mean readings from lab 1 and lab 2 respectively..

• n represent the number of pairs. Here n=6. Note it is not correct to say that our sample size is 12 - there were only 6 samples analyzed.

• $\overline{X}_{1}$ and $\overline{X}_{2}$ represent the sample mean readings of lab 1 and lab 2, respectively.

The hypothesis testing proceeds in much the same fashion as before:

1. Specify the hypotheses:

   We are not really interested in the actual impurity readings for the various samples, but rather only interested in the difference. This can be specified as

   H: $\mu _{1} = \mu _{2}$ or $\mu _{1} - \mu _{2}= 0$ or $\mu _{diff} = 0$ A: $\mu _{1} \ne \mu _{2}$ or $\mu _{1} - \mu _{2} \ne 0$ or $\mu _{diff} \ne 0$ .

   where $\mu _{diff}$ is the mean difference in the readings. Note again that the hypotheses are in terms of population parameters and we are interested in testing if the difference is 0. [A difference of 0 would imply no difference in the readings between laboratories, on average]

2. Collect data:

   The data is entered into JMP in the usual fashion. Because the data is paired, the data should be entered as two separate columns, one for each laboratory. Also, another column should be computed to compute the difference between the two labs (use a formula).
   

   

   Summary statistics can be analyzed in two different, but equivalent ways.

   1. Use the two original variables.

      Using the Analyze->Fit Y by X platform, use Lab 2 as the Y variable, Lab 1 as the X variable, and then use the `Paired t-test' from the Analyze pop-up menu. The following plot occurs:
      

      

      If both labs are measuring the impurities identically, all the points should fall along the line Y=X (the dark, thick line). But there is variation in the readings, due to technical problems, etc. and so the points will not fall exactly on the line. If there is no systematic bias, the points should be randomly scattered above and below this thick dark line. However, if there is a systematic `bias' in one lab, the points will consistently fall below the line or above the line. The line which shows the estimated systematic bias is the second line plotted (in color). It appears that lab 1 is showing a systematic positive bias relative to lab 2. However, at this point, we don't know if there is sufficient evidence to conclude that a systematic bias exists in the population value.

   2. Use the derived variable (the difference column)

      Use the Analyze->Distribution of Y to obtain summary statistics on the difference:
      

      

      The box plot shows that the mean diff is around 0. The `Moment' report indicates that the mean difference is about 1.383 units, but the standard error is rather large. Indeed, the 95% confidence interval indicates that the mean difference might be zero since 0 is included in the interval.

3. Compute a test-statistics and a p-value.

   There are two different (but equivalent) ways of doing the test. Method 2a automatically generates the following table; in Method 2b you will need to generate a hypothesis test for the population mean difference is zero. Here are the two reports:
   

   

   Both report give the same test statistic:

   T = (estimate of diff - hypothesized diff)/ estimated se of diff = (1.38 - 0)/.756 = 1.83

   Both are compared to a t-distribution with df=number of pairs -1 = 6-1 =5.

   Once again, you must decide between the two sided or one-sided p-values. We were doing a two sided test, so the two-sided p-value of .1270 is used.

4. Make a decision.

   Because the p-value is large, we fail to reject the null hypothesis and conclude that there is insufficient evidence against the hypothesis that both laboratories give the same mean readings on the samples.

   As before, we haven't proven that both labs give the same reading. We only failed to detect a difference in the mean readings.

Notes:

• An approximate p-value could be found by again using the t-table with the appropriate df:

• The formal name for this test is a `Paired t-test'. It is important to recognize if, in fact, pairing has taken place.

• The `Paired t-test' is a special case of a `Randomized complete block ANOVA' with 2 treatments. Here the blocks are the sample (which isn't formally used by JMP at this point).

• It doesn't matter which method you use - both will give the same final results. The first method also gives a graphical picture of the relationship between the two readings, while the second gives a graphical picture of the difference in the two readings. Both are relatively easy to do, so why not do both?

• There is a subtle problem if the order of the treatments cannot be randomized, e.g. suppose that the two measurements are before and after a treatment. If there are only two treatments, the subtle problem, fortuitously, does not manifest itself. If there are more than two treatments then you can run into problems. (Refer to notes on randomized complete block analyses).