Study 1: Comparing Popular Diets

 

Dansinger, Griffith, Gleason et al. (2005) report on a randomized, comparative experiment in which 160 subjects were randomly assigned to one of four popular diet plans: Atkins, Ornish, Weight Watchers, and Zone (40 subjects per diet).  These subjects were recruited through newspaper and television advertisements in the greater Boston area; all were overweight or obese with body mass index values between 27 and 42. Among the variables measured were:

  • which diet the subject was assigned to
  • whether or not the subject completed the twelve-month study
  • the subject’s weight loss after two months, six months, and twelve months (in kilograms, with a negative value indicating weight gain)
  • the degree to which the subject adhered to the assigned diet, taken as the average of 12 monthly ratings, each on a 1-10 scale (with 1 indicating complete non-adherence and 10 indicating full adherence)

Data for the 93 subjects who completed the 12-month study are in the Minitab worksheet ComparingDiets.mtw. 

 

Some of the questions that the researchers studied are:

(a) Do the average weight losses after 12 months differ significantly across the four diet plans?

(b) Is there a significant difference in the completion/dropout rates across the four diet plans?

(c) Is there a significant positive association between a subject’s adherence level and his/her amount of weight loss?

(d) Is there strong evidence that dieters actually tend to lose weight on one of these popular diet plans?

 

For each of these research questions, first identify the explanatory variable and the response variable, and classify each as qualitative or quantitative.  Then use graphical and numerical summaries to investigate the question, and summarize your findings.  Next, identify the inference technique that can be used to address the question, and apply that technique.  Be sure to include all aspects of the procedure, including a check of its technical conditions.  Finally, summarize your conclusions for each question. [Hint: To determine the completion rate for each study, count how many of the 93 subjects are in each diet group.]

 

Analysis:

(a) Effect of diet plan on average weight loss: The explanatory variable is diet plan, which is qualitative.  The response variable is amount of weight loss after 12 months, which is quantitative.  Boxplots and numerical summaries follow:

diet         N  Mean  StDev  Median   IQR

Atkins      21  3.92   6.05    3.90  9.15  (all in kilograms)

Ornish      20  6.56   9.29    5.45  6.81  (all in kilograms)

Wgt Watch   26  4.59   5.39    3.60  6.85  (all in kilograms)

Zone        26  4.88   6.92    3.40  8.63  (all in kilograms)

 

These boxplots and statistics seem to indicate that the four diets do not differ substantially with regard to weight loss after twelve months.  The mean and median weight loss are both positive for all four diets, indicating that subjects did tend to lose some weight on these diets, roughly 4-6 kilograms on average.  The boxplots also show substantial overlap between the four distributions.  The means and medians are very similar for three of the diets, with the Ornish diet having a somewhat larger mean and median weight loss (6.56 and 5.45 kilograms, respectively) than the others.  All four distributions of weight loss appear to be fairly symmetric, perhaps a bit skewed to the right.  The variability in weight losses is also similar across all four diet plans, with the Ornish diet having the most variability, largely due to its one small and three large outliers.

 

Because we have a qualitative explanatory variable and a quantitative response variable, we will apply ANOVA to these data.  The technical conditions appear to be met: the subjects were randomly assigned to diet plans, the distributions look fairly normal (see the following dotplots), and the standard deviations are similar enough (ratio of largest to smallest is 9.29/5.39, which is less than 2) for us to assume the population standard deviations are equal.

 

The hypotheses are:

H0: mA = mO = mW = mZ, where mi represents the underlying true treatment mean weight loss after 12 months with diet i.  This hypothesis says that the treatment mean is the same for all four diets. 

Ha: that at least two of the treatment means differ; in other words, that at least one diet does have a different-sized treatment mean than the others

 

Minitab produces the following ANOVA table:

Source  DF      SS    MS     F      P

diet     3    77.6  25.9  0.54  0.659

Error   89  4293.7  48.2

Total   92  4371.3

 

The small F-statistic (F = 0.54) and large p-value (.659) reveals that the experimental data provide essentially no evidence against the null hypothesis.  The p-value reveals that differences among the group means at least as big as those found in this experiment would occur about 66% of the time by randomization alone even if there were no true differences in mean weight loss among the diets.  In other words, the observed treatment means do not differ significantly, and there is no evidence that these four diets truly produce different average amounts of weight loss.

 

(b) Effect of diet plan on completion rate: The explanatory variable is diet plan, which is qualitative.  The response variable is whether the subject completed the study or dropped out, which is also qualitative.

 

The two-way table of completion/dropout status by diet plan, followed by completion proportions and a segmented bar graph:

 

Atkins

Ornish

Weight Watchers

Zone

Completed

21

20

26

26

Dropped out

19

20

14

14

Completion proportion

.525

.500

.650

.650

 

This preliminary analysis appears to reveal that the completion rates are very similar across the four diet plans.  Weight Watchers and Zone tied for the highest completion rate (62.5%), with Ornish having the lowest completion rate (50%), but these do not seem to differ substantially.

 

To test whether these differences in the distributions of the qualitative response variable are statistically significant, we can apply a chi-square test of the hypotheses:

H0: pA = pO = pW = pZ, where pi represents the true underlying completion rate after 12 months with diet i (diet does not have an effect on completion rate)

Ha: at least two of the underlying completion rates differ (there is a difference in underlying completion rates across the 4 diets)

 

Minitab produces the following output:

                         Weight

       Atkins  Ornish  Watchers   Zone  Total

    1      21      20        26     26     93

        23.25   23.25     23.25  23.25

        0.218   0.454     0.325  0.325

 

    2      19      20        14     14     67

        16.75   16.75     16.75  16.75

        0.302   0.631     0.451  0.451

 

Total      40      40        40     40    160

 

Chi-Sq = 3.158, DF = 3, P-Value = 0.368

 

Checking the technical conditions of the chi-square procedure, we note that the subjects were randomly assigned to a diet plan group and that all expected counts in the table are larger than 5 (smallest is 16.75), so we are justified in applying the chi-square test. The p-value of .368 says that if there were no difference in the underlying completion rates (i.e., no treatment effect) of completion among the four diet plans, then it would not be surprising (probability .368) to obtain experimental completion proportions that differ as much as these do.  Because this p-value is not small, we can only conclude that the experimental data do not provide evidence to suggest that the completion proportions differ across these four diet plans.

 

(c) Association between adherence and weight loss: The explanatory variable is adherence level.  This variable is quantitative.  (It could be considered qualitative if it were simply on a 1-10 scale, but it is the average of 12 such values and so should be treated as quantitative.)  The response variable is weight loss, which is quantitative. 

 

A scatterplot of weight loss vs. adherence level follows:

This graph reveals a moderately strong, positive, linear relationship between weight loss and adherence level.  The correlation coefficient can be found to be r=.518.  The scatterplot and correlation coefficient both suggest that there is a positive association between these variables, that subjects with higher adherence levels tend to lose more weight.

 

We can fit a regression line for predicting weight loss from adherence level:

This model indicates that for each additional step on the adherence level scale, the subject is predicted to lose an additional 2.4 kilograms of weight.

 

While there is a moderately strong positive linear relationship between weight loss and adherence level, we cannot draw a casual link between these two variables.  Even though the study was a randomized comparative experiment, the variable imposed by the researchers was the diet plan, not the adherence level.  Therefore, for the purpose of relating adherence level and weight loss, this study is essentially observational.  However, we still might be interested in investigating whether the relationship observed in this sample is strong enough to convince us that it did not arise by chance.  However, these subjects also were not a random sample from a larger population.  (They volunteered for this study in response to advertisements.)  Still, we might cautiously consider them representative of overweight men and women from the Northeast who would consider enrolling on these diets.  With this consideration, we can proceed with a test to determine whether the observed level of association is higher than would be expected by random variation alone.

 

H0: b = 0, where b represents the population slope coefficient. This null hypothesis indicates that there is no linear relationship between adherence level and amount of weight loss.

Ha: b > 0, There is a positive linear relationship in the population.

 

Minitab produces the following output:

Predictor          Coef  SE Coef      T      P

Constant         -8.432    2.306  -3.66  0.000

adherence level  2.3876   0.3971   6.01  0.000

 

So the one-sided p-value is also essentially zero.  Checking the other technical conditions for the regression model, we find the following residual plots:

 

The plot of residuals vs. the explanatory variable does not reveal any serious problems (such as curvature) so we will consider the linearity condition met, although there is a suggestion of increasing variability with larger values.  The histogram of the residuals suggests a bit of a skew to the right but not too much.  This condition is a bit less problematic with the large sample size in this study.  We could consider a transformation to account for the increasing variability, but the increase does not seem substantial enough to warrant a transformation here.  These technical conditions seem to be fairly well met.

 

The test statistic is very large (t=6.01) and the one p-value very small (.000 to three decimal places), and so we can conclude that the experimental data provide extremely strong evidence that there is a positive association between adherence level and weight loss.  The p-value reveals that it would be almost impossible to obtain such large sample correlation and slope coefficients if from random sampling variation alone.  At any reasonable significance level, we conclude that this association is statistically significant.

 

But remember the caveat that we mentioned earlier: the subjects’ adherence levels were observed and not imposed, so we can not draw a cause-and-effect conclusion between adherence level and weight loss.  Furthermore, we must be cautious in stating to what population we are willing to generalize these conclusions.

 

(d) Mean weight loss: To address the question of whether dieters who complete 12 months on one of these popular diet plans actually tend to lose weight, we can begin by combining the weight loss diet across all four diet plans.  This step seems reasonable because of our conclusion in (a) that the there is no evidence of an effect of diet plan on weight loss. While the adherence level does appear to be related to amount of weight lost, the completion rate did not vary significantly across the diets, providing further justification for pooling across the diets.  A histogram of the weight loss amounts (in kilograms) for the 93 subjects who completed the 12-month study follows:

This histogram reveals that the distribution of weight loss amounts is a bit skewed to the right.  The mean weight loss is 4.95 kilograms, with a standard deviation of 6.89 kilograms.  The median is 3.90 kilograms, and most of the subjects had a positive weight loss; in fact, 71 of 93 (76.3%) did.

 

To perform statistical inference with these data, we need to again consider the volunteer nature of the sample and that any randomness here is hypothetical.  We will proceed to conduct tests and make inferences, which will tell us whether the sample results are extreme enough to be unlikely to occur by random variation alone, but we need to keep in mind that the sample may not be representative of any population.

Even though the distribution is a bit skewed, the large sample size (n=93) allows us to perform a t-test of the hypotheses:

H0: m = 0 (the mean weight loss in the population of dieters who could use one of these popular plans is zero)

Ha: m > 0 (the mean weight loss in the population of dieters who could use one of these popular plans is positive)

 

The test statistic turns out to be to t0= , producing a p-value of essentially zero.  This suggests that the sample data provide overwhelming evidence that the population mean weight loss exceeds zero; i.e., that dieters on these plans do tend to lose weight on average.  A 95% confidence interval for m turns out to be (3.53, 6.36), so we can be 95% confident that the population mean weight loss is between 3.53 and 6.36 kilograms.  (Note, this is not the same as predicted the amount of weight an individual would lose on one of these diets.)

 

We can also analyze these data in a slightly different way:

Define p to the proportion of all potential diets who lost weight on one of these diet plans.

H0: p = .5 (half of the population of all potential dieters would lose positive weight on one of these diet plans)

Ha: p > .5 (more than half of the population of all potential dieters would lose positive weight on one of these diet plans)

The data reveal that 71 of 93 subjects had positive weight loss.  A one-sample proportion z-test (valid since we have .5(93)  10) gives:

Test of p = 0.5 vs p > 0.5

Sample   X   N  Sample p    Z-Value  P-Value

1       71  93  0.763441       5.08    0.000

With such a small p-value, this procedure leads to a similar conclusion: overwhelming evidence that more than half of the population would lose positive weight.

 

Because of the volunteer nature of the sample, it is not completely clear to what population we can generalize these results.  Moreover, even though we concluded that the mean weight loss is significantly larger than zero, we cannot attribute the cause to the diet.  Without the use of a comparison group of people who did not participate in a diet plan, we cannot conclude that the diet alone is responsible for the tendency to lose weight.  Perhaps even the power of suggestion from being in the study was a sufficient cause for these individuals to lose weight on average.

 

Summarizing our findings from this study:

  • We do not have convincing evidence to suggest that one of these popular diet plans produces more weight loss on average than another.
  • We do not have convincing evidence to suggest that completion rates differ among the four diet plans.
  • We do have very strong evidence to suggest that dieters on one of these plans will lose weight, between 3.5 and 6.5 kg on average, subject to the caveat that the dieters in this study may not be representative of a larger population of overweight people. 
  • We do have significant evidence that those who adhere to a diet more closely do tend to lose more weight (subject to the same caveat).