Concepts of Statistical Inference: A Randomization-Based Curriculum
	This project was funded by a grant from the National Science Foundation (NSF CCLI-DUE-0633349) and is administered by the Cal Poly Corporation.

Other Primary Personnel

Advisory Group

This project is developing a fundamentally different curriculum for the introductory statistics course that emphasizes the entire process of statistical investigations, from design of data collection through statistical inference, throughout the course. The inference techniques are based on randomness introduced in data collection, specifically randomization and permutation tests, rather than on normal-based probability models. The goal is to lead students to develop a deeper understanding of fundamental concepts of statistical inference and of the process through which statisticians investigate research questions by collecting, analyzing, and drawing conclusions from data.

This project focuses on creating new learning materials and teaching strategies, assessing learning, evaluating innovations, and class-testing curricular materials. As with materials developed by the project team for statistics courses for mathematically intensive majors, these materials consist primarily of learning activities that guide students to discover and explore statistical ideas, but also provide sufficient exposition for a stand-alone text to provide students with reference and reinforcement. Real data from genuine studies motivates all of the activities, which make extensive use of technology.

Assessment and evaluation are especially important aspects of this project. The investigators systematically investigate the effectiveness of this new curriculum in terms of students’ levels of conceptual understanding. The testable hypotheses include that a randomization-based curriculum leads to a deep understanding of p-values, as well as a better understanding of the entire statistical process, than a standard parametric approach. The investigators are also studying two alternative conceptualizations of the concept of confidence to determine whether students develop deeper understandings with one or the other. The investigators are employing a combination of quantitative and qualitative methods in a program of classroom-based research at the institution.

For background on this curricular paradigm, read George Cobb's USCOTS 2005 talk.

• Begin development of curricular materials
• Begin development of assessment items, research plan
• Class-test sample modules at Cal Poly
  
• Begin to collect evaluation data
  

• Continue curriculum development
  
• Complete draft of assessment items
  
• Analyze evaluation data from Spring, inform curriculum development
  

• Expand class-testing at Cal Poly
  
• Collect evaluation data
  
• Continue curriculum development
  

• Analyze evaluation data, inform curriculum development
• Continue curriculum development
• Revise curricular materials based on class testing
• Expand class-testing beyond sample modules to entire units
• Begin disseminating curriculum ideas, research findings through conference presentations

• Analyze evaluation data, inform curriculum development
• Complete draft of curricular materials
• Continue disseminating curriculum ideas, research findings through conference presentations

• Continue class-testing, collecting and analyzing evaluation data

Unit I: Making Comparisons (binary categorical response)

• observational study vs. experiment; confounding, randomization
• basic terminology: observational/experimental unit, variable, explanatory and response variables
• descriptive analyses of two-way tables: conditional proportions, bar graphs, relative risk
• statistical significance, empirical p-value via simulation of randomization test for 2x2 tables
• causation, scope of conclusions
• effect of sample size
  

Unit II: Making Comparisons (quantitative response)

• graphical displays: histograms, stemplots, boxplots
• features of distributions: shape, center, spread, outliers
• numerical summaries: mean, median, std dev, inter-quartile range, five-number summary
• properties (e.g., resistance)
• empirical p-value via simulation of randomization test
• matched pairs design: rationale, simulation of randomization test
• much repetition of key ideas: significance, randomization, scope of conclusions

Unit III: Generalizing Results (binary categorical response)

• sampling, random sampling, sampling variability, sampling distribution
• basic terminology: population vs. sample, parameter vs. statistic, null and alternative hypotheses
• empirical p-value via simulation of (binomial) sampling distribution
• two-sided tests
• confidence interval as inversion of test
  

Unit IV: Generalizing Results (quantitative response)

• bootstrapping for standard error, confidence interval
• test as dual of bootstrap interval
  

Unit V: Normal-Based Approximations (binary cat. response)

• Basics: density curves, normal model, z-scores, normal calculations
• one-proportion z-test, confidence interval for proportion
• two-proportion z-test, confidence interval for difference in proportions, odds ratio
  

Unit VI: Normal-Based Approximations (quantitative response)

• t-distribution, motivation, comparison to z
• one-sample t-test, paired t-test, two-sample t-test
  

Unit VII: Relationships Between Variables

• two categorical variable: empirical randomization test, chi-square test
• categorical explanatory, quantitative response: empirical randomization test, ANOVA
• two quantitative variables: scatterplots, correlation, least squares lines, empirical randomization test, t-test for regression
• quantitative explanatory, categorical response: logistic regression

Unit VIII: Design for Data Production

• experiments and observational studies revisited
• link between randomization and generalization
• strategies for experiments: blocking
• strategies for sampling: stratification
• factorial crossing: two or more factors at once

For advisors: click here.