# Instructor

Professor Nathan Taback

Office: SS6027C

Office hours: TBD

# Class Time

Teaching Assistants: TBD

Classroom sessions: T 14:00-16:00, R 10:00-11:00

# Course Content

This course will provide an introduction to the fundamental concepts of the design of scientific studies including the design of experiments and observational studies. Students will be become acquainted with statistical methods used to design and analyze experiments and observational studies. In particular, this course will cover: experiments versus observational studies, clinical trial design, comparing several groups using a completely randomized design, randomized blocks, Latin squares, incomplete block designs, factorial designs, causal inference in randomized and non-randomized studies, and adjusting for selection bias using propensity score methods.

The learning objectives of this course are:

• Understand the ideas, principles, and considerations that are common to the design and analysis of scientific studies including the statistical design of experiments and observational studies.
• Develop a statistical toolbox of methods for the design and analysis of experiments and observational studies.
• Identify appropriate uses and interpretations of experimental designs, and observational studies, including their strengths and limitations.

## Topics

### Experiments, observational studies, and causal inference

Experiments versus observational studies, and causal inference in randomized experiments.

### Selection Bias in Observational Studies

Causal inference in randomized experiments versus observational studies. Introduction to the propensity score and three ways to use the propensity score to adjust for selection bias: matching; sub classification; direct regression adjustment.

### Probability and Statistics

Mathematical statistics used in experimental design.

### Comparing Several Groups

Comparing several groups in an experimental and observational setting and deciding whether differences that are found are likely to be real or due to chance.

### Power and Sample Size

Power and sample size will be introduced for several designs. Applications will include the design and analysis of clinical trials with continuous or binary endpoints.

### Blocking Techniques

Blocked designs, Latin squares, randomized incomplete block designs.

### Factorial Designs

Factorial, blocked factorial, and fractional factorial designs will be discussed.

### Split Plot Designs

Split plot designs will be discussed as an example of restricted randomization in the design of experiments.

# Course Books

## Optional

1. Statistics for Experimenters: Design, Innovation, and Discovery. Box, G.E.P., Hunter, J.S., Hunter, W.G. Wiley 2nd Ed. 2005

2. Design and Analysis of Experiments. Dean, A., and Voss, D. Springer. 1999. UofT link to electronic copy: http://go.utlib.ca/cat/2573215

3. Design of Observational Studies. Rosenbaum, P. R. Springer 2010. UofT link to electronic copy: http://go.utlib.ca/cat/7890274

4. Experiments: planning, analysis, and optimization. Wu, C.F.J., Hamada, M.S. Wiley, 2009, 2nd ed.

5. Causal inference for statistics, social, and biomedical sciences. Imbens and Rubin. Cambridge University Press, 2015. http://go.utlib.ca/cat/10127748

NB: Textbooks 2,3, 5 are available electronically through the UofT library (i.e., electronic copies of both these textbooks are available at no extra cost)

# Course Materials, including lecture notes

Course materials are provided for the exclusive use of enrolled students. Do not share them with others. I do not want to discover that a student has put any of my materials into the public domain, has sold my materials, or has given my materials to a person or company that is using them to earn money. The University will support me in asserting and pursuing my rights, and my copyrights, in such matters.

# Evaluation

Students will be evaluated according to the University Assessment and Grading Practices Policy.

Undergraduate students will be evaluated according to the following marking scheme.

Weight Date Time
Term Test #1 20% Oct. 8 14:00
Term Test #2 20% Nov. 19 14:00
Draft/proposal project 5% TBD TBD
Final project 10% Dec. 5 23:59
Final Exam 45% Scheduled by Faculty

The tests will be written during class time (14:10 – 15:40) in a location to be announced.

You are allowed a two-sided 8-1/2“x 11” (standard letter size) hand-written aid sheet on the term test and a two-sided hand-written aid sheet on the final exam. You must bring your student identification to the term tests and the final exam.

You will not need to know R syntax on the tests and exam, but you will need to know how to interpret output from R.

# Class Schedule

A tentative class schedule can be found here.

## Marking concerns

Any requests to have marked work re-evaluated must be made in writing to the instructor within one week of the date the work was returned. The request must contain a justification for consideration.

## Missed Tests

• If a test is missed for a valid reason, you must submit documentation to the course instructor.

• If a test is missed for a valid medical reason, you must submit the University of Toronto Verification of Student Illness or Injury form to your instructor within one week of the test.

• The form will only be accepted as valid if the form is filled out according to the instructions on the form.

• The form must indicate that the degree of incapacitation on academic functioning is moderate, serious, or severe in order to be considered a valid medical reason for missing the term test. If the form indicates that the degree of incapacitation on academic functioning is negligible or mild then this will NOT be considered a valid medical reason.

• If a test is missed for a valid reason then half the weight of the test will be shifted to the other midterm and half will be shifted to the final exam. In this case the other term test will be worth 30% and the final exam will be worth 55%.

• If a student misses BOTH term tests for any reason then an oral exam with members of the teaching team will be scheduled at a mutually convenient time in lieu of the two term tests worth 40%.

• Students must complete at least one midterm test or oral exam. If a student misses both midterm tests and does not take an oral exam before the end of term then a grade of zero will be assigned to the term work.

• Other reasons for missing a test will require prior approval by your instructor. If prior approval is not received for a non-medical reason, then you will receive a term test grade of zero.

## Late Project Submission

If the draft/propsal project or final project is submitted after the due date then a late penalty of 20% per day (i.e., 24 hours) will be applied to the part of the project handed in late. For example, if the draft/proposal project is submitted after 5 days (including weekend days) then you will receive a grade of zero for the draft/proposal.

# Computing

We will use R for all examples. R is freely available for download at http://cran.r-project.org for Windows, Mac, and Linux operating systems. For the tests and exam, you will need to know how to interpret output from R.

## Jupyter Notebook

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. (Ref: https://jupyter.org)

R can be run in a Jupyter notebook in any web brwoser by logging into https://utoronto.syzygy.ca with your UTORid.

To get started using R in a Jupyter notebook see this page

## RStudio

RStudio is a fantastic integrated development environment (IDE) for R. It is freely available at https://www.rstudio.com/products/rstudio/

I am assuming that students have never used R before. I will provide you with the R syntax for all examples in lecture, which should be sufficient for you to complete the practice problems.

# Calculators

You will need a calculator. Any calculator that has logarithmic functions will be sufficient. Calculators on phones or other devices equipped to communicate with the outside world (for example, through the internet or cellular or satellite phone networks) will not be permitted during the term tests or final exam.

# Online Discussion Board

This term you will have the option to use Piazza for class discussion. If you decide not to use Piazza it will not disadvantage you in any way, and will not affect official University outcomes (e.g., grades and learning opportunities). If you choose not to opt-into Piazza then you can ask questions or discuss course material with the instructor or TAs during office hours.

The Piazza system is highly catered to getting you help fast and efficiently from classmates, the TA, and the lecturers. Rather than emailing questions to the teaching staff, we encourage you to post your questions on Piazza. To sign up for the discussion forum click on the link :

https://piazza.com/utoronto.ca/fall2019/sta305h1004/home

Need extra help with the coursework? Here are some options: - For continued class discussion and questions outside of class, try posting on the discussion forums. The instructor and TAs will be monitoring them regularly. - You can visit your instructor or the teaching assistants during their office hours.

E-mail should only be used for emergencies or personal matters.

# How to communicate with your instructor

Questions about course material such as:

• How do I do question 3.7 in the textbook?

• What is standard deviation?

• When is the midterm?

Can be posted on the discussion forums. Questions can be posted anonymously (so that the author is anonymous to other students but not to the instructors), if desired.

For private communication, such as: I missed the test because I was ill e-mail your instructor.

Use your utoronto.ca e-mail account to ensure that your message doesn’t automatically go to a Junk folder and include your full name and student number.

# Religious Accommodation

As a student at the University of Toronto, you are part of a diverse community that welcomes and includes students and faculty from a wide range of cultural and religious traditions. For my part, I will make every reasonable effort to avoid scheduling tests, examinations, or other compulsory activities on religious holy days not captured by statutory holidays. Further to University Policy, if you anticipate being absent from class or missing a major course activity (such as a test or in-class assignment) due to a religious observance, please let me know as early in the course as possible, and with sufficient notice (at least two to three weeks), so that we can work together to make alternate arrangements.

# Students with Disabilities or Accommodation Requirements

Students with diverse learning styles and needs are welcome in this course. If you have an acute or ongoing disability issue or accommodation need, you should register with Accessibility Services (AS) at the beginning of the academic year by visiting http://www.studentlife.utoronto.ca/as/new-registration. Without registration, you will not be able to verify your situation with your instructors, and instructors will not be advised about your accommodation needs. AS will assess your situation, develop an accommodation plan with you, and support you in requesting accommodation for your course work. Remember that the process of accommodation is private: AS will not share details of your needs or condition with any instructor, and your instructors will not reveal that you are registered with AS.