Back to blog

Statistics Flashcards: How to Learn Which Test to Use (Not Just the Formulas)

Stats exams rarely fail you on the formula. They fail you on picking the test. How to make scenario cards for t-tests, chi-square, ANOVA, and regression that transfer.

In an intro statistics course, the part that costs people the most marks is usually not the formula. Most exams give you a formula sheet, or the software does the arithmetic for you. What they cannot give you is the decision: you read a paragraph about a study, and you have to figure out, from the description alone, whether this is a paired t-test, a two sample t-test, a chi-square test, or a regression. So the most useful flashcards for statistics are not "what is the formula for a t statistic." They are short scenarios on the front, and the test plus the reason on the back.

(This is the Nebulearn blog, and Nebulearn makes a flashcard app, but these scenario cards work fine on paper. The method is the point here, not the tool.)

The moment it goes wrong

Here is how most people study for a stats exam. You go through your notes, chapter by chapter. Chapter seven is two sample t-tests. You read the section, you look at a worked example, you do two practice problems at the end of the chapter, and you get both right. Chapter eight is chi-square. Same thing, two practice problems, both right. You feel ready.

Then on the exam, question four describes a study where researchers measured the blood pressure of thirty patients before and after a new drug. And you think, "okay, two groups of numbers, before and after... that is a two sample t-test, right?" It is not. It is a paired t-test, because it is the same thirty people measured twice. You knew both tests. You got every practice problem right. What you never practiced was choosing between them, because every practice problem told you which chapter it was from.

That is the core issue. Practice problems at the end of a chapter test whether you can do the test. The exam tests whether you can pick the test. Those are different skills, and only one of them gets practiced if you study chapter by chapter. Flashcards vs practice problems covers the wider gap between doing problems and remembering things, and statistics is a very clean example of it.

The questions that decide the test

Choosing a test mostly comes down to answering a few questions about the study, always in roughly the same order. Learn these questions first, because every scenario card is really just practice at asking them.

  1. What type is the outcome? Is the thing being measured a number (blood pressure, test score, time) or a category (yes or no, which brand, pass or fail)?
  2. What is the predictor, and what type is it? Groups (treatment vs control), or another number (hours studied)?
  3. How many groups? One, two, or more than two?
  4. Are the observations independent or paired? Different people in each group, or the same people measured more than once, or matched pairs?
  5. Are the assumptions reasonable? Roughly normal data, or a large enough sample, or is it very skewed with a small sample?

Each of these can be its own card, with a short example, so you can recognize them quickly:

Front: "Thirty students took a quiz before and after a review session." Independent or paired?

Back: Paired. The same students are measured twice, so each before score has a matching after score.

The decision table

Here is the map most intro courses cover, written the way you would use it on an exam.

OutcomePredictorGroups or setupUsual test
NumberNone, compare to a known valueOne sampleOne sample t-test
NumberTwo groupsIndependent (different people)Two sample t-test
NumberTwo conditionsPaired (same people, or matched)Paired t-test
NumberThree or more groupsIndependentOne way ANOVA
NumberAnother numberRelationshipCorrelation or linear regression
CategoryAnother categoryCounts in a tableChi-square test of independence
CategoryNone, compare to expected proportionsOne variableChi-square goodness of fit
ProportionTwo groupsIndependentTwo proportion z-test
Number, very skewed, small sampleTwo groupsIndependent or pairedA nonparametric test, like Mann-Whitney or Wilcoxon signed rank

Your course may use slightly different names, and some courses do not cover the nonparametric row. Use your own course's table, but notice that the table itself should not be a card. A table with nine rows on the back of a card cannot be graded honestly, for the same reason no long list can: you remember half of it, see all of it, and call it close enough. The table is the reference. The cards are scenarios that make you walk through it.

How to write a scenario card

A good scenario card is a very short story with exactly the information you need to decide, and nothing that gives the answer away.

Front: A researcher wants to know if the proportion of people who prefer tea over coffee differs between three age groups. She surveys 200 people and records their age group and their preference. Which test?

Back: Chi-square test of independence. Both variables are categories (age group, preference), and she is asking if they are related.

Front: A company compares the average delivery time of three different couriers, using 40 different deliveries from each. Which test?

Back: One way ANOVA. The outcome is a number (time), there are three independent groups.

The back always says the test, then one line of reason in terms of the decision questions. That one line is the important part. It is what lets you get the next scenario right, the one you have never seen, because you are learning the reasoning and not just the pairing of this exact story with this exact answer.

A few rules that make these cards better:

  • Do not use the words from the test's name in the front. If the front says "the two groups are independent," you have given away half the answer. Describe the study and let yourself figure out that the groups are independent.
  • Write the story in plain language, the way exam questions do. "Before and after," "the same patients," "randomly assigned to one of two groups." Those phrases are what you are learning to notice.
  • Make near miss pairs. For every paired t-test card, make a two sample t-test card with an almost identical story. For every ANOVA card, a two sample t-test card with only two groups. The pairs are what teach you the boundaries.

The other cards worth having

Scenario cards should be most of the deck, but a few other kinds are worth adding.

Interpretation cards. Exams love to ask what a result means in plain words. "A p value of 0.03 means what, in this study?" Put the plain language answer on the back, and include the wrong interpretation too, because that is what the multiple choice options will try to trick you with. For example: "If there were really no effect, results at least this extreme would show up about 3 percent of the time. It does not mean there is a 3 percent chance the null hypothesis is true."

Assumption cards. "What is one assumption of a two sample t-test, and how do you check it?" One assumption per card.

Vocabulary cards. Type I error, type II error, power, confidence level. These are short definitions and they card easily. Use cloze cards if your course wants the exact wording.

Formula cards, only if your exam has no formula sheet. If it does have one, you do not need to memorize the formulas. You need to know what each symbol on the sheet means, and that is a better card anyway. Math formula flashcards covers how to card formulas when you do need them.

Mix the cards, always

This is the single most important habit in the whole deck. Do not keep your scenario cards in separate decks by chapter. Put them all in one deck and shuffle them.

If your t-test cards are in a "Chapter 7" deck and your chi-square cards are in a "Chapter 8" deck, then when you review the Chapter 7 deck, you already know the answer is some kind of t-test before you read the front. That is exactly the shortcut that failed on the exam. When everything is mixed, every card makes you actually decide, which is what the exam will make you do. This is called interleaving, and studying two subjects with interleaving explains why it works.

Better sources of scenarios

Your course's past exams are the best source of scenarios you will find. Take every past question that asks you to choose a test, strip out the numbers, and turn the story into a card. Those questions were written by the people grading you.

Statistics textbooks often have a decision flowchart in the back or on the inside cover. Those are great as a reference, and some people print one out and keep it on the desk while doing homework. That is a good idea, as long as you eventually practice without it. Many universities also put free stats help pages online that walk through "which test should I use" with examples, and they are worth reading.

Where a flashcard app like Anki or Nebulearn adds something is scheduling a large mixed deck so the scenarios you keep getting wrong come back sooner, and the easy ones get out of your way. In Nebulearn, FSRS is on by default, and you can make cards straight from your lecture slides or a PDF side by side. But the cards themselves are what matter.

When it stops working

The main way scenario cards go wrong is when you memorize the story instead of the reasoning. After a few weeks you will see the coffee and tea card and think, "oh, that is the coffee one, chi-square," without actually thinking about the variables at all. When that happens, the card has stopped testing the skill. It is the context dependency problem in miniature: you are reacting to the story, not to its structure. Rewrite it with a different story that has the same structure, or delete it and write a new one. A deck of scenarios should keep growing a little and changing a little, which is unusual for flashcards but right for this.

Doing the test vs choosing it

Statistics exams test two things: doing a test, and choosing a test. Practice problems handle the doing. Flashcards are best at the choosing, because they let you see a hundred short study descriptions, all mixed together, and make the decision each time. Write scenarios in plain language, put the test and one line of reason on the back, make near miss pairs, and never sort them by chapter.

StatisticsFlashcardsCard DesignExam Prep