Quantitative Methods for
International Politics
IPOL 3270 • Fall 2026
November 8, 2026
Evidentiary standards
How to test any hypothesis
Interpreting tests
Other δs
Math-based tests
Problems with p-values
Parameters are the things we
want to know about the population
| Estimand | Parameter | Statistic | Example |
|---|---|---|---|
| Proportion | p | \widehat{p} | Tunisians who have thought about emigrating |
| Mean | \mu | \bar{x} | Average age of Tunisian adults |
| Difference in proportions | p_1 - p_2 | \widehat{p}_1 - \widehat{p}_2 | Emigration: internet users vs. non-users |
| Difference in means | \mu_1 - \mu_2 | \bar{x}_1 - \bar{x}_2 | Age: urban vs. rural residents |
| Regression slope | \beta_1 | b_1 | Relationship between age and thinking about emigrating |
\delta
(delta)
Your estimate
Whatever statistic answers your research question
Is it accurate?
Is δ a good guess?
Did it pick up the population parameter?
Confidence intervals
Is it real?
Is δ real?
Could it maybe be zero?
Hypothesis tests and p-values
Is it substantive?
Does δ even matter?
Your knowledge of the world—no stats!
Do different groups of Tunisians think about emigrating at different rates?
| Comparison | δ | 95% CI |
|---|---|---|
| Women − men | 0.1 points | -3.8 to 3.9 |
| Internet users − non-users |
26.1 points | 22.3 to 29.9 |
UK, India, Canada, Australia, US, etc.
We never prove innocence; we try (and fail) to reject innocence
We’re afraid of punishing an innocent person!
“For if the Imam makes a mistake in forgiving it would be better than making mistake in punishment”
Jamiʿ at-Tirmidhi 1424
فَإِنَّ الإِمَامَ أَنْ يُخْطِئَ فِي الْعَفْوِ خَيْرٌ مِنْ أَنْ يُخْطِئَ فِي الْعُقُوبَةِ
جامع الترمذي 1424
We never prove that the null is true;
we try (and fail) to reject the null
Is there a difference between men’s and women’s attitudes toward emigration in Lebanon?
Null hypothesis
H_0: p_\text{women} - p_\text{men} = 0
No difference (“innocent”)
Alternative hypothesis
H_A: p_\text{women} - p_\text{men} \neq 0
Some difference (“guilty”)
α (alpha)
0.10
0.05
0.01
Your evidentiary standard: how often you’re willing to make a Type I error
There’s enough evidence to safely declare that δ is “real” and not zero
“Rejecting the null hypothesis”
The probability of seeing a δ at least that extreme in a world where there’s no effect
Wildly misunderstood—by everyone!
Simulations and null worlds


1: Find δ
2: Invent a world where δ is null
3: Look at δ in the null world
4: Find the probability that δ could exist in the null world
5. Decide!
Do Lebanese women and men think about emigrating at different rates?
| Gender | % yes | n |
|---|---|---|
| Women | 32.3% | 1,188 |
| Men | 36.1% | 1,206 |
δ = -3.7 points
Is that real though? Could it be positive? Could it be zero?
If gender doesn’t matter, the categories “woman” and “man” are meaningless
Bootstrap
Resample with replacement
Shows how much δ bounces around
type = "bootstrap"
→ Confidence intervals
Permutation
Shuffle without replacement
Shows what δ looks like when there’s no effect
type = "permute"
→ Hypothesis tests
library(infer)
# δ from the actual data
obs_diff <- lebanon |>
specify(
thought_emigrating ~ gender,
success = "Yes"
) |>
calculate(
stat = "diff in props",
order = c("Female", "Male")
)
# Simulate null world
null_world <- lebanon |>
specify(
thought_emigrating ~ gender,
success = "Yes"
) |>
hypothesize(null = "independence") |>
generate(reps = 1000, type = "permute") |>
calculate(
stat = "diff in props",
order = c("Female", "Male")
)specify(): What are the variables?hypothesize(): What does the null world look like? ("independence" = gender and emigration aren’t related)generate(): Make the null world by shuffling ("permute")calculate(): Find δ in each shuffled datasetProbability of seeing a δ at least as extreme as ours in this null world
# A tibble: 1 × 1
p_value
<dbl>
1 0.064
In a world where Lebanese women and men think about emigrating at the same rate, there’s a 6.4% chance that we’d see a difference at least as big as 3.7 percentage points
p = 0.064
Do we have enough evidence to say that we don’t live in the null world (i.e. to “convict”, or to “reject the null hypothesis”)?
andhs.co/nullworlds
| lower_ci | upper_ci |
|---|---|
| -0.076 | 0.0012 |
The 95% confidence interval includes 0
p is above 0.05
KINDA SORTA SHORTCUT: If a 95% CI includes 0, p > 0.05; if it excludes 0, p < 0.05
thought_emigrating ~ gender
thought_emigrating ~ uses_internet
δ = 0.1 points; p ≈ 1
Totally normal in the null world
δ = 26.1 points; p < 0.001
Way off the chart!
Open hypothesis-testing.qmd in the project

Reject H_0
p < α
δ doesn’t fit comfortably in the null world, so we’re pretty sure we’re not in it, so δ is probably not 0
“Statistically significant”
Fail to reject H_0
p ≥ α
δ could plausibly show up in the null world
“Not statistically significant”
If p ≥ α (like p = 0.387), you’ll want to say there’s no difference, or that δ is 0
No! Don’t! Bad!
You never “accept” the null hypothesis!
“We find the defendant innocent”
“We find the defendant not guilty”
If p is really close to α (like p = 0.052), you’ll want to say that δ is almost significant
No! Don’t! Bad!
It’s either “guilty”
or not!
“We find the defendant almost guilty”
“We find the defendant not guilty”
In a world where there is no δ, the probability of seeing a δ at least as extreme as ours is p
Then you judge if p is below your evidentiary threshold (0.10, 0.05, 0.01, etc.)
A study finds that a 5-point difference has p = 0.032.
A. There’s a 3.2% chance that the difference is just random
B. There’s a 3.2% chance that our hypothesis is wrong
C. In a world where there’s no difference, there’s a 3.2% chance of seeing a difference at least as extreme as 5 points. That’s below 0.05, so we say it’s statistically significant and reject the null.
C! A p-value is about the data in a null world, not about whether your hypothesis is true
In Lebanon, women are 3.7 points less likely than men to have thought about emigrating (p = 0.064).
A. There’s a 6.4% chance that the null hypothesis is true
B. If there were no gender gap among all Lebanese adults, we’d see a gap at least this big in about 6.4% of samples
C. Since p > 0.05, we’ve shown that Lebanese women and men think about emigrating at the same rate
D. With α = 0.10, we’d reject the null hypothesis
B and D! C is “accepting” the null
Everyone uses 0.05, but it’s not magical!
Do Kuwaiti women and men think about emigrating at different rates?
Do people in cities and rural areas have different average ages?
About 67% of the world used the internet in 2023. Is Tunisia different from the rest of the world?
Are older Tunisians less likely to have thought about emigrating?
| Difference in proportions | Difference in means | One proportion | One mean | Regression slope | |
|---|---|---|---|---|---|
specify() |
y ~ group,success = "Yes" |
y ~ group |
response = y,success = "Yes" |
response = y |
y ~ x |
hypothesize() |
null ="independence" |
null ="independence" |
null = "point",p = 0.67 |
null = "point",mu = 40 |
null ="independence" |
generate() |
type ="permute" |
type ="permute" |
type ="draw" |
type ="bootstrap" |
type ="permute" |
calculate() |
stat ="diff in props" |
stat ="diff in means" |
stat ="prop" |
stat ="mean" |
stat ="slope" |
No!
Same intuition:
how weird is δ in a world where it’s 0?
| Method | p |
|---|---|
| Simulation (1,000 shuffles) | 0.064 |
| Math (χ² test) | 0.059 |
Basically the same answer!
Do people in cities and rural areas have different average ages?
| estimate | lower_ci | upper_ci | statistic | p_value |
|---|---|---|---|---|
| 1.19 | 0.64 | 1.75 | 4.24 | 2.31e-05 |
statistic: how many standard errors δ is away from 0p_value: the probability of seeing that in the null worldlower_ci and upper_ci: the 95% confidence interval| term | estimate | std_error | statistic | p_value | lower_ci | upper_ci |
|---|---|---|---|---|---|---|
| intercept | 0.9755 | 0.0275 | 35.5153 | 0 | 0.9217 | 1.0294 |
| age | -0.0124 | 0.0006 | -21.2940 | 0 | -0.0135 | -0.0112 |
statistic is a t statistic, and p_value comes from a t distributionp_value of 0 isn’t really 0; it’s just super tiny (report it as p < 0.001)Even if you never simulate a p-value again, thinking about null worlds is valuable!
Every math-based p-value you see still measures the same thing: how surprising would this δ be in a world where it’s actually 0?


| Lives in | Average age | n |
|---|---|---|
| Rural | 38.5 | 3,702 |
| Urban | 39.7 | 10,342 |
Is δ real? Use stats. Does δ matter? That’s up to you!
Test enough things and something will be significant
“Hack Your Way To Scientific Glory”
Decide on your hypotheses and α before you look at the data, and report everything you tested
Preregistration