Agenda

<aside> 📌 Reminders

• A2 is out </aside>

Score Test

<aside> 💡

Score test: a statistical hypothesis test used to compare a null hypothesis ($H_0$) against an alternative hypothesis ($H_1$), based on the derivative of the likelihood function.

</aside>

1. Idea:

   • Evaluate whether the slope of the log-likelihood function ($\ell(\theta)$) at the null hypothesis value $\theta_0$ indicates a significant deviation from $H_0$.

2. Test Statistic:

   $$ S = \frac{S(\theta_0)^2}{I(\theta_0)}, $$

   where,

   • $S(\theta_0) = \frac{\partial \ell(\theta)}{\partial \theta} \big|_{\theta = \theta_0}$ is the score function (first derivative of the log-likelihood).
   • $I(\theta_0) = -\mathbb{E}\left[\frac{\partial^2 \ell(\theta)}{\partial \theta^2}\right] \big|_{\theta = \theta_0}$ is the Fisher information.

3. Distribution:

   • Under $H_0$, $S \sim \chi^2_1$ (for one parameter).

4. When to Use:

   • Useful when the maximum likelihood estimate (MLE) under the alternative hypothesis is computationally challenging.

5. Example: Suppose $X_1, X_2, \dots, X_n \sim \mathcal{N}(\mu, \sigma^2)$, and we test $H_0: \mu = \mu_0$:

   • Log-likelihood: $\ell(\mu, \sigma^2) = -\frac{n}{2} \log(2\pi \sigma^2) - \frac{1}{2\sigma^2} \sum_{i=1}^n (X_i - \mu)^2$.
   • Score function: $S(\mu) = \frac{\partial \ell}{\partial \mu} = \frac{1}{\sigma^2} \sum_{i=1}^n (X_i - \mu)$.
   • Substitute $\mu_0$ into $S$ for the test statistic.

Wald Test

<aside> 💡

Wald test: another hypothesis test that uses the maximum likelihood estimate (MLE) to assess the validity of $H_0$.

</aside>

1. Idea:

   • Compare the distance between the MLE ($\hat{\theta}$) and the hypothesized value ($\theta_0$) in units of the standard error of $\hat{\theta}$.

2. Test Statistic:

   $$ W = \frac{(\hat{\theta} - \theta_0)^2}{\text{Var}(\hat{\theta})},  $$

   where:

   • $\hat{\theta}$ is the MLE of the parameter.
   • $\text{Var}(\hat{\theta})$ is estimated from the Fisher information.

3. Distribution:

   • Under $H_0$, $W \sim \chi^2_1$ (for one parameter).

4. When to Use:

   • Effective when the MLE can be computed directly and is sufficiently accurate.

5. Example: For $X_1, X_2, \dots, X_n \sim \mathcal{N}(\mu, \sigma^2)$, testing $H_0: \mu = \mu_0$:

   • $\hat{\mu} = \bar{X}$, $\text{Var}(\hat{\mu}) = \frac{\sigma^2}{n}$.
   • Test statistic:

   $$ W = \frac{(\bar{X} - \mu_0)^2}{\frac{\sigma^2}{n}}.  $$

Chi-square Goodness-of-Fit Test

<aside> 💡

Goodness-of-fit test: evaluates whether a sample comes from a specified distribution.

</aside>

• Null Hypothesis ($H_0$): The data follows the specified distribution.