---
title: Person fit with PP
author: Jan Steinfeld and Manuel Reif
output: rmarkdown::html_vignette
vignette: >
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteIndexEntry{Person fit with PP}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>", fig.width = 6, fig.height = 4)
```

A person fit statistic tells whether the answers of a person fit the model: a person who solves hard items but fails easy ones, who guesses, or who stops working in the middle of a test, produces an unlikely response pattern. `Pfit()` computes four statistics at the ability estimate of each person:

- **lz** (Drasgow, Levine & Williams, 1985): the standardized log-likelihood of the response pattern. Negative values indicate misfit. lz is standard normal only if the true ability is known.
- **lz\*** (Snijders, 2001): lz corrected for the estimated ability, so that it is asymptotically standard normal. PP uses the general form for dichotomous and polytomous items (Sinharay, 2016), with the correction for the estimation type used (MLE, WLE or MAP).
- **infit** and **outfit** (Wright & Masters, 1990): information-weighted and unweighted mean squares of the standardized residuals, with an expected value of 1, and their Wilson-Hilferty t transformations `in_t` and `ou_t`. Values above 1 (t > 0) indicate noisy answers; values below 1 indicate answers that are more predictable than the model expects.

All statistics are available for every model in PP (1PL to 4PL, PCM, GPCM and mixed tests) and for estimates of type `"mle"`, `"wle"` and `"map"`. Missing responses are allowed; each person is assessed on the items they answered.

## An example

We simulate a 3PL test of 20 items and 300 persons. Fifteen of them answer at random, a typical source of misfit.

```{r data}
library(PP)
set.seed(1337)
b  <- seq(-2.5, 2.5, length.out = 20)
sl <- round(runif(20, 0.7, 1.6), 2)
la <- round(runif(20, 0.05, 0.2), 2)
x  <- sim_4pl(beta = b, alpha = sl, lowerA = la, upperA = 1, theta = rnorm(300))
x[1:15, ] <- matrix(rbinom(15 * 20, 1, 0.5), 15)     # random answers
```

Person fit needs the ability estimates first. `Pfit()` takes the responses from the estimation object.

```{r pfit}
res <- PP_4pl(respm = x, thres = b, slopes = sl, lowerA = la, type = "wle")
fit <- Pfit(res, fitindices = c("lz", "lzstar", "infit", "outfit"))
```

The result is a list with one matrix per statistic:

```{r parts}
head(fit$lzstar)
head(fit$infit)
```

`summary()` gives an overview, including the number of persons flagged at level `alpha` (two-sided, for lz, lz\* and the t values):

```{r summary}
s <- summary(fit, alpha = 0.05)
```

Misfit shows up as low values of lz\*, so we use a one-sided test here. How many of the random responders does lz\* find?

```{r flagged}
flag <- which(fit$lzstar[, "lzstar"] < qnorm(0.05))
flag
mean(1:15 %in% flag)
```

```{r plot, fig.alt = "lz* of all persons against their ability estimate, random responders in red"}
plot(res$estimates$estimate, fit$lzstar[, "lzstar"],
     col = ifelse(seq_len(300) <= 15, "red", "grey40"), pch = 19,
     xlab = "WLE estimate", ylab = "lz*")
abline(h = qnorm(0.05), lty = 2)
```

Which answers make these persons conspicuous? `plot(res, which = "profile")` shows one panel per person: each answered item is a point, with the expected score at the estimate on the x axis (easy items on the left) and the observed score on the y axis. Answers in line with the model lie close to the diagonal; unexpected answers (|z| > 1.96) are marked and labelled, here right answers to hard items and wrong answers to easy ones. `plot(res, which = "fit")` draws the person fit map with the two-sided flags of `summary()`.

```{r profile, fig.width = 7, fig.height = 4, fig.alt = "Observed against expected score of every item for two random responders, with unexpected answers marked"}
plot(res, which = "profile", person = flag[1:2])
```

## Standard errors of the fit statistics

With `SE = TRUE`, `Pfit()` adds jackknife standard errors of the statistics (columns `lz_se`, `lzs_se`, `in_se`, `ou_se`): every answered item is left out once, the ability is estimated again and the statistic is recomputed.

```{r se}
fit_se <- Pfit(res, fitindices = c("lzstar", "outfit"), SE = TRUE)
head(fit_se$lzstar)
```

## Polytomous and mixed tests

The same call works for the (generalized) partial credit model and for mixed tests. Missing responses are skipped.

```{r gpcm}
thres <- rbind(c(-1.5, -1, -0.5, 0, 0.5, 1),
               c(0, 0.5, 1, 1.2, 1.5, 2))
xg <- sim_gpcm(thres = thres, alpha = c(1, 1.2, 0.8, 1.1, 0.9, 1), theta = rnorm(200))
xg[sample(length(xg), 100)] <- NA
resg <- PP_gpcm(respm = xg, thres = thres, slopes = c(1, 1.2, 0.8, 1.1, 0.9, 1), type = "mle")
fitg <- Pfit(resg)
summary(fitg)
```

Persons whose estimate does not have status `"ok"` (for example `"extreme"` with MLE, or `"boundary"`) get `NA`. For persons who answered only one dichotomous item, lz\* is `NA` as well. `PPass()` computes the estimates and the person fit in one step and returns them as one table (`vignette("ppass_erm")`).

## Interpretation

The lz and lz\* values are interpreted as z values: the smaller (more negative), the stronger the misfit. Persons below a critical value, e.g. `qnorm(0.05)` = −1.64 one-sided, are flagged (Magis, Raîche & Béland, 2012). `summary()` uses a two-sided test, so it also counts persons with very high values, whose answers fit better than expected. lz itself is not standard normal when the ability is estimated (Molenaar & Hoijtink, 1990); its variance is too small, so it flags too few persons. lz\* corrects this and should be preferred.

## References

Drasgow, F., Levine, M. V., & Williams, E. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. *British Journal of Mathematical and Statistical Psychology, 38*(1), 67–86.

Magis, D., Raîche, G., & Béland, S. (2012). A didactic presentation of Snijders's lz\* index of person fit with emphasis on response model selection and ability estimation. *Journal of Educational and Behavioral Statistics, 37*(1), 57–81.

Molenaar, I. W., & Hoijtink, H. (1990). The many null distributions of person fit indices. *Psychometrika, 55*(1), 75–106.

Sinharay, S. (2016). Asymptotically correct standardization of person-fit statistics beyond dichotomous items. *Psychometrika, 81*(4), 992–1013.

Snijders, T. A. B. (2001). Asymptotic null distribution of person fit statistics with estimated person parameter. *Psychometrika, 66*(3), 331–342.

Wright, B. D., & Masters, G. N. (1990). Computation of OUTFIT and INFIT statistics. *Rasch Measurement Transactions, 3*(4), 84–85.