that's very true when I make AI poetry I really can't tell the difference. The moment I say sound like Shakespeare it because difficult to tell this was not human 

PCA indicated that the random intercept for participants and the random slope for the identity of the poet were unnecessary and were causing the overparameterization. This conclusion is borne out in the data; looking at the proportion of “written by a human” responses for each participant, the variance is only 0.021; the variance between poets is only 0.00013. The lower-than-expected variance in the data simply does not support the complex random-effects structure. We therefore fit a reduced model with random intercepts for poems as the only random effect. Using ANOVA to compare model fit, we found that the full model containing our original set of random effects (npar = 76, AIC = 22385, BIC = 22970, logLik = -11116) did not provide a significantly better fit than the reduced model (npar = 21, AIC = 22292.5, BIC = 22454.2, logLik = -11125.2). We therefore proceed with the reduced model.

[Article](https://www.nature.com/articles/s41598-024-76900-1)

Participants were more likely to guess that AI-generated poems were written by humans than they were for actual human-written poems (χ2(2, N = 16340) = 247.04, w = 0.123, p < 0.0001). The five poems with the lowest rates of “human” ratings were all written by actual human poets; four of the five poems with the highest rates of “human” ratings were generated by AI.

## Study 1: distinguishing AI-generated from human-written poems
As specified in our pre-registration (https://osf.io/5j4w9), we predicted that participants would be at chance when trying to identify AI-generated vs. human-written poems, setting the significance level at 0.00518; p’s between 0.05 and 0.005 are “suggestive”. Observed accuracy was in fact slightly lower than chance (46.6%, χ2(1, N = 16340) = 75.13, p < 0.0001). Observed agreement between participants was poor, but was higher than chance (Fleiss’s kappa = 0.005, p < 0.001). Poor agreement suggests that, as expected, participants found the task very difficult, and were at least in part answering randomly. However, as in10, the below-chance performance and the significant agreement between participants led us to conclude that participants were not answering entirely at random; they must be using at least some shared, yet mistaken, heuristics to distinguish AI-generated poems from human-written poems.

We used a general linear mixed model logistic regression analysis (fit to a binomial distribution) to predict participant responses (“written by a human” or “generated by AI”) with poem’s authorship (human or AI), the identity of the poet, and their interaction as fixed effects. We used a sum coding for the identity of the poet, to interpret more easily the main effect of authorship across poets. As specified in our pre-registration, we initially included three random effects: random intercepts for participants (since we took 10 repeated measurements, one per poem, for each participant), random intercepts for poems, and random slopes for the identity of the poet for each poem. Following19, we used principal component analysis (PCA) to check for overparameterization, and determined that the model was indeed overparameterized.