Experimental Evidence on Negative Impact of Generative AI on Scientific Learning Outcomes. (arXiv:2311.05629v1 [cs.CY])

In this study, I explored the impact of Generative AI on learning efficacy in
academic reading materials using experimental methods. College-educated
participants engaged in three cycles of reading and writing tasks. After each
cycle, they responded to comprehension questions related to the material. After
adjusting for background knowledge and demographic factors, complete reliance
on AI for writing tasks led to a 25.1% reduction in accuracy. In contrast,
AI-assisted reading resulted in a 12% decline. Interestingly, using AI for
summarization significantly improved both quality and output. Accuracy
exhibited notable variance in the AI-assisted section. Further analysis
revealed that individuals with a robust background in the reading topic and
superior reading/writing skills benefitted the most. I conclude the research by
discussing educational policy implications, emphasizing the need for educators
to warn students about the dangers of over-dependence on AI and provide
guidance on its optimal use in educational settings.