AI’s risk to scientific trust is centered on concerns that AI-generated data, writing, and misrepresented findings could erode the novelty, accuracy, and integrity of scientific research and peer review.

Broader implications include the tension between using AI for large-scale data processing and preserving trust in science’s methods, with potential consequences for funding, publication, and real-world innovations if integrity is undermined.

Case studies illustrate the landscape: DeepMind’s AlphaFold has transformed biology, while AI models may overstate results in materials science and generate phantom papers or AI fingerprints across biomedical literature.

Detection and governance challenges persist, as AI-generated content is hard to distinguish from human work, detection tools remain imperfect, and there are calls for stronger human guardrails and accountability in AI-assisted research.

Impact on journals and peer review is evident as publishers like Elsevier adopt integrity signals and guidelines, arXiv bans invented references, and journals strive to preserve human oversight and responsibility.

Proposed safeguards offer a hopeful path: prioritize ‘research-grade AI’ with humans in the loop, ensure transparent processes and rigorous validation, and balance AI benefits with integrity, noting positive possibilities such as AI-assisted advances in cancer research.

AI’s influence on mathematics has sparked debate after a public dispute among mathematicians and a notable Navier–Stokes problem claim tied to OpenAI tools.