Traditional AI methods analyze fixed regions or patches, which can miss sparse cancer signals across billions of pixels in a whole slide.

AI that mimics human pathologists’ dynamic, context-aware analysis improves cancer detection by moving beyond fixed patches and preselected regions.

SlideChat consistently outperforms models like GPT-4o, LLaVA-Med, Quilt-LLaVA, HistoGPT, and PRISM on tasks such as tumor subtyping, staging, and context-rich report generation.

Training data come from multiple public and controlled-access cohorts, including TCGA and HISTAI, with proper approvals and curation.

The system shows transparent reasoning via question-guided attention heatmaps and ablation studies, highlighting the slide-level encoder and two-stage training as essential, while noting occasional cross-turn inconsistencies and hallucinations.

Applications span education and clinical decision support, enabling interactive teaching and rapid extraction of staging and prognostic information from whole slides.

SlideChat was trained on SlideInstruction, a dataset of over 274,000 multimodal samples with a strict quality-control process verified by pathologists.

The model overcomes patch-level limits by integrating a patch-level encoder with a slide-level encoder and connecting to a pretrained large language model for fluent, clinically relevant answers.

Training data, evaluation data, code, and model weights are publicly released on Hugging Face and GitHub to foster openness and further research.

The research moves toward slide-level clinical reasoning, though prospective validation, regulatory review, and reliability improvements are needed before clinical deployment.

SlideChat is a multimodal AI assistant that interprets gigapixel whole-slide images across 31 cancer types and generates diagnostic-style reports.

Evaluation across five cohorts and 31 cancer types shows SlideChat outperforms baselines on closed-ended questions by 19.1 percentage points, with higher-quality reports and top scores on open-ended questions by expert pathologists.