University of Florida / NVIDIA · Healthcare
GatorTron
One of the largest clinical language models at 8.9 billion parameters, trained on over 90 billion words of clinical and biomedical text.
Overview
GatorTron is a large-scale clinical language model developed through a collaboration between the University of Florida and NVIDIA. Trained on over 90 billion words combining clinical notes from the UF Health system, PubMed articles, and Wikipedia, it represents one of the most ambitious efforts to build a comprehensive clinical NLP model. GatorTron achieves state-of-the-art results on five major clinical NLP tasks and demonstrates the benefits of scaling in the healthcare domain.
Parameters
8.9B
Architecture
BERT-style transformer (Megatron-LM)
Training Data
90B+ words (clinical notes, PubMed, Wikipedia)
Context Window
512 tokens
GPU Training
992 NVIDIA A100 GPUs
Capabilities
Clinical concept extraction
Medical relation extraction
Semantic textual similarity in clinical context
Natural language inference for clinical text
Medical question answering
Use Cases
Automating extraction of medical concepts from clinical notes at scale
Building clinical decision support systems
De-identifying patient records for research use
Supporting clinical trial matching from unstructured EHR data
Pros
- +Massive scale captures nuanced clinical language patterns
- +Trained on diverse clinical and biomedical corpora
- +State-of-the-art on five clinical NLP benchmark tasks
- +Developed with real-world clinical note data
Cons
- -Very large model requiring significant compute resources
- -Clinical training data from a single health system may introduce bias
- -Complex deployment requiring specialized infrastructure
- -Research-focused; not directly production-ready
Pricing
Open-source for research use. Self-hosting requires substantial GPU infrastructure. NVIDIA NeMo framework recommended for deployment.