Continued in-domain pre-training, teacher model training, and student distillation
Continued In-Domain Pre-training
1M+ Policies
Idle
Teacher Model
T5 11B
Idle
Distillation
Student Model (BERT)
Idle
Production Serving
Automated Compliance
Idle
Checks applies continued in-domain pre-training over 1M+ policies, trains a T5 11B teacher, and distills into BERT for production serving.