OASIS: A Rubric-Based Multimodal Assessment Platform Using Large Language Models

OASIS is infrastructure for making AI-assisted assessment reviewable. Given a scoring rubric and recorded video, audio or text, it produces a score for each criterion alongside supporting evidence and explanations. It also preserves the execution records and human-review status, allowing reviewers to inspect how an assessment was produced. The paper describes a system developed in medical education.

Its contribution is organizing the surrounding assessment process. The abstract lists rubric versioning, provenance capture and human review as what deployment at scale needs beyond scoring a single recording, and says the execution artifacts are preserved for audit. The publication provides a report and project information, rather than an installable software release.

“OASIS pairs a standalone command-line interface with a canonical integrated Elephant + MAPLES stack for encounter management and multimodal grading orchestration.”

“Distinctive features include rubric-as-program compilation, progressive execution plans, content-addressable grading identity, transcript-augmented multimodal grading, explicit review state, and a shared command surface for humans and autonomous agents.”

“In production at UT Southwestern Medical Center since Fall 2023, the platform has processed more than 7,000 encounters.”