MediScribe — AI Clinical Documentation Assistant
Turns typed or dictated consultation notes into structured medical summaries, action items, and patient-ready letters — in 19 languages.
Architecture
The pipeline runs in three stages: audio input is transcribed with Whisper, the transcript (or typed notes) is passed to an LLM structuring step that produces a clinical summary, an action-item list, and a patient-facing letter, and a translation layer generates the patient letter natively in the requested language rather than machine-translating a fixed English template.
- Whisper handles audio-to-text so dictation and typed notes share the same downstream pipeline
- A single structuring prompt produces three distinct outputs (summary, action items, letter) from one consultation note, keeping the pipeline to one LLM call per generation rather than three
- Translation is generated per-language rather than templated, so idioms and clinical tone hold up across all 19 supported languages instead of reading like a literal translation
Thought Process & Design Decisions
The core design bet was treating this as a product shell rather than a single script: authentication (Clerk), a persistent UI, and a reusable pipeline layer, so the same note-processing logic could sit behind different frontends later if needed.
Patient data handling was treated as a first-class constraint from the start rather than an afterthought — the data model and UI copy are written with HIPAA-style handling in mind, even ahead of a formal compliance pass.
Challenges & Trade-offs
- Balancing generation latency against output quality — clinicians want the summary fast, but action items and patient letters need to be accurate enough to trust without heavy editing
- Keeping multilingual output consistent in tone across languages with very different clinical/formal registers, rather than optimizing only for English quality