DeveloperCare plansCare planLhr
AI Transcription Service
Overview
The AI Transcription service provides real-time voice-to-text conversion and intelligent clinical documentation assistance during patient encounters. It helps physicians by automatically transcribing conversations and suggesting structured clinical data.
Purpose: Reduce documentation burden on healthcare providers while improving accuracy and completeness of medical records.
Why AI Transcriptions are Needed
Current Documentation Challenges
- Time Consuming: Physicians spend 2-3 hours documenting for every hour of patient care
- Interrupts Care: Manual typing during visits reduces eye contact and patient engagement
- Inconsistent Quality: Rushed documentation often lacks detail or contains errors
- Cognitive Load: Remembering to document everything while focusing on patient care
LHR Benefits with AI Transcription
- Improved Accuracy: Real-time capture prevents forgotten details
- Enhanced Patient Experience: More natural conversations without typing interruptions
- Structured Data Extraction: AI automatically identifies symptoms, medications, diagnoses
- Longitudinal Context: AI can reference patient history for more intelligent suggestions
- Quality Improvement: Consistent, comprehensive documentation across all encounters
Database Schema
AI Transcriptions Table
# db/migrate/[timestamp]_create_ai_transcriptions.rb
class CreateAiTranscriptions < ActiveRecord::Migration[7.0]
def change
create_table :ai_transcriptions do |t|
# Reference to the encounter being documented
t.references :encounter, null: false, foreign_key: { to_table: :encounters }
t.references :created_by_user, null: false, foreign_key: { to_table: :users }
# Audio and transcription data
t.text :raw_transcript # Complete transcribed text
t.jsonb :transcript_segments, default: [] # Time-stamped segments
t.text :cleaned_transcript # Processed/cleaned version
# AI-extracted clinical entities
t.jsonb :extracted_entities, default: {} # Symptoms, medications, procedures, etc.
t.jsonb :suggested_codes, default: [] # ICD/SNOMED code suggestions
t.jsonb :ai_suggestions, default: {} # Structured clinical suggestions
# Processing metadata
t.string :transcription_status, default: 'processing' # processing, completed, failed
t.string :ai_processing_status, default: 'pending' # pending, processing, completed, failed
t.decimal :transcription_confidence, precision: 3, scale: 2 # 0.00 to 1.00
t.float :processing_duration_seconds
t.text :processing_errors
# Audio file information
t.string :audio_file_path # S3 or local path to audio file
t.integer :audio_duration_seconds
t.string :audio_format # 'wav', 'mp3', 'webm'
t.integer :audio_file_size_bytes
# User interaction tracking
t.jsonb :accepted_suggestions, default: [] # Which suggestions were accepted
t.jsonb :rejected_suggestions, default: [] # Which suggestions were rejected
t.jsonb :modified_suggestions, default: [] # Which suggestions were modified
t.boolean :physician_reviewed, default: false
t.datetime :reviewed_at
# Integration with encounter data
t.boolean :auto_populated_encounter, default: false # Did AI auto-fill encounter fields?
t.jsonb :populated_fields, default: [] # Which encounter fields were auto-populated
t.timestamps
end
# Indexes for performance
add_index :ai_transcriptions, :transcription_status
add_index :ai_transcriptions, :ai_processing_status
add_index :ai_transcriptions, :physician_reviewed
add_index :ai_transcriptions, :created_at
add_index :ai_transcriptions, [:encounter_id, :created_at]
# Full-text search on transcripts
add_index :ai_transcriptions, :raw_transcript, using: :gin
add_index :ai_transcriptions, :cleaned_transcript, using: :gin
end
endTranscription Sessions (for continuous recording)
# db/migrate/[timestamp]_create_transcription_sessions.rb
class CreateTranscriptionSessions < ActiveRecord::Migration[7.0]
def change
create_table :transcription_sessions do |t|
# Session identification
t.references :encounter, null: false, foreign_key: { to_table: :encounters }
t.references :user, null: false, foreign_key: true
t.string :session_id, null: false # UUID for real-time updates
# Session timing
t.datetime :started_at, null: false
t.datetime :ended_at
t.integer :total_duration_seconds
# Real-time transcription data
t.text :live_transcript # Continuously updated during session
t.jsonb :live_suggestions, default: {} # Real-time AI suggestions
# Session metadata
t.string :status, default: 'active' # active, paused, completed, cancelled
t.string :recording_quality # excellent, good, fair, poor
t.jsonb :session_settings, default: {} # User preferences for this session
# Multiple transcription chunks per session
t.has_many :ai_transcriptions, dependent: :destroy
t.timestamps
end
add_index :transcription_sessions, :session_id, unique: true
add_index :transcription_sessions, :status
add_index :transcription_sessions, [:encounter_id, :started_at]
end
endModel Implementation
AI Transcription Model
# app/models/ai_transcription.rb
class AiTranscription < ApplicationRecord
belongs_to :encounter
belongs_to :created_by_user, class_name: 'User'
belongs_to :transcription_session, optional: true
# Enums for better organization
enum transcription_status: {
processing: 'processing',
completed: 'completed',
failed: 'failed',
cancelled: 'cancelled'
}
enum ai_processing_status: {
pending: 'pending',
processing: 'processing',
completed: 'completed',
failed: 'failed'
}
# Validations
validates :encounter_id, :created_by_user_id, presence: true
validates :transcription_confidence, inclusion: { in: 0.0..1.0 }, allow_nil: true
# Callbacks
after_create :start_transcription_processing
after_update :trigger_ai_processing, if: :saved_change_to_transcription_status?
after_update :populate_encounter_fields, if: :saved_change_to_ai_processing_status?
# Scopes
scope :recent, -> { order(created_at: :desc) }
scope :by_confidence, ->(min_confidence) { where('transcription_confidence >= ?', min_confidence) }
scope :reviewed, -> { where(physician_reviewed: true) }
scope :unreviewed, -> { where(physician_reviewed: false) }
# Instance methods
def processing_time
return nil unless processing_duration_seconds
"#{processing_duration_seconds.round(2)} seconds"
end
def confidence_percentage
return nil unless transcription_confidence
(transcription_confidence * 100).round(1)
end
def extracted_symptoms
extracted_entities['symptoms'] || []
end
def extracted_medications
extracted_entities['medications'] || []
end
def extracted_diagnoses
extracted_entities['diagnoses'] || []
end
def extracted_vitals
extracted_entities['vitals'] || {}
end
def suggestion_acceptance_rate
total_suggestions = ai_suggestions.values.flatten.size
return 0 if total_suggestions == 0
accepted_count = accepted_suggestions.size
(accepted_count.to_f / total_suggestions * 100).round(1)
end
def mark_as_reviewed(reviewing_user)
update!(
physician_reviewed: true,
reviewed_at: Time.current
)
end
def accept_suggestion(suggestion_id, suggestion_type)
accepted_suggestions << {
id: suggestion_id,
type: suggestion_type,
accepted_at: Time.current
}
save!
end
def reject_suggestion(suggestion_id, suggestion_type, reason = nil)
rejected_suggestions << {
id: suggestion_id,
type: suggestion_type,
reason: reason,
rejected_at: Time.current
}
save!
end
def modify_suggestion(suggestion_id, suggestion_type, original_value, modified_value)
modified_suggestions << {
id: suggestion_id,
type: suggestion_type,
original_value: original_value,
modified_value: modified_value,
modified_at: Time.current
}
save!
end
# Audio file management
def audio_file_url
return nil unless audio_file_path
if Rails.env.production?
# Return S3 signed URL
AWS::S3.presigned_url(audio_file_path, expires_in: 1.hour)
else
# Return local file path
Rails.application.routes.url_helpers.rails_blob_path(audio_file_path)
end
end
def delete_audio_file
return unless audio_file_path
if Rails.env.production?
AWS::S3.delete_object(audio_file_path)
else
File.delete(audio_file_path) if File.exist?(audio_file_path)
end
update!(audio_file_path: nil)
end
private
def start_transcription_processing
# Queue transcription job
TranscriptionProcessingJob.perform_later(id)
end
def trigger_ai_processing
return unless transcription_status == 'completed'
# Queue AI processing job
AiProcessingJob.perform_later(id)
end
def populate_encounter_fields
return unless ai_processing_status == 'completed'
return if auto_populated_encounter
# Queue encounter population job
PopulateEncounterFromAiJob.perform_later(id)
end
endService Layer
AI Transcription Service
# app/services/ai_transcription_service.rb
class AiTranscriptionService
def initialize
@transcription_provider = Rails.configuration.transcription_provider || 'mock'
@ai_processing_provider = Rails.configuration.ai_processing_provider || 'mock'
end
# Start a new transcription session
def start_session(encounter_id, user_id, options = {})
session = TranscriptionSession.create!(
encounter_id: encounter_id,
user_id: user_id,
session_id: SecureRandom.uuid,
started_at: Time.current,
status: 'active',
session_settings: options
)
Rails.logger.info "Started transcription session #{session.session_id} for encounter #{encounter_id}"
session
end
# Process audio chunk and return transcription
def process_audio_chunk(session_id, audio_data, options = {})
session = TranscriptionSession.find_by!(session_id: session_id)
# Create transcription record
transcription = session.ai_transcriptions.create!(
encounter: session.encounter,
created_by_user: session.user,
transcription_status: 'processing',
audio_format: options[:format] || 'webm',
audio_duration_seconds: options[:duration] || 0
)
# Store audio file
audio_path = store_audio_file(audio_data, transcription.id, options[:format])
transcription.update!(audio_file_path: audio_path)
# Start transcription processing
case @transcription_provider
when 'aws_transcribe'
process_with_aws_transcribe(transcription)
when 'google_speech'
process_with_google_speech(transcription)
when 'openai_whisper'
process_with_openai_whisper(transcription)
else
process_with_mock_transcription(transcription)
end
transcription
end
# Get live transcription updates for a session
def get_live_updates(session_id)
session = TranscriptionSession.find_by!(session_id: session_id)
{
session_id: session.session_id,
status: session.status,
live_transcript: session.live_transcript,
live_suggestions: session.live_suggestions,
recent_transcriptions: session.ai_transcriptions
.recent
.limit(5)
.includes(:encounter)
}
end
# End transcription session
def end_session(session_id)
session = TranscriptionSession.find_by!(session_id: session_id)
session.update!(
status: 'completed',
ended_at: Time.current,
total_duration_seconds: (Time.current - session.started_at).to_i
)
# Trigger final AI processing for the entire session
ProcessSessionTranscriptionJob.perform_later(session.id)
Rails.logger.info "Ended transcription session #{session_id}"
session
end
# Extract clinical entities from transcript
def extract_clinical_entities(transcript_text)
case @ai_processing_provider
when 'aws_comprehend_medical'
extract_with_aws_comprehend(transcript_text)
when 'google_healthcare_nlp'
extract_with_google_healthcare(transcript_text)
when 'openai_gpt'
extract_with_openai_gpt(transcript_text)
else
extract_with_mock_ai(transcript_text)
end
end
# Generate clinical suggestions from extracted entities
def generate_clinical_suggestions(entities, encounter_context)
suggestions = {
complaints: generate_complaint_suggestions(entities[:symptoms], encounter_context),
medications: generate_medication_suggestions(entities[:medications], encounter_context),
diagnoses: generate_diagnosis_suggestions(entities[:diagnoses], encounter_context),
vitals: generate_vital_suggestions(entities[:vitals], encounter_context),
tests: generate_test_suggestions(entities, encounter_context)
}
# Filter out low-confidence suggestions
filter_suggestions_by_confidence(suggestions)
end
# Auto-populate encounter fields from AI suggestions
def auto_populate_encounter(transcription_id, confidence_threshold = 0.8)
transcription = AiTranscription.find(transcription_id)
encounter = transcription.encounter
populated_fields = []
return if transcription.auto_populated_encounter
ActiveRecord::Base.transaction do
# Auto-populate high-confidence suggestions
suggestions = transcription.ai_suggestions
# Populate complaints
if suggestions['complaints']&.any?
high_confidence_complaints = suggestions['complaints'].select { |c| c['confidence'] > confidence_threshold }
high_confidence_complaints.each do |complaint|
encounter.encounter_complaints.create!(
name: complaint['text'],
problem_status: 'active',
onset_date: Date.current,
coding_method: 'ai_generated',
coding_confidence: complaint['confidence']
)
populated_fields << "complaint: #{complaint['text']}"
end
end
# Populate medications (be more conservative with medications)
if suggestions['medications']&.any?
medication_threshold = [confidence_threshold + 0.1, 0.9].min
high_confidence_meds = suggestions['medications'].select { |m| m['confidence'] > medication_threshold }
high_confidence_meds.each do |medication|
encounter.encounter_medicines.create!(
medicine_name: medication['name'],
dosage: medication['dosage'],
frequency: medication['frequency'],
duration_unit: medication['duration_value'],
duration_frequency: medication['duration_unit'],
medication_status: 'active',
prescription_type: medication['type'] || 'acute'
)
populated_fields << "medication: #{medication['name']}"
end
end
# Populate vitals
if suggestions['vitals']&.any?
vitals_data = suggestions['vitals']
if vitals_data['confidence'] > confidence_threshold
# Create or update patient vitals
vital_record = encounter.patient.patient_vitals.build(
systolic_bp: vitals_data['systolic_bp'],
diastolic_bp: vitals_data['diastolic_bp'],
heart_rate: vitals_data['heart_rate'],
temperature: vitals_data['temperature'],
weight: vitals_data['weight'],
created_at: encounter.created_at
)
vital_record.save! if vital_record.has_valid_vitals?
populated_fields << "vitals: #{vitals_data.keys.join(', ')}" if vital_record.persisted?
end
end
# Mark as auto-populated
transcription.update!(
auto_populated_encounter: true,
populated_fields: populated_fields
)
end
Rails.logger.info "Auto-populated encounter #{encounter.id} with: #{populated_fields.join('; ')}"
populated_fields
end
private
def store_audio_file(audio_data, transcription_id, format)
filename = "transcription_#{transcription_id}_#{Time.current.to_i}.#{format}"
if Rails.env.production?
# Store in S3
s3_path = "transcriptions/#{Date.current.strftime('%Y/%m/%d')}/#{filename}"
AWS::S3.upload(s3_path, audio_data)
s3_path
else
# Store locally
dir_path = Rails.root.join('tmp', 'transcriptions')
FileUtils.mkdir_p(dir_path)
file_path = dir_path.join(filename)
File.binwrite(file_path, audio_data)
file_path.to_s
end
end
# Mock transcription implementation
def process_with_mock_transcription(transcription)
# Simulate processing delay
sleep(1) if Rails.env.development?
# Generate mock transcript
mock_transcript = generate_mock_transcript(transcription.encounter)
transcription.update!(
raw_transcript: mock_transcript,
cleaned_transcript: clean_transcript(mock_transcript),
transcription_status: 'completed',
transcription_confidence: rand(0.85..0.98).round(2),
processing_duration_seconds: rand(2.0..5.0)
)
end
# Mock AI entity extraction
def extract_with_mock_ai(transcript_text)
# Simple keyword-based extraction for demo purposes
{
symptoms: extract_mock_symptoms(transcript_text),
medications: extract_mock_medications(transcript_text),
diagnoses: extract_mock_diagnoses(transcript_text),
vitals: extract_mock_vitals(transcript_text),
procedures: extract_mock_procedures(transcript_text)
}
end
def extract_mock_symptoms(text)
symptom_keywords = ['headache', 'fever', 'cough', 'pain', 'nausea', 'dizzy', 'tired', 'fatigue']
found_symptoms = []
symptom_keywords.each do |symptom|
if text.downcase.include?(symptom)
found_symptoms << {
text: symptom,
confidence: rand(0.7..0.95).round(2),
severity: ['mild', 'moderate', 'severe'].sample,
duration: "#{rand(1..7)} days"
}
end
end
found_symptoms
end
def extract_mock_medications(text)
medication_patterns = [
/(\w+cillin)\s+(\d+\s*mg)/i, # Antibiotics
/(ibuprofen|acetaminophen|aspirin)\s+(\d+\s*mg)/i, # Pain relievers
/(metformin|insulin)\s+(\d+\s*mg|\d+\s*units)/i # Diabetes meds
]
found_medications = []
medication_patterns.each do |pattern|
text.scan(pattern) do |match|
found_medications << {
name: match[0],
dosage: match[1],
confidence: rand(0.8..0.95).round(2)
}
end
end
found_medications
end
def extract_mock_vitals(text)
vitals = {}
# Blood pressure pattern: "120 over 80" or "120/80"
if text =~ /(\d+)\s*(?:over|\/)\s*(\d+)/
vitals[:systolic_bp] = $1.to_i
vitals[:diastolic_bp] = $2.to_i
vitals[:confidence] = 0.9
end
# Heart rate pattern: "72 beats per minute" or "HR 72"
if text =~ /(?:heart rate|hr)\s*:?\s*(\d+)|(\d+)\s*(?:beats per minute|bpm)/i
vitals[:heart_rate] = ($1 || $2).to_i
vitals[:confidence] = [vitals[:confidence] || 0, 0.85].max
end
# Temperature pattern: "98.6 degrees"
if text =~ /(\d+\.?\d*)\s*(?:degrees|°)/i
vitals[:temperature] = $1.to_f
vitals[:confidence] = [vitals[:confidence] || 0, 0.85].max
end
vitals
end
def generate_mock_transcript(encounter)
patient = encounter.patient
complaint_text = encounter.encounter_complaints.pluck(:name).join(', ')
# Generate realistic medical conversation
transcript_parts = []
transcript_parts << "Good morning, #{patient.title}. How are you feeling today?"
transcript_parts << "I've been experiencing #{complaint_text.presence || 'some discomfort'} for the past few days."
transcript_parts << "Can you describe the symptoms in more detail?"
# Add mock vital signs discussion
if rand > 0.5
transcript_parts << "Let me check your vital signs. Your blood pressure is 120 over 80, heart rate is 72 beats per minute."
transcript_parts << "Temperature is 98.6 degrees. These all look normal."
end
# Add mock medication discussion
if rand > 0.6
transcript_parts << "Are you currently taking any medications?"
transcript_parts << "I'm taking metformin 500 mg twice daily for diabetes."
transcript_parts << "Good, continue with that. I'm also going to prescribe some ibuprofen 400 mg for the pain."
end
transcript_parts << "I'd like to schedule a follow-up appointment in two weeks."
transcript_parts << "Please call if symptoms worsen or if you have any concerns."
transcript_parts.join(' ')
end
def clean_transcript(raw_transcript)
# Basic transcript cleaning
cleaned = raw_transcript.dup
# Remove filler words and clean up
filler_words = ['um', 'uh', 'er', 'ah', 'like', 'you know']
filler_words.each do |filler|
cleaned.gsub!(/\b#{filler}\b/i, '')
end
# Clean up extra spaces
cleaned.gsub!(/\s+/, ' ')
cleaned.strip
end
endBackground Jobs
Transcription Processing Jobs
# app/jobs/transcription_processing_job.rb
class TranscriptionProcessingJob < ActiveJob::Base
queue_as :transcription
def perform(transcription_id)
transcription = AiTranscription.find(transcription_id)
service = AiTranscriptionService.new
Rails.logger.info "Processing transcription #{transcription_id}"
# Process the audio file to get transcript
service.send(:process_with_mock_transcription, transcription)
rescue => e
Rails.logger.error "Transcription processing failed for #{transcription_id}: #{e.message}"
transcription.update!(
transcription_status: 'failed',
processing_errors: e.message
)
end
end
# app/jobs/ai_processing_job.rb
class AiProcessingJob < ActiveJob::Base
queue_as :ai_processing
def perform(transcription_id)
transcription = AiTranscription.find(transcription_id)
service = AiTranscriptionService.new
return unless transcription.raw_transcript.present?
Rails.logger.info "AI processing transcription #{transcription_id}"
# Extract clinical entities
entities = service.extract_clinical_entities(transcription.raw_transcript)
# Generate clinical suggestions
suggestions = service.generate_clinical_suggestions(entities, transcription.encounter)
# Update transcription with AI results
transcription.update!(
extracted_entities: entities,
ai_suggestions: suggestions,
ai_processing_status: 'completed'
)
rescue => e
Rails.logger.error "AI processing failed for #{transcription_id}: #{e.message}"
transcription.update!(
ai_processing_status: 'failed',
processing_errors: e.message
)
end
end
# app/jobs/populate_encounter_from_ai_job.rb
class PopulateEncounterFromAiJob < ActiveJob::Base
queue_as :encounter_population
def perform(transcription_id)
service = AiTranscriptionService.new
populated_fields = service.auto_populate_encounter(transcription_id)
Rails.logger.info "Auto-populated encounter from transcription #{transcription_id}: #{populated_fields.join(', ')}"
rescue => e
Rails.logger.error "Failed to populate encounter from transcription #{transcription_id}: #{e.message}"
end
endAPI Endpoints
Transcription API Controllers
# Add to routes
namespace :ai_transcription do
post :start_session
post :process_audio
get :live_updates
post :end_session
resources :transcriptions, only: [:show, :index] do
member do
post :accept_suggestion
post :reject_suggestion
post :modify_suggestion
post :mark_reviewed
end
end
end
# app/controllers/api/ai_transcription_controller.rb
class Api::AiTranscriptionController < ApiController
before_action :authenticate_user!
# POST /api/ai_transcription/start_session
def start_session
encounter = Encounter.find(params[:encounter_id])
# Check permissions
authorize encounter, :update?
service = AiTranscriptionService.new
session = service.start_session(encounter.id, current_user.id, session_params)
render json: {
success: true,
session: {
session_id: session.session_id,
encounter_id: session.encounter_id,
status: session.status,
started_at: session.started_at
}
}
end
# POST /api/ai_transcription/process_audio
def process_audio
session_id = params[:session_id]
audio_data = params[:audio_data] # Base64 encoded audio data
service = AiTranscriptionService.new
# Decode audio data
decoded_audio = Base64.decode64(audio_data)
transcription = service.process_audio_chunk(
session_id,
decoded_audio,
format: params[:format] || 'webm',
duration: params[:duration]&.to_i
)
render json: {
success: true,
transcription: {
id: transcription.id,
status: transcription.transcription_status,
confidence: transcription.transcription_confidence
}
}
end
# GET /api/ai_transcription/live_updates?session_id=xxx
def live_updates
service = AiTranscriptionService.new
updates = service.get_live_updates(params[:session_id])
render json: {
success: true,
updates: updates
}
end
# POST /api/ai_transcription/end_session
def end_session
service = AiTranscriptionService.new
session = service.end_session(params[:session_id])
render json: {
success: true,
session: {
session_id: session.session_id,
status: session.status,
ended_at: session.ended_at,
duration: session.total_duration_seconds
}
}
end
# POST /api/ai_transcription/transcriptions/:id/accept_suggestion
def accept_suggestion
transcription = AiTranscription.find(params[:id])
authorize transcription.encounter, :update?
transcription.accept_suggestion(
params[:suggestion_id],
params[:suggestion_type]
)
render json: { success: true }
end
# POST /api/ai_transcription/transcriptions/:id/mark_reviewed
def mark_reviewed
transcription = AiTranscription.find(params[:id])
authorize transcription.encounter, :update?
transcription.mark_as_reviewed(current_user)
render json: { success: true }
end
private
def session_params
params.permit(:recording_quality, :auto_populate, :confidence_threshold)
end
endIntegration with Encounter Flow
Simple Encounter Enhancement
# Enhanced encounter model to support AI transcription
class Encounter < ApplicationRecord
has_many :ai_transcriptions, dependent: :destroy
has_many :transcription_sessions, dependent: :destroy
# Check if encounter has AI transcription data
def has_ai_transcription?
ai_transcriptions.completed.any?
end
def latest_transcription
ai_transcriptions.completed.recent.first
end
def transcription_summary
return nil unless has_ai_transcription?
latest = latest_transcription
{
transcript: latest.cleaned_transcript,
confidence: latest.transcription_confidence,
extracted_entities: latest.extracted_entities,
suggestion_acceptance_rate: latest.suggestion_acceptance_rate,
reviewed: latest.physician_reviewed
}
end
# Simple callback to handle AI suggestions after encounter creation
after_create :setup_ai_transcription_if_enabled
private
def setup_ai_transcription_if_enabled
# Only setup if AI transcription is enabled for this clinic
return unless clinic&.ai_transcription_enabled?
# Create a placeholder for potential AI transcription
# Actual transcription starts when user begins recording
Rails.logger.info "Encounter #{id} ready for AI transcription"
end
end
# Simple integration with patient history save
# Enhanced parts/clinic/services/patient_history/save_history.rb
module PatientHistory
class SaveHistory < ActiveJob::Base
def self.perform(obj, key)
# ... existing legacy save logic ...
legacy_result = perform_legacy_save(obj, key)
# Add AI transcription processing for careplans
if legacy_result && key == 'CAREPLAN' && obj.has_ai_transcription?
# Process AI transcription data in longitudinal health context
LongitudinalHealth::SaveLongitudinalData.perform(obj, 'ENCOUNTER_WITH_AI_DATA')
end
legacy_result
end
end
endUsage Flow Example
Complete AI Transcription Workflow
// Frontend JavaScript example for AI transcription workflow
class AITranscriptionManager {
constructor(encounterId) {
this.encounterId = encounterId;
this.sessionId = null;
this.mediaRecorder = null;
this.isRecording = false;
}
// 1. Start transcription session
async startSession() {
try {
const response = await fetch('/api/ai_transcription/start_session', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
encounter_id: this.encounterId,
auto_populate: true,
confidence_threshold: 0.8
})
});
const data = await response.json();
this.sessionId = data.session.session_id;
console.log('AI transcription session started:', this.sessionId);
// Start live updates polling
this.startLiveUpdates();
} catch (error) {
console.error('Failed to start transcription:', error);
}
}
// 2. Start audio recording
async startRecording() {
try {
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
this.mediaRecorder = new MediaRecorder(stream);
this.mediaRecorder.ondataavailable = (event) => {
if (event.data.size > 0) {
this.processAudioChunk(event.data);
}
};
this.mediaRecorder.start(1000); // Send chunks every second
this.isRecording = true;
} catch (error) {
console.error('Failed to start recording:', error);
}
}
// 3. Process audio chunks
async processAudioChunk(audioBlob) {
const reader = new FileReader();
reader.onload = async () => {
const base64Audio = btoa(reader.result);
try {
await fetch('/api/ai_transcription/process_audio', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
session_id: this.sessionId,
audio_data: base64Audio,
format: 'webm',
duration: 1
})
});
} catch (error) {
console.error('Failed to process audio:', error);
}
};
reader.readAsArrayBuffer(audioBlob);
}
// 4. Get live updates
startLiveUpdates() {
this.updateInterval = setInterval(async () => {
try {
const response = await fetch(
`/api/ai_transcription/live_updates?session_id=${this.sessionId}`
);
const data = await response.json();
// Update UI with live transcript and suggestions
this.updateTranscriptUI(data.updates.live_transcript);
this.updateSuggestionsUI(data.updates.live_suggestions);
} catch (error) {
console.error('Failed to get updates:', error);
}
}, 2000);
}
// 5. End session
async endSession() {
if (this.mediaRecorder && this.isRecording) {
this.mediaRecorder.stop();
this.isRecording = false;
}
clearInterval(this.updateInterval);
try {
await fetch('/api/ai_transcription/end_session', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ session_id: this.sessionId })
});
console.log('Transcription session ended');
} catch (error) {
console.error('Failed to end session:', error);
}
}
// UI update methods
updateTranscriptUI(transcript) {
document.getElementById('live-transcript').textContent = transcript;
}
updateSuggestionsUI(suggestions) {
const container = document.getElementById('ai-suggestions');
// Render suggestions with accept/reject buttons
// Implementation depends on your frontend framework
}
}
// Usage in encounter page
const aiTranscription = new AITranscriptionManager(encounterId);
aiTranscription.startSession();Benefits of AI Transcription Integration:
- Reduced Documentation Time: Physicians spend less time typing, more time with patients
- Improved Accuracy: Real-time capture prevents missed details
- Enhanced Longitudinal Records: AI suggestions improve completeness of LHR data
- Clinical Decision Support: AI can suggest diagnoses and medications based on conversation
- Quality Improvement: Consistent, comprehensive documentation across encounters
Next: See service-architecture.md for the simplified encounter saving implementation.