AI Scribe QA Guide

Scope: Verifying AI Scribe access gating end-to-end on dev, staging, and production. Covers facility-level enablement, per-capability toggles, and the practitioner-override surface.

Prerequisites: Backend + frontend dev servers running, an encounter that belongs to a facility you can configure, and (on staging/prod) Rails console access.

Companion docs: Developer detail · Product access model

1. Enabling AI Scribe for a facilityCopy Anchor Link

There is no feature flag any longer. Two columns and one optional API call. Pick whichever path matches the surface you're testing.

1.1 Rails console (dev / staging fast path)Copy Anchor Link

facility = Identity::Facility.find('<facility_id>')
settings = Identity::FacilityAiSettings.find_or_create_for_facility!(facility)
settings.update!(
  is_enabled: true,
  transcription_enabled: true,
  suggestions_enabled: true,        # optional — leave default if testing transcription only
  auto_documentation_enabled: false # optional
)

Shared::AiSettingsResolver.for(facility: facility).transcription_enabled?
# => true  (guard now passes)

1.2 Admin UI (clinic-admin self-serve)Copy Anchor Link

CMS → Settings → Practice Settings → AI Scribe tab.

• Signed in as a tenant owner or admin (or any membership with manage_facilities).
• Toggle the master switch + per-capability flags → Save.
• Round-trips through GET / PATCH /identity/facilities/:facility_id/ai_settings. Both endpoints idempotently materialize the facility_ai_settings row on first hit via find_or_create_for_facility!, so a brand-new facility can be enabled in one toggle without a separate seed step.

Use this when testing the end-to-end admin path. A practitioner-role membership should not see the nav surface; if they navigate directly, the policy returns 403.

1.3 Admin API (curl)Copy Anchor Link

curl -X PATCH 'https://<host>/identity/facilities/<facility_id>/ai_settings' \
  -H 'Authorization: Bearer <admin_token>' \
  -H 'Content-Type: application/json' \
  --data-raw '{"facility_ai_settings":{"is_enabled":true,"transcription_enabled":true}}'

Same gate as 1.2 — token must belong to a tenant owner/admin (or a membership with manage_facilities). Use this for scripted enablement or controller-level testing.

1.4 Brand-new facility checkCopy Anchor Link

Every facility created after the after_create :build_default_ai_settings callback (identity/models/facility.rb) ships with a settings row. Verify on a freshly created facility:

facility = Identity::Facility.create!(<minimum required attrs>)
facility.ai_settings.present?       # => true
facility.ai_settings.is_enabled     # => false (default — admin still has to turn it on)

If ai_settings is nil, the callback is wired wrong — file a bug.

2. Test casesCopy Anchor Link

2.1 Master toggle offCopy Anchor Link

Setup: is_enabled = false, transcription_enabled = true.

Expect:

• POST /clinical/transcription_sessions → 403 ai_scribe_disabled.
• AI Scribe sidebar in the encounter UI is hidden / disabled.
• Shared::AiSettingsResolver#transcription_enabled? returns false.

2.2 Capability toggle off (master on)Copy Anchor Link

Setup: is_enabled = true, transcription_enabled = false, suggestions_enabled = true.

Expect:

• POST /clinical/transcription_sessions → 403 ai_scribe_disabled.
• Suggestions endpoints work normally.
• Confirms capabilities are independently gated.

2.3 Both on — happy pathCopy Anchor Link

Setup: is_enabled = true, transcription_enabled = true.

Expect:

• POST /clinical/transcription_sessions → 201 Created.
• Streaming and finalize flows proceed end-to-end.

2.4 Practitioner-override smoke testCopy Anchor Link

Setup: Facility enabled. Create Identity::PractitionerAiSettings for a specific practitioner with language: 'hi', transcription_enabled: false.

Identity::PractitionerAiSettings.create!(
  tenant_id: facility.tenant_id,
  practitioner_id: practitioner.id,
  facility_id: facility.id,
  language: 'hi',
  transcription_enabled: false   # this MUST be ignored — it's not in PRACTITIONER_OVERRIDABLE_KEYS
)

resolver = Shared::AiSettingsResolver.for(facility: facility, practitioner: practitioner)
resolver.transcription_enabled? # => true  (facility wins, practitioner row ignored)
resolver.language               # => 'hi' (override applies — language is overridable)

Expect: practitioner cannot turn off transcription via their row; language override is respected.

2.5 Missing settings row (legacy facilities)Copy Anchor Link

Setup: A facility created before the auto-create callback existed, with no row in identity.facility_ai_settings.

Expect:

• Resolver returns false for every gate (treats nil as off).
• POST /clinical/transcription_sessions → 403 ai_scribe_disabled.
• Backfilling via find_or_create_for_facility! and flipping toggles unblocks the facility.

3. Reproducing the staging bug from issue #1388Copy Anchor Link

If you see 403 ai_scribe_disabled on staging for a facility that "worked locally":

1. Identify the facility ID from the failing request payload.
2. On a staging Rails console, check Identity::FacilityAiSettings.find_by(facility_id: '<id>').
3. If nil, run the Rails-console enablement recipe in §1.1.
4. If present but is_enabled / transcription_enabled is false, flip them.
5. Retry the failing API call.

This is configuration drift between environments, not a regression. There is no feature flag to seed; the columns are the only source of truth.

4. Regression checklist before shipping AI Scribe changesCopy Anchor Link

• §2.1 returns 403 (master toggle off blocks transcription).
• §2.2 returns 403 for transcription, 200 for suggestions (capabilities are independent).
• §2.3 returns 201 (happy path still works).
• §2.4 confirms practitioner row cannot disable transcription.
• New facility created in this run has a settings row (auto-create callback intact).
• Any new key added to Shared::AiSettingsResolver::PRACTITIONER_OVERRIDABLE_KEYS is reflected in the developer doc and reviewed with product.