Kulcare Docs
DeveloperSearch

Typesense Search Integration

Last Updated: 2026-05-31

Overview

Kulcare uses Typesense as the primary search engine for public entity discovery (facilities and practitioners). Typesense replaced the legacy Elasticsearch 6.2.4 setup for the following reasons:

  • Simpler operations: Typesense is a single binary with no JVM dependency, compared to Elasticsearch's Java-based stack with significant memory overhead.
  • Typo tolerance built in: Typesense handles fuzzy matching out of the box without custom analyzer configuration.
  • Geo-search native: First-class geopoint field type with radius filtering and distance-based sorting.
  • Faceted search: Built-in faceting support without aggregation queries.
  • Lower resource footprint: Runs on a fraction of the memory that Elasticsearch requires.

The system includes an automatic PostgreSQL fallback so the application remains functional if Typesense is unavailable.

Note: earlier revisions of this guide described a :typesense_search_enabled feature flag. No such flag exists in the catalog (backend/lib/data/basic_feature_flags.json) and nothing in the code reads one — Typesense search is always on, with the fallback as its safety net.

Architecture

+-------------------+      +-------------------+      +------------------+
|  Frontend         |      |  Rails API        |      |  Typesense       |
|  (React Query)    | ---> |  SearchController | ---> |  Server          |
|                   |      |                   |      |                  |
|  useFacilitySearch|      |  search service   |      |  facilities_env  |
|  usePractitioner  |      |  Typesense / PG   |      |  practitioners_  |
|  Search           |      |  fallback         |      |  env             |
+-------------------+      +--------+----------+      +------------------+
                                    |
                                    v (fallback)
                           +-------------------+
                           |  PostgreSQL        |
                           |  ILIKE queries     |
                           +-------------------+

+-------------------+      +-------------------+      +------------------+
|  Model callbacks  | ---> |  TypesenseSyncQueue| ---> |  TypesenseSyncJob|
|  (after_commit)   |      |  (buffer table)   |      |  (Sidekiq)       |
+-------------------+      +-------------------+      +------------------+

Key components

ComponentLocationPurpose
Typesense initializerbackend/config/initializers/typesense.rbClient setup, health check on boot
Shared client wrapperbackend/domains/shared/services/typesense/client.rbError-handling wrapper around TYPESENSE_CLIENT
Base collectionbackend/domains/shared/services/typesense/base_collection.rbAbstract base with search, index, reindex methods
Facilities collectionbackend/domains/identity/services/typesense/facilities_collection.rbSchema + serialization for facilities
Practitioners collectionbackend/domains/identity/services/typesense/practitioners_collection.rbSchema + serialization for practitioner roles
Search controllerbackend/domains/identity/controllers/search_controller.rbPublic API endpoints with fallback
Sync queue modelbackend/domains/shared/models/typesense_sync_queue.rbBuffer table for async sync
Sync jobbackend/domains/shared/jobs/typesense_sync_job.rbSidekiq job that processes the sync queue
Indexable concernbackend/domains/shared/concerns/typesense_indexable.rbafter_commit callbacks for models
Rake tasksbackend/lib/tasks/typesense.rakeCLI management commands
Frontend APIfrontend/packages/shared-services/src/domains/identity/api.tssearchFacilities, searchPractitioners
Frontend hooksfrontend/packages/shared-services/src/domains/identity/hooks.tsuseFacilitySearch, usePractitionerSearch
Frontend typesfrontend/packages/shared-services/src/domains/identity/types.tsFacilitySearchParams, PractitionerSearchResult, etc.
Frontend query keysfrontend/packages/shared-services/src/domains/identity/keys.tsidentityKeys.search.*

Setup

Typesense server

Install and run Typesense locally for development:

# macOS (Homebrew)
brew install typesense/tap/typesense-server
typesense-server --data-dir=/tmp/typesense-data --api-key=dev_typesense_api_key

# Docker
docker run -p 8108:8108 \
  -v /tmp/typesense-data:/data \
  typesense/typesense:27.1 \
  --data-dir /data \
  --api-key=dev_typesense_api_key

Environment variables

Credentials are read from Rails credentials first, then fall back to environment variables:

VariableDefaultDescription
TYPESENSE_URLhttp://localhost:8108Typesense server URL
TYPESENSE_API_KEYdev_typesense_api_keyAdmin API key

For Rails credentials (preferred for staging/production):

rails credentials:edit
typesense_url: "https://typesense.example.com"
typesense_api_key: "your_production_api_key"

Feature flag

Typesense search is not feature-flagged; it is enabled wherever the Typesense connection is configured.

When disabled, all search requests fall through to the PostgreSQL fallback.

Bootstrap collections

After starting Typesense, create the collections and populate them:

bundle exec rake typesense:health
bundle exec rake typesense:create_collections
bundle exec rake typesense:reindex

Collections

Facilities collection

Collection name: facilities_<Rails.env> (e.g., facilities_development)

Source: Identity::Facility with Identity::FacilityLocation

Schema:

FieldTypeFacetNotes
idstringnoFacility ID
tenant_idstringyes
unique_idstringnoOptional
namestringnoPrimary search field
display_namestringnoFalls back to name
service_levelstringyese.g., hospital, clinic
citystringyesFrom primary location
statestringyes
country_codestringyes
postal_codestringnoNot indexed
locationgeopointno[lat, lng] for geo-search
offers_telemedicineboolyes
is_virtual_onlyboolno
bed_countint32no
activeboolnoAlways filtered to true
specialtiesstring[]yesArray of specialty names
phonestringnoNot indexed
popularity_scoreint32noDefault sort field

Search fields: name, display_name, city, specialties

Practitioners collection

Collection name: practitioners_<Rails.env> (e.g., practitioners_development)

Source: Identity::PractitionerRole (denormalized with Identity::Practitioner and Identity::Facility)

Each document represents one PractitionerRole. A practitioner at 3 clinics produces 3 documents.

Schema:

FieldTypeFacetNotes
idstringnoPractitionerRole ID
practitioner_idstringno
namestringnoFull name
display_namestringno
genderstringyes
qualificationsstring[]noQualification codes
languagesstring[]yes
avatar_urlstringnoNot indexed
biostringnoNot indexed
tenant_idstringyes
facility_idstringyes
facility_namestringno
role_typestringyes
practitioner_typestringyes
statusstringyes
specialtystringyesSpecialty name
consultation_feefloatnoFilterable by range
is_primary_facilityboolno
search_keywordsstring[]noDerived keywords
citystringyes
statestringyes
country_codestringyes
locationgeopointnoFrom facility's primary location
activeboolnoDerived from status == 'active'
popularity_scoreint32noDefault sort field

Search fields: name, display_name, specialty, facility_name, city, qualifications, search_keywords

API Endpoints

Both endpoints are public (no authentication required) and defined in backend/domains/identity/routes.rb:

get 'facilities/search', to: 'search#facilities'
get 'practitioners/search', to: 'search#practitioners'

GET /identity/facilities/search

Search for facilities by name, location, or filters.

Parameters:

ParameterTypeDescription
qstringSearch query (default: * for all)
latfloatLatitude for geo-search
lngfloatLongitude for geo-search
radiusintegerRadius in km (default: 10)
citystringFilter by city
service_levelstringFilter by service level
telemedicinestring'true' to filter telemedicine facilities
pageintegerPage number (default: 1)
per_pageintegerResults per page (default: 20, max: 250)

Example request:

GET /identity/facilities/search?q=downtown&city=Bangalore&page=1&per_page=10

Example response:

{
  "data": [
    {
      "id": "abc-123",
      "name": "Downtown Medical Center",
      "display_name": "Downtown Medical Center",
      "service_level": "hospital",
      "city": "Bangalore",
      "state": "Karnataka",
      "active": true,
      "specialties": ["Cardiology", "Orthopedics"],
      "offers_telemedicine": true
    }
  ],
  "meta": {
    "source": "typesense",
    "found": 42,
    "page": 1,
    "search_time_ms": 3
  }
}

GET /identity/practitioners/search

Search for practitioners by name, specialty, location, or filters.

Parameters:

ParameterTypeDescription
qstringSearch query (default: * for all)
specialtystringFilter by specialty name
citystringFilter by city
genderstringFilter by gender
max_feefloatMaximum consultation fee
role_typestringFilter by role type
latfloatLatitude for geo-search
lngfloatLongitude for geo-search
radiusintegerRadius in km (default: 10)
facetsstring'true' to include facet counts
pageintegerPage number (default: 1)
per_pageintegerResults per page (default: 20, max: 250)

Example request:

GET /identity/practitioners/search?q=cardio&specialty=Cardiology&facets=true

Example response:

{
  "data": [
    {
      "id": "role-456",
      "practitioner_id": "doc-789",
      "name": "Dr. Priya Sharma",
      "display_name": "Dr. Priya Sharma",
      "specialty": "Cardiology",
      "facility_name": "Heart Care Hospital",
      "city": "Mumbai",
      "consultation_fee": 800.0,
      "gender": "female",
      "qualifications": ["MBBS", "MD"],
      "languages": ["English", "Hindi"],
      "active": true
    }
  ],
  "meta": {
    "source": "typesense",
    "found": 15,
    "page": 1,
    "search_time_ms": 2
  }
}

When facets=true, the response includes facet counts for specialty, city, languages, gender, and role_type.

Both endpoints support geo-search. When lat and lng are provided, results are:

  1. Filtered to the specified radius (default: 10 km)
  2. Sorted by distance from the provided coordinates (nearest first)
GET /identity/facilities/search?lat=12.9716&lng=77.5946&radius=5

Indexing and Sync

How data gets into Typesense

There are two paths for data to reach Typesense:

1. Real-time sync via model callbacks

Models that include the TypesenseIndexable concern automatically enqueue sync entries on create, update, and destroy:

# backend/domains/shared/concerns/typesense_indexable.rb
module Shared::Concerns::TypesenseIndexable
  extend ActiveSupport::Concern

  included do
    after_commit :enqueue_typesense_upsert, on: %i[create update]
    after_commit :enqueue_typesense_delete, on: :destroy
  end
end

To add this to a model:

class Identity::Facility < Identity::BaseRecord
  include Shared::Concerns::TypesenseIndexable
end

2. Sync queue and background job

Model callbacks write to the typesense_sync_queue buffer table instead of calling Typesense directly. This ensures:

  • Database transactions complete before indexing
  • Batch processing for efficiency
  • Retry logic for transient failures
  • No impact on request latency

The Shared::TypesenseSyncJob is enqueued every 1 minute by the system crontab (config/schedule.rb, regenerated on each deploy via whenever/capistrano). It:

  1. Fetches up to 500 pending entries from the queue
  2. Deduplicates by (record_type, record_id) keeping the latest entry
  3. Groups entries by record type
  4. Batch-upserts or deletes documents in Typesense
  5. Marks entries as processed (processed_at = NOW()) or failed

History — see #1592: an earlier version of this doc claimed "every 5 seconds via sidekiq-scheduler". That was aspirational — the schedule never shipped, so the queue sat unread for weeks and search drifted from Postgres. The pipeline now runs from system cron (whenever), wired into Capistrano so deploys regenerate the crontab.

Special handling for Practitioners: When a Practitioner record changes, the job re-indexes all of that practitioner's PractitionerRole documents (since practitioner data is denormalized into role documents).

Cascade on Facility / FacilityLocation changes: practitioners_<env> denormalizes facility name, city, address, lat/lng, and phone onto every role doc. To prevent stale search data when a facility is renamed or moved:

  • Identity::Facility after_commit on update fans out to every PractitionerRole at the facility, but only when one of DENORMALIZED_TO_PRACTITIONERS (name, display_name, service_level, active) changes. Unrelated column touches don't trigger the fan-out.
  • Identity::FacilityLocation after_commit enqueues the parent facility AND every role at it, gated on DENORMALIZED_TO_FACILITY (address_line_1, city, latitude, longitude, is_primary, active, etc.).
  • Both fan-outs use Shared::TypesenseSyncQueue.bulk_enqueue — one INSERT … VALUES round-trip instead of N create! calls, so the after_commit doesn't stall the commit thread on large facilities.
  • Both callbacks rescue StandardError and log to Rails.logger.error. The queue is best-effort: a transient DB blip in the enqueue path must not 500 the user's update. The nightly reindex (below) is the safety net.

Retry and dead letter: Entries are retried up to 5 times (MAX_RETRIES). After that, they are considered dead-letter entries and can be inspected via Shared::TypesenseSyncQueue.dead_letter.

Nightly safety-net reindex: rake typesense:reindex runs at 3:00 AM (config/schedule.rb). It rebuilds both collections from scratch and catches anything the per-row enqueue missed — dead-lettered rows, transient DB failures, or new denormalized fields not yet wired into a cascade.

Queue pruning: Shared::TypesenseSyncQueue marks processed rows with processed_at but never deletes them. Without pruning the table would grow indefinitely (the Facility cascade alone can enqueue thousands of rows per day). rake typesense:prune_processed runs nightly at 3:30 AM and deletes rows where processed_at < 30.days.ago. The retention window is generous enough to support post-hoc missed-sync investigations; older rows are dead weight.

3. Full reindex via rake task

For initial population or recovery, use the rake task:

bundle exec rake typesense:reindex

This iterates over the source_scope in batches of 1000, serializes each record, and imports them into Typesense using the batch import API.

Rake Tasks

All rake tasks are defined in backend/lib/tasks/typesense.rake:

TaskDescription
typesense:healthCheck Typesense cluster health
typesense:create_collectionsCreate all Typesense collections (idempotent)
typesense:drop_collectionsDrop all collections (destructive)
typesense:recreate_collectionsDrop and recreate all collections (destructive)
typesense:reindexFull reindex of all collections from the database
typesense:reindex_collection[name]Reindex a specific collection (e.g., facilities or practitioners)
typesense:queue_statusShow pending, dead-letter, and total sync queue counts
typesense:prune_processed[days]Delete processed sync queue rows older than N days (default 30). Scheduled nightly at 3:30 AM.

Examples:

# Check health
bundle exec rake typesense:health

# Create collections (safe to run multiple times)
bundle exec rake typesense:create_collections

# Full reindex
bundle exec rake typesense:reindex

# Reindex only practitioners
bundle exec rake typesense:reindex_collection[practitioners]

# Check sync queue status
bundle exec rake typesense:queue_status
# Output:
#   Typesense Sync Queue:
#     Pending:     12
#     Dead letter: 0
#     Total:       1543

PostgreSQL Fallback

When Typesense is unavailable (server down, timeout, or feature flag disabled), the search controller falls through to PostgreSQL queries automatically.

How it works

  1. If disabled, it runs the PostgreSQL fallback directly.
  2. If enabled, it calls Typesense. If Typesense raises ServerError, TimeoutError, or HTTPError, the controller rescues and runs the PostgreSQL fallback.
  3. The response includes meta.fallback: true when the fallback was used, and meta.source: 'postgres'.
def facilities
  results = if typesense_enabled?
              search_facilities_typesense
            else
              search_facilities_postgres
            end

  render json: { data: results[:hits], meta: results[:meta] }
rescue ::Typesense::Error::ServerError,
       ::Typesense::Error::TimeoutError,
       ::Typesense::Error::HTTPError => e
  Rails.logger.warn("Typesense unavailable, PG fallback: #{e.message}")
  results = search_facilities_postgres
  render json: { data: results[:hits], meta: results[:meta].merge(fallback: true) }
end

Fallback limitations

The PostgreSQL fallback uses ILIKE queries, which means:

  • No typo tolerance
  • No geo-search (lat/lng parameters are ignored)
  • No faceting
  • Slower on large datasets
  • Results capped at 100 per page (vs 250 for Typesense)

The fallback ensures the application remains functional for basic text search while Typesense is being restored.

Frontend Integration

API functions

The frontend API layer is in frontend/packages/shared-services/src/domains/identity/api.ts:

import { apiClient } from '../../client';
import type {
  FacilitySearchParams,
  FacilitySearchResult,
  PractitionerSearchParams,
  PractitionerSearchResult,
  SearchResponse,
} from './types';

export async function searchFacilities(
  params?: FacilitySearchParams
): Promise<SearchResponse<FacilitySearchResult>> {
  const response = await apiClient.get<SearchResponse<FacilitySearchResult>>(
    '/identity/facilities/search',
    { params }
  );
  return response.data;
}

export async function searchPractitioners(
  params?: PractitionerSearchParams
): Promise<SearchResponse<PractitionerSearchResult>> {
  const response = await apiClient.get<SearchResponse<PractitionerSearchResult>>(
    '/identity/practitioners/search',
    { params }
  );
  return response.data;
}

React Query hooks

The hooks are in frontend/packages/shared-services/src/domains/identity/hooks.ts:

import { useQuery } from '@tanstack/react-query';
import { identityKeys } from './keys';
import { searchFacilities, searchPractitioners } from './api';
import type { FacilitySearchParams, PractitionerSearchParams } from './types';

export function useFacilitySearch(params?: FacilitySearchParams) {
  return useQuery({
    queryKey: identityKeys.search.facilities(params),
    queryFn: () => searchFacilities(params),
    enabled: !!(params?.q && params.q.length >= 2) || !!(params?.lat && params?.lng),
    staleTime: 60_000,
  });
}

export function usePractitionerSearch(params?: PractitionerSearchParams) {
  return useQuery({
    queryKey: identityKeys.search.practitioners(params),
    queryFn: () => searchPractitioners(params),
    enabled:
      !!(params?.q && params.q.length >= 2) ||
      !!(params?.lat && params?.lng) ||
      !!params?.specialty,
    staleTime: 60_000,
  });
}

Query key structure

Query keys use a hierarchical factory pattern for proper cache invalidation:

identityKeys.search.all()                   // ['identity', 'search']
identityKeys.search.facilities(params)      // ['identity', 'search', 'facilities', params]
identityKeys.search.practitioners(params)   // ['identity', 'search', 'practitioners', params]

Usage in components

import { useFacilitySearch, usePractitionerSearch } from '@kulcare/shared-services';

function DoctorFinder() {
  const [query, setQuery] = useState('');
  const [specialty, setSpecialty] = useState('');

  const { data, isLoading } = usePractitionerSearch({
    q: query,
    specialty,
    per_page: 20,
  });

  return (
    <div>
      <input value={query} onChange={(e) => setQuery(e.target.value)} />
      {isLoading && <Spinner />}
      {data?.data.map((practitioner) => (
        <DoctorCard key={practitioner.id} doctor={practitioner} />
      ))}
      <p>Found {data?.meta.found} results in {data?.meta.search_time_ms}ms</p>
      {data?.meta.fallback && <p>Using fallback search (limited functionality)</p>}
    </div>
  );
}

TypeScript types

All types are defined in frontend/packages/shared-services/src/domains/identity/types.ts:

interface FacilitySearchParams {
  q?: string;
  lat?: number;
  lng?: number;
  radius?: number;
  city?: string;
  service_level?: string;
  telemedicine?: string;
  page?: number;
  per_page?: number;
}

interface PractitionerSearchParams {
  q?: string;
  specialty?: string;
  city?: string;
  gender?: string;
  max_fee?: number;
  role_type?: string;
  lat?: number;
  lng?: number;
  radius?: number;
  facets?: string;
  page?: number;
  per_page?: number;
}

interface SearchResponse<T> {
  data: T[];
  meta: {
    source: 'typesense' | 'postgres';
    found: number;
    page: number;
    search_time_ms?: number;
    fallback?: boolean;
  };
}

Adding a New Collection

Follow these steps to index a new entity type in Typesense:

1. Create the collection class

Create a new file under the appropriate domain, inheriting from Shared::Typesense::BaseCollection:

# backend/domains/<domain>/services/typesense/<entity>_collection.rb
module <Domain>
  module Typesense
    class <Entity>Collection < Shared::Typesense::BaseCollection
      class << self
        def collection_name
          "<entity>_#{Rails.env}"
        end

        def schema
          {
            'name' => collection_name,
            'fields' => [
              { 'name' => 'id', 'type' => 'string' },
              { 'name' => 'name', 'type' => 'string' },
              # Add your fields here. See Typesense docs for field types:
              # string, int32, int64, float, bool, string[], geopoint
              # Use 'facet' => true for fields you want to filter on.
              # Use 'optional' => true for nullable fields.
              # Use 'index' => false for fields you only want to store, not search.
              { 'name' => 'active', 'type' => 'bool' },
              { 'name' => 'popularity_score', 'type' => 'int32' }
            ],
            'default_sorting_field' => 'popularity_score'
          }
        end

        def serialize(record)
          {
            'id' => record.id.to_s,
            'name' => record.name,
            'active' => record.active?,
            'popularity_score' => 0
          }.compact
        end

        def source_scope
          <Domain>::<Entity>.where(active: true)
        end

        protected

        def build_search_params(params)
          search = {
            'q' => params[:q].presence || '*',
            'query_by' => 'name',
            'per_page' => [(params[:per_page] || 20).to_i, 250].min,
            'page' => (params[:page] || 1).to_i
          }

          filters = []
          filters << "active:=true"
          # Add your filters here

          search['filter_by'] = filters.join(' && ') if filters.any?
          search
        end

        def default_query_by
          'name'
        end
      end
    end
  end
end

2. Add the model concern

Include TypesenseIndexable in the model so changes are synced automatically:

class <Domain>::<Entity> < <Domain>::BaseRecord
  include Shared::Concerns::TypesenseIndexable
end

3. Register in the sync job

Add the mapping to Shared::TypesenseSyncJob::COLLECTION_MAP:

COLLECTION_MAP = {
  'Identity::Facility' => 'Identity::Typesense::FacilitiesCollection',
  'Identity::PractitionerRole' => 'Identity::Typesense::PractitionersCollection',
  '<Domain>::<Entity>' => '<Domain>::Typesense::<Entity>Collection',
}.freeze

4. Register in rake tasks

Add your collection to the typesense_collections helper in backend/lib/tasks/typesense.rake:

def typesense_collections
  [
    Identity::Typesense::FacilitiesCollection,
    Identity::Typesense::PractitionersCollection,
    <Domain>::Typesense::<Entity>Collection,
  ]
end

5. Add controller actions

Add search endpoints following the pattern in Identity::SearchController. Include PostgreSQL fallback logic.

6. Add routes

get '<entities>/search', to: 'search#<entities>'

7. Add frontend types, API function, and hook

Follow the patterns in frontend/packages/shared-services/src/domains/identity/:

  • Add types to types.ts
  • Add API function to api.ts
  • Add query keys to keys.ts
  • Add React Query hook to hooks.ts

8. Create collection and reindex

bundle exec rake typesense:create_collections
bundle exec rake typesense:reindex_collection[<entity>]

Reference Data Endpoints

The search frontend often needs reference data for filter dropdowns. These are served by the identity domain:

EndpointDescription
GET /identity/reference/specialtiesList of medical specialties
GET /identity/reference/languagesList of supported languages

These endpoints return static or infrequently changing data and should be cached aggressively on the frontend (use queryConfigs.static from @kulcare/ui-kit).

Key Files

FilePurpose
backend/config/initializers/typesense.rbClient initialization and health check
backend/domains/shared/services/typesense/client.rbError-handling client wrapper
backend/domains/shared/services/typesense/base_collection.rbAbstract base for all collections
backend/domains/shared/models/typesense_sync_queue.rbSync queue buffer table model
backend/domains/shared/jobs/typesense_sync_job.rbBackground job for processing sync queue
backend/domains/shared/concerns/typesense_indexable.rbModel concern for auto-sync callbacks
backend/domains/identity/services/typesense/facilities_collection.rbFacilities collection definition
backend/domains/identity/services/typesense/practitioners_collection.rbPractitioners collection definition
backend/domains/identity/controllers/search_controller.rbPublic search API with fallback
backend/domains/identity/routes.rbRoute definitions (lines 276-278)
backend/lib/tasks/typesense.rakeRake tasks for management
frontend/packages/shared-services/src/domains/identity/api.tsFrontend API functions
frontend/packages/shared-services/src/domains/identity/hooks.tsReact Query hooks
frontend/packages/shared-services/src/domains/identity/types.tsTypeScript types
frontend/packages/shared-services/src/domains/identity/keys.tsQuery key factory

On this page