Typesense Search Integration
Last Updated: 2026-05-31
Overview
Kulcare uses Typesense as the primary search engine for public entity discovery (facilities and practitioners). Typesense replaced the legacy Elasticsearch 6.2.4 setup for the following reasons:
- Simpler operations: Typesense is a single binary with no JVM dependency, compared to Elasticsearch's Java-based stack with significant memory overhead.
- Typo tolerance built in: Typesense handles fuzzy matching out of the box without custom analyzer configuration.
- Geo-search native: First-class
geopointfield type with radius filtering and distance-based sorting. - Faceted search: Built-in faceting support without aggregation queries.
- Lower resource footprint: Runs on a fraction of the memory that Elasticsearch requires.
The system includes an automatic PostgreSQL fallback so the application remains functional if Typesense is unavailable.
Note: earlier revisions of this guide described a
:typesense_search_enabledfeature flag. No such flag exists in the catalog (backend/lib/data/basic_feature_flags.json) and nothing in the code reads one — Typesense search is always on, with the fallback as its safety net.
Architecture
+-------------------+ +-------------------+ +------------------+
| Frontend | | Rails API | | Typesense |
| (React Query) | ---> | SearchController | ---> | Server |
| | | | | |
| useFacilitySearch| | search service | | facilities_env |
| usePractitioner | | Typesense / PG | | practitioners_ |
| Search | | fallback | | env |
+-------------------+ +--------+----------+ +------------------+
|
v (fallback)
+-------------------+
| PostgreSQL |
| ILIKE queries |
+-------------------+
+-------------------+ +-------------------+ +------------------+
| Model callbacks | ---> | TypesenseSyncQueue| ---> | TypesenseSyncJob|
| (after_commit) | | (buffer table) | | (Sidekiq) |
+-------------------+ +-------------------+ +------------------+Key components
| Component | Location | Purpose |
|---|---|---|
| Typesense initializer | backend/config/initializers/typesense.rb | Client setup, health check on boot |
| Shared client wrapper | backend/domains/shared/services/typesense/client.rb | Error-handling wrapper around TYPESENSE_CLIENT |
| Base collection | backend/domains/shared/services/typesense/base_collection.rb | Abstract base with search, index, reindex methods |
| Facilities collection | backend/domains/identity/services/typesense/facilities_collection.rb | Schema + serialization for facilities |
| Practitioners collection | backend/domains/identity/services/typesense/practitioners_collection.rb | Schema + serialization for practitioner roles |
| Search controller | backend/domains/identity/controllers/search_controller.rb | Public API endpoints with fallback |
| Sync queue model | backend/domains/shared/models/typesense_sync_queue.rb | Buffer table for async sync |
| Sync job | backend/domains/shared/jobs/typesense_sync_job.rb | Sidekiq job that processes the sync queue |
| Indexable concern | backend/domains/shared/concerns/typesense_indexable.rb | after_commit callbacks for models |
| Rake tasks | backend/lib/tasks/typesense.rake | CLI management commands |
| Frontend API | frontend/packages/shared-services/src/domains/identity/api.ts | searchFacilities, searchPractitioners |
| Frontend hooks | frontend/packages/shared-services/src/domains/identity/hooks.ts | useFacilitySearch, usePractitionerSearch |
| Frontend types | frontend/packages/shared-services/src/domains/identity/types.ts | FacilitySearchParams, PractitionerSearchResult, etc. |
| Frontend query keys | frontend/packages/shared-services/src/domains/identity/keys.ts | identityKeys.search.* |
Setup
Typesense server
Install and run Typesense locally for development:
# macOS (Homebrew)
brew install typesense/tap/typesense-server
typesense-server --data-dir=/tmp/typesense-data --api-key=dev_typesense_api_key
# Docker
docker run -p 8108:8108 \
-v /tmp/typesense-data:/data \
typesense/typesense:27.1 \
--data-dir /data \
--api-key=dev_typesense_api_keyEnvironment variables
Credentials are read from Rails credentials first, then fall back to environment variables:
| Variable | Default | Description |
|---|---|---|
TYPESENSE_URL | http://localhost:8108 | Typesense server URL |
TYPESENSE_API_KEY | dev_typesense_api_key | Admin API key |
For Rails credentials (preferred for staging/production):
rails credentials:edittypesense_url: "https://typesense.example.com"
typesense_api_key: "your_production_api_key"Feature flag
Typesense search is not feature-flagged; it is enabled wherever the Typesense connection is configured.
When disabled, all search requests fall through to the PostgreSQL fallback.
Bootstrap collections
After starting Typesense, create the collections and populate them:
bundle exec rake typesense:health
bundle exec rake typesense:create_collections
bundle exec rake typesense:reindexCollections
Facilities collection
Collection name: facilities_<Rails.env> (e.g., facilities_development)
Source: Identity::Facility with Identity::FacilityLocation
Schema:
| Field | Type | Facet | Notes |
|---|---|---|---|
id | string | no | Facility ID |
tenant_id | string | yes | |
unique_id | string | no | Optional |
name | string | no | Primary search field |
display_name | string | no | Falls back to name |
service_level | string | yes | e.g., hospital, clinic |
city | string | yes | From primary location |
state | string | yes | |
country_code | string | yes | |
postal_code | string | no | Not indexed |
location | geopoint | no | [lat, lng] for geo-search |
offers_telemedicine | bool | yes | |
is_virtual_only | bool | no | |
bed_count | int32 | no | |
active | bool | no | Always filtered to true |
specialties | string[] | yes | Array of specialty names |
phone | string | no | Not indexed |
popularity_score | int32 | no | Default sort field |
Search fields: name, display_name, city, specialties
Practitioners collection
Collection name: practitioners_<Rails.env> (e.g., practitioners_development)
Source: Identity::PractitionerRole (denormalized with Identity::Practitioner and Identity::Facility)
Each document represents one PractitionerRole. A practitioner at 3 clinics produces 3 documents.
Schema:
| Field | Type | Facet | Notes |
|---|---|---|---|
id | string | no | PractitionerRole ID |
practitioner_id | string | no | |
name | string | no | Full name |
display_name | string | no | |
gender | string | yes | |
qualifications | string[] | no | Qualification codes |
languages | string[] | yes | |
avatar_url | string | no | Not indexed |
bio | string | no | Not indexed |
tenant_id | string | yes | |
facility_id | string | yes | |
facility_name | string | no | |
role_type | string | yes | |
practitioner_type | string | yes | |
status | string | yes | |
specialty | string | yes | Specialty name |
consultation_fee | float | no | Filterable by range |
is_primary_facility | bool | no | |
search_keywords | string[] | no | Derived keywords |
city | string | yes | |
state | string | yes | |
country_code | string | yes | |
location | geopoint | no | From facility's primary location |
active | bool | no | Derived from status == 'active' |
popularity_score | int32 | no | Default sort field |
Search fields: name, display_name, specialty, facility_name, city, qualifications, search_keywords
API Endpoints
Both endpoints are public (no authentication required) and defined in backend/domains/identity/routes.rb:
get 'facilities/search', to: 'search#facilities'
get 'practitioners/search', to: 'search#practitioners'GET /identity/facilities/search
Search for facilities by name, location, or filters.
Parameters:
| Parameter | Type | Description |
|---|---|---|
q | string | Search query (default: * for all) |
lat | float | Latitude for geo-search |
lng | float | Longitude for geo-search |
radius | integer | Radius in km (default: 10) |
city | string | Filter by city |
service_level | string | Filter by service level |
telemedicine | string | 'true' to filter telemedicine facilities |
page | integer | Page number (default: 1) |
per_page | integer | Results per page (default: 20, max: 250) |
Example request:
GET /identity/facilities/search?q=downtown&city=Bangalore&page=1&per_page=10Example response:
{
"data": [
{
"id": "abc-123",
"name": "Downtown Medical Center",
"display_name": "Downtown Medical Center",
"service_level": "hospital",
"city": "Bangalore",
"state": "Karnataka",
"active": true,
"specialties": ["Cardiology", "Orthopedics"],
"offers_telemedicine": true
}
],
"meta": {
"source": "typesense",
"found": 42,
"page": 1,
"search_time_ms": 3
}
}GET /identity/practitioners/search
Search for practitioners by name, specialty, location, or filters.
Parameters:
| Parameter | Type | Description |
|---|---|---|
q | string | Search query (default: * for all) |
specialty | string | Filter by specialty name |
city | string | Filter by city |
gender | string | Filter by gender |
max_fee | float | Maximum consultation fee |
role_type | string | Filter by role type |
lat | float | Latitude for geo-search |
lng | float | Longitude for geo-search |
radius | integer | Radius in km (default: 10) |
facets | string | 'true' to include facet counts |
page | integer | Page number (default: 1) |
per_page | integer | Results per page (default: 20, max: 250) |
Example request:
GET /identity/practitioners/search?q=cardio&specialty=Cardiology&facets=trueExample response:
{
"data": [
{
"id": "role-456",
"practitioner_id": "doc-789",
"name": "Dr. Priya Sharma",
"display_name": "Dr. Priya Sharma",
"specialty": "Cardiology",
"facility_name": "Heart Care Hospital",
"city": "Mumbai",
"consultation_fee": 800.0,
"gender": "female",
"qualifications": ["MBBS", "MD"],
"languages": ["English", "Hindi"],
"active": true
}
],
"meta": {
"source": "typesense",
"found": 15,
"page": 1,
"search_time_ms": 2
}
}When facets=true, the response includes facet counts for specialty, city, languages, gender, and role_type.
Geo-search
Both endpoints support geo-search. When lat and lng are provided, results are:
- Filtered to the specified
radius(default: 10 km) - Sorted by distance from the provided coordinates (nearest first)
GET /identity/facilities/search?lat=12.9716&lng=77.5946&radius=5Indexing and Sync
How data gets into Typesense
There are two paths for data to reach Typesense:
1. Real-time sync via model callbacks
Models that include the TypesenseIndexable concern automatically enqueue sync entries on create, update, and destroy:
# backend/domains/shared/concerns/typesense_indexable.rb
module Shared::Concerns::TypesenseIndexable
extend ActiveSupport::Concern
included do
after_commit :enqueue_typesense_upsert, on: %i[create update]
after_commit :enqueue_typesense_delete, on: :destroy
end
endTo add this to a model:
class Identity::Facility < Identity::BaseRecord
include Shared::Concerns::TypesenseIndexable
end2. Sync queue and background job
Model callbacks write to the typesense_sync_queue buffer table instead of calling Typesense directly. This ensures:
- Database transactions complete before indexing
- Batch processing for efficiency
- Retry logic for transient failures
- No impact on request latency
The Shared::TypesenseSyncJob is enqueued every 1 minute by the system crontab (config/schedule.rb, regenerated on each deploy via whenever/capistrano). It:
- Fetches up to 500 pending entries from the queue
- Deduplicates by
(record_type, record_id)keeping the latest entry - Groups entries by record type
- Batch-upserts or deletes documents in Typesense
- Marks entries as processed (
processed_at = NOW()) or failed
History — see #1592: an earlier version of this doc claimed "every 5 seconds via sidekiq-scheduler". That was aspirational — the schedule never shipped, so the queue sat unread for weeks and search drifted from Postgres. The pipeline now runs from system cron (
whenever), wired into Capistrano so deploys regenerate the crontab.
Special handling for Practitioners: When a Practitioner record changes, the job re-indexes all of that practitioner's PractitionerRole documents (since practitioner data is denormalized into role documents).
Cascade on Facility / FacilityLocation changes: practitioners_<env> denormalizes facility name, city, address, lat/lng, and phone onto every role doc. To prevent stale search data when a facility is renamed or moved:
Identity::Facilityafter_commiton update fans out to everyPractitionerRoleat the facility, but only when one ofDENORMALIZED_TO_PRACTITIONERS(name,display_name,service_level,active) changes. Unrelated column touches don't trigger the fan-out.Identity::FacilityLocationafter_commitenqueues the parent facility AND every role at it, gated onDENORMALIZED_TO_FACILITY(address_line_1,city,latitude,longitude,is_primary,active, etc.).- Both fan-outs use
Shared::TypesenseSyncQueue.bulk_enqueue— oneINSERT … VALUESround-trip instead of Ncreate!calls, so the after_commit doesn't stall the commit thread on large facilities. - Both callbacks
rescue StandardErrorand log toRails.logger.error. The queue is best-effort: a transient DB blip in the enqueue path must not 500 the user's update. The nightly reindex (below) is the safety net.
Retry and dead letter: Entries are retried up to 5 times (MAX_RETRIES). After that, they are considered dead-letter entries and can be inspected via Shared::TypesenseSyncQueue.dead_letter.
Nightly safety-net reindex: rake typesense:reindex runs at 3:00 AM (config/schedule.rb). It rebuilds both collections from scratch and catches anything the per-row enqueue missed — dead-lettered rows, transient DB failures, or new denormalized fields not yet wired into a cascade.
Queue pruning: Shared::TypesenseSyncQueue marks processed rows with processed_at but never deletes them. Without pruning the table would grow indefinitely (the Facility cascade alone can enqueue thousands of rows per day). rake typesense:prune_processed runs nightly at 3:30 AM and deletes rows where processed_at < 30.days.ago. The retention window is generous enough to support post-hoc missed-sync investigations; older rows are dead weight.
3. Full reindex via rake task
For initial population or recovery, use the rake task:
bundle exec rake typesense:reindexThis iterates over the source_scope in batches of 1000, serializes each record, and imports them into Typesense using the batch import API.
Rake Tasks
All rake tasks are defined in backend/lib/tasks/typesense.rake:
| Task | Description |
|---|---|
typesense:health | Check Typesense cluster health |
typesense:create_collections | Create all Typesense collections (idempotent) |
typesense:drop_collections | Drop all collections (destructive) |
typesense:recreate_collections | Drop and recreate all collections (destructive) |
typesense:reindex | Full reindex of all collections from the database |
typesense:reindex_collection[name] | Reindex a specific collection (e.g., facilities or practitioners) |
typesense:queue_status | Show pending, dead-letter, and total sync queue counts |
typesense:prune_processed[days] | Delete processed sync queue rows older than N days (default 30). Scheduled nightly at 3:30 AM. |
Examples:
# Check health
bundle exec rake typesense:health
# Create collections (safe to run multiple times)
bundle exec rake typesense:create_collections
# Full reindex
bundle exec rake typesense:reindex
# Reindex only practitioners
bundle exec rake typesense:reindex_collection[practitioners]
# Check sync queue status
bundle exec rake typesense:queue_status
# Output:
# Typesense Sync Queue:
# Pending: 12
# Dead letter: 0
# Total: 1543PostgreSQL Fallback
When Typesense is unavailable (server down, timeout, or feature flag disabled), the search controller falls through to PostgreSQL queries automatically.
How it works
- If disabled, it runs the PostgreSQL fallback directly.
- If enabled, it calls Typesense. If Typesense raises
ServerError,TimeoutError, orHTTPError, the controller rescues and runs the PostgreSQL fallback. - The response includes
meta.fallback: truewhen the fallback was used, andmeta.source: 'postgres'.
def facilities
results = if typesense_enabled?
search_facilities_typesense
else
search_facilities_postgres
end
render json: { data: results[:hits], meta: results[:meta] }
rescue ::Typesense::Error::ServerError,
::Typesense::Error::TimeoutError,
::Typesense::Error::HTTPError => e
Rails.logger.warn("Typesense unavailable, PG fallback: #{e.message}")
results = search_facilities_postgres
render json: { data: results[:hits], meta: results[:meta].merge(fallback: true) }
endFallback limitations
The PostgreSQL fallback uses ILIKE queries, which means:
- No typo tolerance
- No geo-search (lat/lng parameters are ignored)
- No faceting
- Slower on large datasets
- Results capped at 100 per page (vs 250 for Typesense)
The fallback ensures the application remains functional for basic text search while Typesense is being restored.
Frontend Integration
API functions
The frontend API layer is in frontend/packages/shared-services/src/domains/identity/api.ts:
import { apiClient } from '../../client';
import type {
FacilitySearchParams,
FacilitySearchResult,
PractitionerSearchParams,
PractitionerSearchResult,
SearchResponse,
} from './types';
export async function searchFacilities(
params?: FacilitySearchParams
): Promise<SearchResponse<FacilitySearchResult>> {
const response = await apiClient.get<SearchResponse<FacilitySearchResult>>(
'/identity/facilities/search',
{ params }
);
return response.data;
}
export async function searchPractitioners(
params?: PractitionerSearchParams
): Promise<SearchResponse<PractitionerSearchResult>> {
const response = await apiClient.get<SearchResponse<PractitionerSearchResult>>(
'/identity/practitioners/search',
{ params }
);
return response.data;
}React Query hooks
The hooks are in frontend/packages/shared-services/src/domains/identity/hooks.ts:
import { useQuery } from '@tanstack/react-query';
import { identityKeys } from './keys';
import { searchFacilities, searchPractitioners } from './api';
import type { FacilitySearchParams, PractitionerSearchParams } from './types';
export function useFacilitySearch(params?: FacilitySearchParams) {
return useQuery({
queryKey: identityKeys.search.facilities(params),
queryFn: () => searchFacilities(params),
enabled: !!(params?.q && params.q.length >= 2) || !!(params?.lat && params?.lng),
staleTime: 60_000,
});
}
export function usePractitionerSearch(params?: PractitionerSearchParams) {
return useQuery({
queryKey: identityKeys.search.practitioners(params),
queryFn: () => searchPractitioners(params),
enabled:
!!(params?.q && params.q.length >= 2) ||
!!(params?.lat && params?.lng) ||
!!params?.specialty,
staleTime: 60_000,
});
}Query key structure
Query keys use a hierarchical factory pattern for proper cache invalidation:
identityKeys.search.all() // ['identity', 'search']
identityKeys.search.facilities(params) // ['identity', 'search', 'facilities', params]
identityKeys.search.practitioners(params) // ['identity', 'search', 'practitioners', params]Usage in components
import { useFacilitySearch, usePractitionerSearch } from '@kulcare/shared-services';
function DoctorFinder() {
const [query, setQuery] = useState('');
const [specialty, setSpecialty] = useState('');
const { data, isLoading } = usePractitionerSearch({
q: query,
specialty,
per_page: 20,
});
return (
<div>
<input value={query} onChange={(e) => setQuery(e.target.value)} />
{isLoading && <Spinner />}
{data?.data.map((practitioner) => (
<DoctorCard key={practitioner.id} doctor={practitioner} />
))}
<p>Found {data?.meta.found} results in {data?.meta.search_time_ms}ms</p>
{data?.meta.fallback && <p>Using fallback search (limited functionality)</p>}
</div>
);
}TypeScript types
All types are defined in frontend/packages/shared-services/src/domains/identity/types.ts:
interface FacilitySearchParams {
q?: string;
lat?: number;
lng?: number;
radius?: number;
city?: string;
service_level?: string;
telemedicine?: string;
page?: number;
per_page?: number;
}
interface PractitionerSearchParams {
q?: string;
specialty?: string;
city?: string;
gender?: string;
max_fee?: number;
role_type?: string;
lat?: number;
lng?: number;
radius?: number;
facets?: string;
page?: number;
per_page?: number;
}
interface SearchResponse<T> {
data: T[];
meta: {
source: 'typesense' | 'postgres';
found: number;
page: number;
search_time_ms?: number;
fallback?: boolean;
};
}Adding a New Collection
Follow these steps to index a new entity type in Typesense:
1. Create the collection class
Create a new file under the appropriate domain, inheriting from Shared::Typesense::BaseCollection:
# backend/domains/<domain>/services/typesense/<entity>_collection.rb
module <Domain>
module Typesense
class <Entity>Collection < Shared::Typesense::BaseCollection
class << self
def collection_name
"<entity>_#{Rails.env}"
end
def schema
{
'name' => collection_name,
'fields' => [
{ 'name' => 'id', 'type' => 'string' },
{ 'name' => 'name', 'type' => 'string' },
# Add your fields here. See Typesense docs for field types:
# string, int32, int64, float, bool, string[], geopoint
# Use 'facet' => true for fields you want to filter on.
# Use 'optional' => true for nullable fields.
# Use 'index' => false for fields you only want to store, not search.
{ 'name' => 'active', 'type' => 'bool' },
{ 'name' => 'popularity_score', 'type' => 'int32' }
],
'default_sorting_field' => 'popularity_score'
}
end
def serialize(record)
{
'id' => record.id.to_s,
'name' => record.name,
'active' => record.active?,
'popularity_score' => 0
}.compact
end
def source_scope
<Domain>::<Entity>.where(active: true)
end
protected
def build_search_params(params)
search = {
'q' => params[:q].presence || '*',
'query_by' => 'name',
'per_page' => [(params[:per_page] || 20).to_i, 250].min,
'page' => (params[:page] || 1).to_i
}
filters = []
filters << "active:=true"
# Add your filters here
search['filter_by'] = filters.join(' && ') if filters.any?
search
end
def default_query_by
'name'
end
end
end
end
end2. Add the model concern
Include TypesenseIndexable in the model so changes are synced automatically:
class <Domain>::<Entity> < <Domain>::BaseRecord
include Shared::Concerns::TypesenseIndexable
end3. Register in the sync job
Add the mapping to Shared::TypesenseSyncJob::COLLECTION_MAP:
COLLECTION_MAP = {
'Identity::Facility' => 'Identity::Typesense::FacilitiesCollection',
'Identity::PractitionerRole' => 'Identity::Typesense::PractitionersCollection',
'<Domain>::<Entity>' => '<Domain>::Typesense::<Entity>Collection',
}.freeze4. Register in rake tasks
Add your collection to the typesense_collections helper in backend/lib/tasks/typesense.rake:
def typesense_collections
[
Identity::Typesense::FacilitiesCollection,
Identity::Typesense::PractitionersCollection,
<Domain>::Typesense::<Entity>Collection,
]
end5. Add controller actions
Add search endpoints following the pattern in Identity::SearchController. Include PostgreSQL fallback logic.
6. Add routes
get '<entities>/search', to: 'search#<entities>'7. Add frontend types, API function, and hook
Follow the patterns in frontend/packages/shared-services/src/domains/identity/:
- Add types to
types.ts - Add API function to
api.ts - Add query keys to
keys.ts - Add React Query hook to
hooks.ts
8. Create collection and reindex
bundle exec rake typesense:create_collections
bundle exec rake typesense:reindex_collection[<entity>]Reference Data Endpoints
The search frontend often needs reference data for filter dropdowns. These are served by the identity domain:
| Endpoint | Description |
|---|---|
GET /identity/reference/specialties | List of medical specialties |
GET /identity/reference/languages | List of supported languages |
These endpoints return static or infrequently changing data and should be cached aggressively on the frontend (use queryConfigs.static from @kulcare/ui-kit).
Key Files
| File | Purpose |
|---|---|
backend/config/initializers/typesense.rb | Client initialization and health check |
backend/domains/shared/services/typesense/client.rb | Error-handling client wrapper |
backend/domains/shared/services/typesense/base_collection.rb | Abstract base for all collections |
backend/domains/shared/models/typesense_sync_queue.rb | Sync queue buffer table model |
backend/domains/shared/jobs/typesense_sync_job.rb | Background job for processing sync queue |
backend/domains/shared/concerns/typesense_indexable.rb | Model concern for auto-sync callbacks |
backend/domains/identity/services/typesense/facilities_collection.rb | Facilities collection definition |
backend/domains/identity/services/typesense/practitioners_collection.rb | Practitioners collection definition |
backend/domains/identity/controllers/search_controller.rb | Public search API with fallback |
backend/domains/identity/routes.rb | Route definitions (lines 276-278) |
backend/lib/tasks/typesense.rake | Rake tasks for management |
frontend/packages/shared-services/src/domains/identity/api.ts | Frontend API functions |
frontend/packages/shared-services/src/domains/identity/hooks.ts | React Query hooks |
frontend/packages/shared-services/src/domains/identity/types.ts | TypeScript types |
frontend/packages/shared-services/src/domains/identity/keys.ts | Query key factory |