Reports & Analytics Features and Flows
Reports & Analytics Features and Flows Complete feature list, actor journeys, state flows, business rules, edge cases, and diagrams for Reports & Analytics.
Source Type Files or Docs What Was Extracted Technical design .omc/plans/Reports-analytics/master-plan.md, consumer-handoff.mdDesign rationale, the frozen contract API admin/dashboard/analytics-dashboard-admin.controller.ts, admin/reports/analytics-reports-admin.controller.ts, admin/exports/analytics-exports-admin.controller.ts, admin/sync/analytics-sync-admin.controller.tsRoutes, permissions, rate limits Backend shared/*.service.ts, shared/sync/*.ts, shared/reports/*.tsBehavior, ETL, rollups, exports Schema packages/mongodb/src/schemas/analytics/*.schema.ts18 collections, access patterns, retention Jobs packages/jobs/src/index.tsQueueName.ANALYTICS, AnalyticsJob, payloads
Field Value Module analyticsSubmodule dashboard, reports, exports, sync (all admin)Primary user value A shop answers "why is the business performing this way" and "what happened" without queries over the live order book Actors admin (reader), admin (operator), system (ETL crons, export workers)Main entry points 19 admin routes under /api/admin/analytics/; 5 cron jobs; 8 queue job kinds Main outputs Dashboard figures, report rows, export files, sync-state documents, Mongo facts and rollups Related docs Backend , API
Actor Can Do Cannot Do Auth Requirement Notes Admin (reader) Read every dashboard/analytics route, list and run permitted reports, request/read/download their own exports, read sync state Trigger syncs, backfills, reconciles; see reports outside their Reports*_READ grants; see others' exports Analytics_READ (+ the report's own for reports/exports)GET /reports returns only what they may runAdmin (operator) Everything a reader can, plus `POST /sync/refresh backfill reconcile` Delete or reset analytics data System (cron) Sync all streams (2 min), rollup (5 min), drain pending (5 min), reconcile orders (hourly), sweep exports (hourly) Move money or stock queue/internal Failures log and skip — analytics never affects orders System (worker) Project streams, rebuild rollups, build exports, sweep files — queue/internal One @Processor per queue
Capability Surface Actor Route/Trigger State Read State Written Linked API Section Dashboard summary Admin Reader GET /api/admin/analytics/dashboarddaily rollups — §8.1 KPIs with comparison Admin Reader GET /api/admin/analytics/kpisdaily rollups — §8.2 Daily trend Admin Reader GET /api/admin/analytics/trendsdaily rollups — §8.3 Conversion funnel Admin Reader GET /api/admin/analytics/funneldaily rollups — §8.4 Channel split Admin Reader GET /api/admin/analytics/channelsdaily rollups — §8.5 Payment-method split Admin Reader GET /api/admin/analytics/payment-methodsdaily rollups — §8.6 Top products Admin Reader GET /api/admin/analytics/top-productsorder line facts — §8.7 Geography Admin Reader GET /api/admin/analytics/geographyorder facts — §8.8 Review sentiment Admin Reader GET /api/admin/analytics/reviewsreview facts — §8.9 List permitted reports Admin Reader GET /api/admin/analytics/reportsreport registry — §8.10 Run a report Admin Reader GET /api/admin/analytics/reports/{reportId}fact collections / rollups — §8.11 Request an export Admin Reader POST /api/admin/analytics/exportsregistry, existing exports export record (Mongo) + queue job §8.12 List own exports Admin Reader GET /api/admin/analytics/exportsexport records — §8.13 Poll an export Admin Reader GET /api/admin/analytics/exports/{exportId}export record — §8.14 Download an export Admin Reader GET .../exports/{exportId}/downloadexport record, file — §8.15 Read sync state Admin Reader GET /api/admin/analytics/syncsync-state documents — §8.16 Force a sync Admin Operator POST /api/admin/analytics/sync/refreshsync-state queue job §8.17 Backfill a stream Admin Operator POST /api/admin/analytics/sync/backfillsync-state queue job §8.18 Reconcile a window Admin Operator POST /api/admin/analytics/sync/reconcilesync-state queue job §8.19 Incremental sync System Cron 2 min analytics.sync_allPostgreSQL, sync-state Mongo facts §9 Rollup rebuild System Cron 5 min analytics.rollupfacts daily rollups §9 Pending drain System Cron 5 min analytics.drain_pendingpending work order facts, rollups §9 Reconcile sweep System Cron hourly analytics.reconcilefacts facts (in place) §9 Export sweep System Cron hourly analytics.sweep_exportsexport records, files files (deleted), re-enqueued jobs §9
Nothing on this surface is customer-facing. There is no /mobile route and no storefront
impact. The flows below are the administrator's.
The dashboard is split into independent widget endpoints so one slow ranking widget never
blocks the whole page. The summary endpoint returns rollup-backed tiles (cheap); rankings
read the fact collections on their own routes.
The first sync has completed — otherwise every endpoint returns 503
ANALYTICS_NOT_YET_SYNCED rather than zeros.
Step Actor/System Action Result Source 1 Admin GET /dashboardTiles for today / week-to-date / month-to-date, stock and moderation counts analytics-dashboard.service.ts2 Admin GET /kpis?period=this_monthKPIs with like-for-like comparison same 3 Admin GET /trends, /funnel, /channels, …Independent widgets same
Branch Condition Behavior Error/Result Never synced No lastSuccessAt on a contributing stream 503 ANALYTICS_NOT_YET_SYNCED Retry shortly; never render zeros Stream behind lastBatchTruncatedbehind: true in metaFigures valid but incomplete — subtle indicator Yesterday had no revenue Zero denominator todayVsYesterdayPercent: nullThe first sale after a blank day is not "0% growth" No checkouts started Zero denominator checkoutConversionPercent: nullUndefined, not zero
One generic route runs any of the 19 registry reports. Filters travel as a single JSON
parameter; the report's own allowlist validates them by name — an unsupported key is refused,
never silently dropped.
The caller holds the report's Reports*_READ permission. GET /reports lists only what
they may run.
Step Actor/System Action Result Source 1 Admin GET /reportsDescriptors the caller may run, with columns and filters analytics-report.service.ts2 Admin GET /reports/{reportId}?period=…&filters={...}&sortBy=…Report rows + meta (period, paging, dataAsOf) same
Branch Condition Behavior Error/Result Unsupported filter key Key not on the report 400 ANALYTICS_FILTER_NOT_SUPPORTED, message names the key and lists supported Fix the filter Unsupported sort sortBy not in sortable400 ANALYTICS_SORT_NOT_SUPPORTED Use a listed key No such report / not permitted — 404 ANALYTICS_REPORT_NOT_FOUND Deliberately indistinguishable — ids cannot be enumerated Period invalid from after to / custom incomplete400 ANALYTICS_INVALID_PERIOD Fix dates Period too long Over 3 years 400 ANALYTICS_PERIOD_TOO_LONG Narrow, or export it Point-in-time report pointInTime: truemeta.period: nullDo not show a date range Search term prefix-anchored filters={"customerName":"ram"}Matches "Ram Bahadur", not "Sita Ram"
Asynchronous by design — a year of orders can aggregate for minutes. POST queues the build and
returns an exportId; poll until ready, then download.
Caller holds the report's permission; idempotencyKey is required.
Step Actor/System Action Result Source 1 Admin POST /exports with reportId, format, period, filters, idempotencyKeyexportId, status: queuedanalytics-export.service.ts2 Worker analytics.build_exportFile written; record ready analytics-export-builder.service.ts3 Admin Poll GET /exports/{exportId} Status ready, downloadUrl set same 4 Admin GET .../downloadFile streamed; permission re-checked controller
Branch Condition Behavior Error/Result Double-click Same idempotencyKey Returns the existing export No second aggregation Row cap Over 50,000 rows truncated: true on record AND warning line in the fileShow the flag Still building queued/processing400 ANALYTICS_EXPORT_NOT_READY Keep polling Failed build Worker error 400 ANALYTICS_EXPORT_FAILED, error carries the reason Request again Expired file Past 48h, swept 410 ANALYTICS_EXPORT_EXPIRED Request the export again Not yours Another admin's export 404 ANALYTICS_EXPORT_NOT_FOUND Same as nonexistent — no probing Permission revoked Role changed after queueing Download 404s Re-check is against the LIVE role
POST /sync/refresh (stream or all), /backfill (resumable, walks slices, seeds the
watermark on completion), /reconcile (re-projects a window ignoring the watermark, reports
driftCount). All three return as soon as the job is queued — poll GET /sync and watch
lastSuccessAt/driftCount move. 409 ANALYTICS_SYNC_ALREADY_RUNNING is an answer, not a
failure: the work is already happening.
Entity From Event/Action To Guard Condition Side Effects analytics_report_exportqueuedWorker claims processing— File build starts analytics_report_exportprocessingBuild completes readyFile written downloadUrl becomes availableanalytics_report_exportprocessingBuild throws failed— error carries the reasonanalytics_report_exportreadyHourly sweep expiredPast TTL (48h) File deleted from disk analytics_report_exportqueuedHourly sweep re-enqueued queued > 15 min, startedAt nullanalytics.build_export re-added (job-id deduped)
Entity From Event/Action To Guard Condition Side Effects analytics_sync_stateidleRun claims lock runningLock free or expired Watermark scan starts analytics_sync_staterunningRun completes idle— Watermark advanced (conditional); lastSuccessAt set; failures reset analytics_sync_staterunningRun throws failed— consecutiveFailures++; watermark NOT advancedanalytics_sync_statefailedNext run succeeds idle— Failures reset to 0 analytics_sync_staterunningBackfill slice backfillingMore slices remain backfillCursor records the resume pointanalytics_sync_statebackfillingFinal slice idledoneWatermark seeded where the backfill finished
Flow DB Writes Cache Effects Jobs Realtime Analytics Notifications Incremental sync Mongo facts per stream — analytics.sync_stream fan-out— — — Rollup rebuild daily rollups (replaceOne) — analytics.rollup— — — Pending drain order facts, rollup days — analytics.drain_pending— — — Export request export record (Mongo) — analytics.build_export— — — Export sweep export records, files — analytics.sweep_exports— — — Sync triggers — — sync_stream / sync_all / backfill / reconcile— — —
No PostgreSQL table is written by this module. Mongo is the only store it writes.
Scenario Trigger User/System Experience Recovery Source Never synced First sync not done 503 ANALYTICS_NOT_YET_SYNCED Retry shortly or trigger refresh analytics-freshness.service.tsStream failed Mongo/DB error consecutiveFailures++, figures ageNext run re-scans the same range (idempotent upserts) analytics-sync-state.service.tsStream behind Page budget hit behind: true on responsesNext tick continues; no data lost (watermark only advanced to processed) same Export enqueue lost Redis blip after record write Record stuck queued Hourly sweep re-enqueues analytics-export.service.tsCron enqueue failed Redis blip Logged and skipped Next tick (2 min); watermark did not move analytics-maintenance.scheduler.tsRollup half-projected Duplicate/out-of-order events Wrong day for one tick Full recompute next tick — no counter to corrupt analytics-rollup.service.tsRefund settled late Old order touched January net revenue stale analytics_pending_work queues the day; drain fixes itpending drain
Feature Minor Behavior Actor Trigger User/System Result Backend Side Effect Source Dashboard Independent widget endpoints Admin GET /dashboard + 8 moreNo widget blocks another Rollup-backed tiles vs fact-backed rankings dashboard service Freshness dataAsOf on every responseAdmin any read Age visible next to the figure Oldest contributing stream wins freshness service Comparison truncatedForFairnessAdmin GET /kpisSame-elapsed-days baseline Calendar-aligned windows period util Report registry Descriptor as data Admin GET /reportsPickers built from the response 19 descriptors, one code path report-registry.ts Filters Refused by name Admin GET /reports/{id}No confident wrong answers Allowlist + coercion per kind report service Search Prefix-anchored Admin report filter "Starts with" semantics ^escaped regex on lowercased indexsame Point-in-time meta.period: nullAdmin never-purchased / low-stock / top-customers No false date range pointInTime: true on descriptorregistry types Export idempotency Unique per admin Admin POST /exportsDouble-click safe Unique index + duplicate-key return export service Export truncation In-file warning Admin big export Cap visible truncated flag + warning rowbuilder service Export permission re-check Live role Admin download Revoked access dies at download assertStillPermittedexport service Sync state parent_driven streamOperator GET /syncorder_lines explained, not alarming ANALYTICS_PARENT_DRIVEN_STREAMSoperator service Backfill Resumable slices Operator POST /sync/backfillPress twice, continues backfillCursorsame Reconcile Never advances watermark Operator POST /sync/reconcileCan't mask a stuck stream completeReconcilesync-state service
The ETL pipeline:
Export pipeline:
Rule Business Reason Actor Impact Enforced In API Impact Backend Impact Tests Net revenue deducts settled refunds only Approved = liability, not deduction Two refund figures differ rollup aggregate approvedRefundedAmount vs settledRefundedAmountnet = gross - settledRefundedrollup spec Never-synced returns 503, not zeros Zero revenue ≠ no trading Dashboard refuses to lie freshness service ANALYTICS_NOT_YET_SYNCED— freshness spec Watermark overlap + conditional advance updated_at predates commitNo silent row loss sync-state service — 900s overlap; Math.max advance sync int spec Pending work for child-touched parents Settlement never touches the order row Refunds reach net revenue pending drain — enqueuePending from projectorssync int spec Rollups recomputed, never incremented Standalone Mongo, at-least-once Duplicates harmless rollup service — replaceOne(upsert)rollup spec Filters refused by name Dropped filter = wrong answer No confident lies report service 400 ANALYTICS_FILTER_NOT_SUPPORTED allowlist report spec Permission checked before the pipeline runs Cost control No paid-for refusals report service — assertPermitted firstreport spec Report 404 = no-such OR not-permitted Ids must not be enumerable Ops can't probe revenue reports report service one 404 shape — report spec Export permission re-checked at download Queueing ≠ snapshot access Revocation sticks export service + controller — assertStillPermittedexport spec Export cap writes into the file Silent truncation is worse Cap visible builder service truncated flagwarning row builder spec No reset or delete action Repair = re-project No destructive surface operator service no such route — — Day boundaries Asia/Kathmandu Local midnight is the business day Client must not UTC-bucket period util timeZone in metapinned at projection period spec
Product Decision User Benefit Engineering Benefit Alternative Tradeoff Risk MongoDB projection over PostgreSQL Cheap reads, no load on the live store Analytics never competes with orders Direct queries Up to 2 min staleness Mitigated by visible dataAsOf Rollups instead of live aggregation Instant dashboards Bounded reads $group over factsRecomputed on a 5-min tick Brief staleness, self-correcting Registry instead of per-report endpoints One code path No 60-copy drift Controller per report Swagger can't enumerate shapes Columns exposed via GET /reports Report-permission split Ops without revenue, finance without stock Least privilege One Reports_READ More grants to administer The point of the module Async capped exports No proxy timeouts Queued builds Synchronous stream 50k row cap; 48h TTL Both surfaced loudly No product-view funnel stage Honest funnel — Fabricated denominator Product→cart rate missing Stated in unavailableStages Prefix-only search Index-served Bounded cost Substring "starts with" semantics Stated in docs and handoff
Flow Edge Case Trigger Expected Behavior User/System Feedback Source Dashboard Never synced First request 503 Retry or trigger refresh freshness service Dashboard Zero yesterday Blank day Null percent No fake "0% growth" dashboard service Report Unknown filter key Typo 400 naming the key Fix the filter report service Report Unknown sort key Typo 400 listing valid keys Use a listed key same Report Zero totals Empty period $count = 0, paging total 0Empty state same Export Double submit Same idempotencyKey Existing record returned No duplicate work export service Export Stuck queued Enqueue lost Re-enqueued by sweep Status eventually moves same Export Past TTL Late download 410 expired Request again controller Sync Two operators trigger Both press refresh 409 ALREADY_RUNNING "It's already running" operator service Sync Backfill pressed twice Operator Resumes from cursor No restart same Stream Bulk update stamps one timestamp 1,500 rows, one now() Keyset tiebreaker holds the page boundary No lost rows sync-state service Reconcile Healthy stream Hourly driftCount 0 Diagnosis: incremental path fine same
Flow Reads Writes Cache Jobs/Events Response Fields Dashboard summary daily rollups, product facts (stock) — — — tiles + windows meta KPIs daily rollups (2 windows) — — — metrics + comparison meta Funnel daily rollups — — — stages, conversions, unavailableStages Report run fact collection / rollups — — — items, total, report meta Export request registry, existing exports export record — build_exportexportId, status, expiresAt Export download export record, file — — — file bytes Sync triggers sync-state — — 4 job kinds queued, correlationId
API doc: /docs/developer/analytics/api
Backend doc: /docs/developer/analytics/backend
TDD: not yet published
Reports & Analytics Backend Documentation
Backend architecture, data model, ETL pipeline, watermark mechanics, rollups, queues, and operational behavior for Reports & Analytics.
EMI Calculator Module Overview
The exact-rational EMI estimator — bank offerability, tenure vocabulary, rate history, and two customer read routes, all off by default.