Date: 2026-10-29 (rewritten after a full assessment)
Repos: FLW-API, Identity-API, TM-API, MMU-API, Common-API (all on release-3.12.0), AMRIT-DB (main; no release-3.12.0 branch), STOP-TB-App
Related:STOP-TB-App PR #321 (sync reliability fixes); STOP-TB sync gap analysis and Aug 2026 field log analysis (available on request)
Summary
How it works today
Requirements
Problem 1: "What changed since X?"
Problem 2: All data on the tablet
Problem 3: Station hand-offs and overwrites
Target design
Order of work
Testing and measurement
Rollout
Decisions needed
Appendix: evidence
Three problems, in the order to solve them:
"What changed since X?" cannot be answered today.
Every pull downloads everything. With 4,000 beneficiaries, a pull is about 400 page requests plus every TB record. In camp mode pulls run back to back while the app is open.
Timestamps alone can't fix this:
several write paths freeze, backdate or null LastModDate;
rows that reach the hub from central keep central's older timestamp;
deletes aren't recorded;
the columns are second-precision with no indexes.
Fix: a change log on each camp laptop (hub), filled by database triggers. Tablets read changes with an opaque token from that laptop, through one endpoint. Central is not involved.
Tablets need all the data, not one village. After login, users take tablets to villages and work offline.
Fix: the scope becomes all villages assigned to the user. Endpoints accept multiple villages, the village picker becomes a view filter, and photos are downloaded separately and lazily.
Stations overwrite each other. Registrar → screening → nurse → lab technician → counsellor each work on their own tablet.
Only edited records are pushed, but each is sent with all its fields, not just the ones that changed. The server keeps the last write with no version check.
A stale tablet can undo a registrar's correction, roll back a weight, or erase a lab result.
Fix, in order of value:
field ownership by role;
pushes send only changed fields;
a version check that uses the change log;
a "fetch this beneficiary now" call for when a person arrives at a station.
Problem 1 is the foundation: problems 2 and 3 both depend on it.
Diagram 1: Current data flow (Mermaid source below; image attachment d1.png)
flowchart LR
subgraph Village["Village (offline)"]
T1[Tablet<br/>Room DB]
end
subgraph Camp["Camp (local Wi-Fi)"]
T2[Registrar tablet]
T3[Nurse tablet]
T4[Lab / counsellor tablets]
HUB[(Camp hub laptop<br/>FLW / Identity / TM / MMU APIs<br/>own MySQL, VanID = N)]
end
C[(Central server<br/>stoptb.piramalswasthya.org)]
T2 -- "push edited records (all fields of each)<br/>pull EVERYTHING ~every 10s" --> HUB
T3 -- "push edited records (all fields of each)<br/>pull EVERYTHING" --> HUB
T4 -- "push / pull" --> HUB
HUB -- "upsync (manual / 30 min, off by default)<br/>keeps hub LastModDate" --> C
C -- "downsync (manual, VanID = N only)<br/>keeps central LastModDate" --> HUB
T1 -. "syncs when back at a camp" .-> HUB |
What a pull does now (STOP-TB app):
Beneficiaries:flw-api/beneficiary/getBeneficiaryData, village-scoped, 10 per page with OFFSET, ordered by registration date.
New registrations during a pull shift the pages, so records get skipped or repeated.
On the server, each record costs about 8 queries plus one HTTP call to FHIR.
9 TB and form datasets, each the full village list in one response with no paging: screening, general examination, general OPD, diagnostics, suspected, confirmed, counselling-completed, and 4 contact-tracing/TPT form types.
Confirmed has no village filter; it returns every confirmed case for the user.
Suspected re-runs incentive logic on every pull.
Diagnostic order status: 6 calls, plus one result call per completed order.
Master data: 3 tables (visit categories, chief complaints, drugs) deleted and rewritten on every pull.
Hub ↔ central (MMU-API data sync):
Upsync (hub → central): sends rows with Processed != 'P'.
Central inserts or updates each row by (VanSerialNo, VanID); the hub always wins.
Then the hub marks the row processed, which moves its LastModDate to the hub's current time.
Downsync (central → hub): started manually, and only for rows with VanID = N.
Updates keep central's LastModDate; inserts end up with the hub's current time by accident.
Downsync also creates worklist rows, without which tablets don't see the beneficiary at all.
IDs:
BenRegId / BeneficiaryID are the same on hub and central.
Every other primary key differs: central keeps the hub's key as VanSerialNo, the hub keeps central's as CentralID.
Tablets talk only to a hub (since 1f19f944). Each hub has its own clock and its own data.
# | Requirement |
|---|---|
R1 | A pull with nothing new costs about 1 small request and moves almost no data. |
R2 | A pull with changes downloads only the changed records: creates, updates and deletes. |
R3 | Correct no matter which code path wrote the row: JPA, native SQL, hub↔central sync, or manual fixes. |
R4 | The tablet holds all data for the user's scope and works fully offline. |
R5 | A record saved at one station appears at the next station within about 30 seconds. On demand when the person walks up, it appears within seconds. |
R6 | One station never silently overwrites another station's newer data. |
R7 | Backward compatible: old apps keep working, and a new app against an old server falls back to a full pull. |
R8 | No Room migrations in the app for sync state. Sync positions go in encrypted shared preferences. |
LastModDate > X fails todayHole | Where | Effect |
|---|---|---|
Writable mappings freeze or replace the timestamp |
| Stored as the tablet's clock, NULL, or unchanged, so the change is never selected |
| TM | Moved back to midnight |
Central → hub downsync keeps central's timestamp | MMU | A row updated on central reaches the hub with an old time, so the hub's cursor has passed it |
Hub → central upsync keeps the hub's timestamp |
| Same problem in the other direction |
No |
| Bulk or native updates aren't seen |
Hard deletes, no | TM visit and complaint deletes; | Deletes can't be synced |
Child changes don't touch the parent | complaints, drugs, vitals, answers | A changed parent can't be detected from its children |
Second precision; commits land after their timestamp; JVM and DB time zones not pinned | Most columns | A change can fall behind a cursor that has already moved on |
No indexes on the change columns | Most tables | Slow scans on large tables |
You could patch each of these one by one (§4.4 option B), but holes would keep coming back as new code is written.
Each camp laptop (hub) gets a change log in each schema it uses, db_iemr and db_identity. Production central gets neither the table nor the triggers, and the log is never synced to central: it isn't in the upsync table list (m_synctabledetail) and must not be added. Tablets only talk to hubs, so the log is only needed there. The one exception is UAT central (uatstoptb.piramalswasthya.org), which gets its own log so tablets can be tested against it directly (§7 explains the gate).
CREATE TABLE t_sync_change_log (
seq BIGINT AUTO_INCREMENT PRIMARY KEY, -- order of changes, local to this server
table_name VARCHAR(64) NOT NULL,
row_pk BIGINT NOT NULL,
op ENUM('I','U','D') NOT NULL,
ben_reg_id BIGINT NULL, -- for scope filtering and parent changes
village_id INT NULL, -- denormalised scope key (see §5)
changed_at DATETIME(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
KEY (village_id, seq), KEY (ben_reg_id, seq), KEY (table_name, row_pk),
KEY (changed_at) -- for the nightly clean-up of old lines (§4.6)
); |
Triggers:AFTER INSERT, AFTER UPDATE and AFTER DELETE on each STOP-TB table write a row. Child tables (complaints, drugs, answers, vitals) also write a row for their parent and carry ben_reg_id.
Why triggers: they fire for every write path. That includes JPA, native SQL, rows arriving on the hub by downsync from central, Flyway backfills and manual fixes. They fire for hard deletes too. The tablet never relies on LastModDate.
The token: what the tablet keeps is {serverId, seq}, where serverId is the hub's VanID.
If the tablet presents a token from a different server, that server starts it again from a fresh snapshot.
This handles a tablet that moves between hubs, and it never compares clocks between machines.
Late commits:AUTO_INCREMENT values are taken before commit, so a transaction holding seq 41 can commit after one holding 42. To be safe:
the server returns only up to a safe point, the highest seq older than a short lag (for example 5 seconds);
the tablet re-reads from slightly before its token;
duplicates are harmless because the app applies the server copy only when it's newer.
Retention: keep 30 days of log. A token older than that gets a fresh snapshot.
Worked example: how the token picks up an edit to an older record
The number the tablet remembers is the line number in the log (seq), not a record ID. Every change appends a new line at the end, whichever record it touches:
Log (seq = line number, always increasing) ... #1000 beneficiary 900 updated ← this tablet has read up to here #1001 tb_screening 455 inserted #1002 beneficiary 900 updated ← another tablet edits record 900 again #1003 vitals 77 inserted |
The tablet's token says "read up to #1000".
Another tablet edits record 900. That writes a new line, #1002, at the end of the log.
On its next sync the tablet asks for everything after #1000. It gets #1001 to #1003, sees that record 900 changed, and the server sends the current version of record 900.
So an edit to an old record always reaches every tablet: the edit creates a new line after each tablet's bookmark.
Several edits to one record: if record 900 changes five times between two syncs, the server sends it once, in its latest state.
Deletes: a delete also writes a line (beneficiary 900 deleted), so tablets learn about deletes the same way.
Why this beats timestamps: the database assigns the line number at the moment of the change, so unlike LastModDate it can't be frozen, backdated or copied from another server.
Diagram 2: Change log filled by triggers (Mermaid source below; image attachment d2.png)
flowchart TB
subgraph Writers["Anything that writes"]
A[FLW/Identity/TM/MMU APIs<br/>JPA + native SQL]
B[Downsync from central /<br/>upsync bookkeeping]
M[Manual fixes / Flyway backfills]
end
subgraph DB["MySQL on the camp laptop (hub)"]
TBL[(STOP-TB tables<br/>beneficiary, TB, OPD, forms, orders)]
TRG{{AFTER INSERT / UPDATE / DELETE<br/>triggers}}
LOG[(t_sync_change_log<br/>seq, table, pk, op, ben_reg_id, village_id)]
end
FEED[FLW-API<br/>GET /stoptb/sync/changes?token=]
TAB[Tablet]
A --> TBL
B --> TBL
M --> TBL
TBL --> TRG --> LOG
LOG --> FEED -->|"changes + nextToken"| TAB |
POST flw-api/stoptb/sync/changes, served by FLW-API on every camp laptop, and on UAT central for testing. Production central doesn't serve it; tablets only talk to hubs.
Request: { token: "<opaque or null>", scope: { villageIds: [...] }, limit: 500 }
Response: { serverId, items: [ { type: "beneficiary" | "household" | "tbScreening" | "generalExam" | "generalOpdVisit"
| "diagnostics" | "suspected" | "confirmed" | "diagnosticOrder" | "formResponse",
op: "upsert" | "delete", key: {...}, version: <seq>, data: {...} } ],
nextToken, hasMore, snapshot: true|false } |
Token null: snapshot.
The server notes the current safe point, then pages through everything in scope ordered by (type, benRegId, pk) using keyset paging, not OFFSET.
When the last page is sent, nextToken holds the noted safe point. Changes made during the snapshot are then picked up as ordinary changes.
Token present: changes since that point.
The server reads log rows in scope with seq > token, up to limit, removes duplicates by (table, pk), and loads the current rows for them.
A deleted row comes back as op: delete.
Payload builders: these reuse the mappers that today's getAll endpoints use (StopTBServiceImpl, TBSuspectedServiceImpl, BeneficiaryServiceImpl's record builder), but batched, so there are no per-record round trips.
The FHIR health-ID lookup is batched or moved out of this path.
Photos are not included; see §5.3.
version = the latest seq for that row. Push uses it for the version check (§6).
Old endpoints stay as they are (R7).
Diagram 3: Changes feed: snapshot, then changes (Mermaid source below; image attachment d3.png)
sequenceDiagram
participant T as Tablet
participant H as Camp hub (FLW-API)
participant L as Change log
T->>H: changes(token=null, scope)
H->>L: note current safe point S
loop snapshot pages (keyset)
H-->>T: items (snapshot=true), nextToken=page cursor
end
H-->>T: last page, nextToken={hubId, S}
Note over T: saves token in encrypted prefs
loop every tick (camp) / on reconnect
T->>H: changes(token={hubId, S})
H->>L: seq > S AND scope, ≤ safe point
H-->>T: only the changed rows (often 0), nextToken={hubId, S'}
end
T->>H: changes(token={otherHub, X})
H-->>T: token is from another server → snapshot again |
Option | Correct? | Cost | Verdict |
|---|---|---|---|
A. | No (§4.1) | None | Rejected |
B. Fix every write path and add | Mostly; keeps breaking as new code is written; hub↔central timestamps still need a fix in the sync code | Many small changes in 5 repos | Fallback if triggers are refused |
C. Trigger-filled change log plus one changes endpoint | Yes, for every write path including deletes and hub↔central sync | One migration per schema, one endpoint, payload builders | Recommended |
Things to check for option C: how much the triggers slow writes on i_ben_flow_outreach and the identity tables (measure on a hub), that a trigger never causes an insert to fail (keep the body minimal), and that the log is never replicated: it stays on each camp laptop.
Where | Change | Needed for delta pull? |
|---|---|---|
AMRIT-DB (camp laptops only) |
| Yes |
FLW-API: new endpoint | Reads the log and returns the changed records | Yes |
FLW-API: record builders | The code that turns a database row into the JSON the app receives already exists inside today's | Yes (refactor, not new logic) |
Common/FLW user detail | Fix the | Yes, for full offline data |
Existing pull endpoints ( | No change. Old app versions keep using them. | No |
Existing save endpoints | Field ownership and version check (§6) | No; later, for station conflicts |
App | New sync engine: one endpoint, saves the token, applies changes and deletes | Yes |
The tablet sends token = {hub 415, N = 1000}, scope villages [12, 14].
Step 1: find what changed. The log query:
SELECT table_name, row_pk, MAX(seq) AS last_seq, MAX(op = 'D') AS deleted FROM t_sync_change_log WHERE village_id IN (12, 14) AND seq > 1000 AND seq <= :safePoint GROUP BY table_name, row_pk ORDER BY last_seq LIMIT 500; |
This returns a short list, e.g. "beneficiary 900, tb_screening 455, vitals 77". A record changed five times appears once.
Step 2: load the current records, one query per table:SELECT … FROM tb_screening WHERE id IN (455, …), the same for beneficiaries, vitals and so on. The existing builders turn them into JSON; deleted ones go out as op: delete.
Step 3: return the records plus nextToken = {hub 415, last_seq of the last item}. If there were more than 500, hasMore = true and the tablet calls again with the new token.
Every query is either indexed or a lookup by ID, so the work is proportional to what changed, not to the size of the village.
This is the snapshot path. It's the same for re-login, a new tablet, a reinstall, a different hub, or a token older than the log's 30-day retention.
On logout: the app wipes its database and preferences (clearAllTables, deleteForLogout), so the saved token is gone.
On the next login: the tablet sends token = null, and the server:
notes the log's current line number, e.g. S = 5,230;
sends everything in the user's scope, read from the real tables rather than the log, page by page with keyset paging, so no records are skipped while new ones are being added;
on the last page, returns nextToken = {hub 415, S = 5230}.
Edits made by other tablets during the download have log lines after 5,230, so the very next sync picks them up and nothing is missed.
After that, every sync is a delta.
So re-login costs one full, properly paged download, then only changes.
Caution on logout: wiping the database at logout deletes records that haven't synced. The logout dialog warns with the count (fixed in PR #321), but it still lets the user go ahead. With tablets working offline in villages, consider blocking logout until everything has synced, or at least requiring a second confirmation (decision 8 in §11).
The change log is one table, but each row is small: a line number, table name, row ID, change type, beneficiary ID, village ID and time. That's roughly 100–150 bytes per line including indexes, and none of it is patient data. The log notes that something changed; the data itself stays in the normal tables.
Size per camp laptop (estimates; to be measured in step 1):
Where | Changes per day | Kept 30 days | Size |
|---|---|---|---|
One camp hub (~300 people/day, ~50 changes each) | ~15,000 | ~450,000 lines | ~60 MB |
Why reads stay fast at any size:
A tablet always asks for "lines after #N for my villages". The (village_id, seq) index jumps straight to the new lines and never scans old ones, so a full 30 days of lines is about as fast as a few thousand.
A quiet sync costs one index lookup that returns nothing.
Compared with today, where every pull re-reads every record in the village, this is far cheaper.
Keeping it small:
Nightly clean-up. A scheduled job deletes lines older than the retention period (30 days), in small batches (for example 10,000 rows at a time) so it never holds long locks. The changed_at index keeps each batch quick. At hub scale that's about 15,000 lines a day, which is trivial.
Log the parent, not noisy child rows. Counselling answers are deleted and re-inserted on every save, so 40 questions would mean 80 log lines. Instead, log one line for the parent form response. The same goes for drug and complaint rows under an OPD visit. The feed then sends the parent with its children.
Camp laptops only (plus UAT central for testing). Production central has no log or triggers, and the log is never synced to central. A laptop's log is therefore bounded by one camp's activity (about 60 MB at 30 days), whatever the number of camps. UAT central's log covers only test traffic.
Cost on saves: each save also writes one small extra row, in the same transaction. That's microseconds per save, but it must be measured on a real camp laptop (step 1). Saves running at the same time don't block each other on the counter in MySQL's default configuration (innodb_autoinc_lock_mode = 2).
Alternative if triggers are a concern: read MySQL's own change history (the binlog) with a tool such as Debezium. It adds nothing to saves, but needs Kafka or another extra service running on every camp laptop, which is a lot for field hardware. For hubs, triggers are the simpler choice.
Option | Size | Privacy | Notes |
|---|---|---|---|
Selected village (today) | Small | Lowest exposure | Doesn't work offline across villages |
All villages assigned to the user | Typically 4k–20k beneficiaries | Matches existing access rights | Recommended. The app already partly supports this: |
The camp's catchment area | Medium | Medium | Needs a camp → village mapping that doesn't exist yet |
Whole service area ( | 100k+ | Highest exposure | Rejected as the default |
Fix needed first:UserServiceImpl only reads the user's first role-mapping row (getUserRole(userId).get(0)), so villages on other rows are dropped. It should combine villages from all rows.
Beneficiaries | Records (no photos) | SQLite on tablet | Photos (~50% have one) |
|---|---|---|---|
4,000 | ~20 MB | ~15–20 MB | ~70–100 MB |
20,000 | ~100 MB | ~80–100 MB | ~350–500 MB |
100,000 | ~500 MB | ~400–500 MB | ~1.7–2.5 GB |
These figures are for the first download. With the changes feed, later pulls move only what changed.
Today: photos are sent inline as base64 inside every beneficiary record, on every pull. They are the bulk of the download.
Target: the changes feed sends only a photo reference (ID plus hash). The tablet downloads photos in the background, Wi-Fi only:
first for beneficiaries in today's worklist;
the rest over time;
or on demand when a record is opened.
Security: photos are stored encrypted. Today they are plain JPEG files in filesDir.
Village picker: the selected village becomes a view filter only. Sync always covers the whole scope.
Paging: large lists move to Paging 3. About 40 Flow<List<…>> queries load whole tables into memory and re-run on every write during sync.
Database: add indexes or precomputed columns for age and TB-status filters (the BEN_BASIC_CACHE view scans every row).
Room migrations are needed for this. That conflicts with R8, but these are schema changes for the UI, not sync state, and can come later.
fallbackToDestructiveMigration was removed in PR #321, so migrations are now safe to add.
Security:
SQLCipher on every field build; today it is off in debug builds.
Show a storage estimate and progress before the first download.
A policy for deleting data the user is no longer assigned to.
Station (role) | Writes |
|---|---|
Registrar | Beneficiary demographics; anthropometry; TB screening (both registrar and nurse have full screening rights) |
Nurse | Anthropometry, vitals/general exam, general OPD, TB screening, diagnostics orders, TB suspected, diagnostic order push |
Lab technician | Diagnostic results ( |
Counsellor | Counselling and contact-tracing forms, TB confirmed, TB suspected, beneficiary death fields |
How saves work today:
Only records edited on the tablet (marked unsynced) are pushed. Each one is sent with all its fields from that tablet's copy, not just the changed ones (asNetworkPostModel, TBScreeningSaveRequest.from(cache)).
Anthropometry, screening status and death outcome mark the whole beneficiary record as changed. The next push then re-sends demographics as they are on that tablet.
The server matches records by natural key and overwrites every column. There is no @Version anywhere.
# | Scenario | Result |
|---|---|---|
L1 | The registrar corrects a name; the nurse's tablet still has the old copy and saves anthropometry | Nurse's push restores the old name. The registrar's next pull overwrites the correct local copy too. |
L2 | Registrar records 52 kg, nurse 50 kg; later, any push from the registrar's tablet | 52 kg is appended as the "latest" anthropometry row again |
L3 | Nurse saves diagnostics orders, lab tech saves results from an older copy (one row per beneficiary) | Last push wins on every column; nulls erase the other's data |
L4 | Server writes a lab result into |
|
L5 | Registrar and nurse both screen the same day (same visit key) | The last push replaces every symptom |
L6 | Two counsellors answer the same form question | Last write wins (lower risk: the merge is per question) |
Field ownership by role, checked on the server and respected in the app:
anthropometry gets its own push, not the beneficiary record;
updateScreeningStatus stops marking the whole record as changed;
the server ignores demographic fields from non-registrar users;
diagnostics: the nurse owns orders and referrals, the lab tech and the server own results;
tb/suspected/saveAll never overwrites result fields: configure modelMapper to skip nulls, or use explicit setters.
Partial pushes: send only the fields that changed, and treat a missing key as "leave it alone". The liquidCultureResult handling already does this; extend it to every field.
Version check, using the change log:
each pushed record carries the version (seq) the tablet last pulled;
if the server's row has a newer version, the server returns a conflict with its current row;
the app re-applies only the fields its role owns, then pushes again. No user prompt for owned fields.
This version is the same one the changes feed returns, so no new column is needed.
"Fetch this beneficiary now": a quick hand-off when a person reaches a station.
GET flw-api/stoptb/beneficiary/{benRegId}/all returns the beneficiary plus all their TB, form and order records, with versions.
The Examine screen calls it when it opens, if the device is online.
Identity getByBenId already exists, but the app's call to it is commented out.
Diagram 4: Station hand-off with version check (Mermaid source below; image attachment d4.png)
sequenceDiagram
participant R as Registrar tablet
participant H as Camp hub
participant N as Nurse tablet
R->>H: push beneficiary (changed fields: name) v=40
H-->>R: ok, v=57
Note over N: beneficiary walks up to the nurse
N->>H: fetch beneficiary 123 (all records)
H-->>N: beneficiary v=57 + TB rows
N->>H: push anthropometry (own endpoint, base v=57)
H-->>N: ok, v=61
Note over N,H: if the nurse's base were older than the server's version<br/>for a field she owns, she gets a conflict:<br/>re-apply her owned fields, push again
loop every camp tick
R->>H: changes(token)
H-->>R: beneficiary 123 v=61 (weight)
end |
Diagram 5: Target design (Mermaid source below; image attachment d5.png)
flowchart LR
subgraph Tablet
UI[Screens<br/>village = view filter]
DB[(Room DB<br/>whole user scope)]
SY[Sync engine<br/>token per server,<br/>field-level push]
end
subgraph Hub["Camp hub laptop"]
API["FLW-API<br/>/sync/changes<br/>/beneficiary/:id/all<br/>save endpoints + version check"]
MY[(MySQL + t_sync_change_log<br/>triggers)]
PH[Photo store]
end
C[(Central)]
UI --> DB
DB --> UI
DB --> SY
SY --> DB
SY -->|"changes(token, scope)"| API
SY -->|"push changed fields + version"| API
SY -->|"photos, Wi-Fi only, lazy"| PH
API --> MY
MY --> API
MY -->|"upsync"| C
C -->|"downsync (triggers record arrivals)"| MY |
Layer | Change |
|---|---|
AMRIT-DB |
|
FLW-API |
|
Identity-API | Version check and field ownership on |
Common-API / FLW user detail | Combine villages from every role-mapping row ( |
MMU-API (hub sync) | Nothing strictly required: triggers on the hub record rows arriving by downsync. Never add |
STOP-TB-App | Sync engine on the changes feed with a token per server; full-scope sync; village as a view filter; lazy photos; field-level push with conflict handling; on-demand fetch; Paging 3 |
How the change-log migration is gated. The same db.war runs on central, state servers and camp laptops. The change log must be created only on hubs: camp laptops, plus UAT central (uatstoptb.piramalswasthya.org, used for testing). Never on production central or state servers.
One switch: AMRIT_HUB=true means "this server is a hub". It is a general flag, not tied to the change log: any future hub-only migration goes into the _hub folders and runs under the same switch.
Camp laptops: add AMRIT_HUB=true to camp-laptop-setup/config/env.template. The amrit-db service loads it from /opt/amrit/config/.env (EnvironmentFile in camp-laptop-setup/systemd/amrit-db.service).
UAT central: AMRIT-DB common_ci.properties gets amrit.hub=@env.AMRIT_HUB@, and only Jenkinsfile_UAT_STOPTB sets env.AMRIT_HUB = true.
Everywhere else (production central, state servers, other pipelines): unset, so the unresolved placeholder means off.
Why not reuse STOPTB_VAN_ID: it describes the van, not the server's role, and setting it on UAT central would be wrong because FLW-API uses it to stamp a van ID on every save (CampConfigService.resolveVanID, STOPTB_ENFORCE_VANID).
The pattern, already in AMRIT-DB:DbFleetConfigured migrates db_fleet only when FLEET_ENABLED=true and a real URL are set. Otherwise the Flyway bean is never created.
The design:
Condition: a new HubConfigured condition, true only when AMRIT_HUB (or amrit.hub) is exactly true. Unset values, false, or unreplaced CI placeholders all mean off.
Separate migration folders:db/migration/dbiemr_hub and db/migration/dbidentity_hub. The first migrations there are the change log table, its triggers and the clean-up; later hub-only changes are added to the same folders.
Separate Flyway beans, created only under that condition, with their own history table (flyway_schema_history_hub). Central's main history and version numbers are never touched.
Run order:FlywayMigrator runs them after the main migrations, the same way it runs db_fleet today.
Fallback: if the flag is missing on a laptop, no log is created, /stoptb/sync/changes reports that it isn't supported, and the app keeps using today's full pull. Nothing breaks. Setting the flag later and restarting amrit-db applies the hub migrations then.
FLW-API: checks the same AMRIT_HUB flag before serving the changes endpoint, so UAT central serves /stoptb/sync/changes too and tablets can be tested against it directly.
Step | Work | Repos | Unlocks |
|---|---|---|---|
0 | Quick app-only trims: master data at most once a day; confirmed cases scoped by village | App | Less load now (small) |
1 | Change log plus triggers on a hub-sized DB; measure write overhead | AMRIT-DB | Everything below |
2 |
| FLW-API | R1–R3 |
3 | App sync engine on the changes feed (one dataset type at a time, falling back to the old endpoints), token in encrypted prefs | App | R1–R3 on devices |
4 | Full scope: combined user villages, multi-village feed, village as a view filter, Paging 3 for big lists | Common/FLW-API, App | R4 |
5 | Photos as references plus lazy encrypted download | FLW/Identity, App | R4 at 20k+ |
6 | Field ownership: split the anthropometry push, stop whole-record dirtying, server ignores fields the user's role doesn't own, | Identity, FLW, App | Removes most L1–L5 |
7 | Version check on push (conflict + re-apply owned fields); fetch-beneficiary-now | FLW, Identity, App | R5, R6 |
8 | Delete flags and app handling of deletes | AMRIT-DB, FLW, App | R2 deletes |
Steps 1–3 deliver "what changed since X?". Step 4 can start in parallel once step 2 has its scope parameter. Steps 6–7 can run in parallel with 4–5.
Change log:
every write path creates a log row, including JPA, native SQL, the downsync insert/update, upsync, and a hard delete;
two parallel transactions that commit out of order are both delivered;
a token from another hub triggers a new snapshot.
Changes feed:
a snapshot followed by changes equals a fresh full pull;
0 items when nothing changed;
deletes arrive;
paging returns no duplicates and no gaps while rows are inserted during the snapshot.
Conflicts: run L1–L6 as tests. None loses data after step 7; L1, L2, L4 and L5 are already fixed by step 6.
Load at camp scale (one hub, 5 tablets, 4k and 20k beneficiaries): requests per minute, bytes per hour, hub CPU and DB load, and hand-off time (target under 30 seconds, under 3 seconds on demand).
Field comparison: a day of sync logs against the August baseline. Each record should be downloaded at most once per change.
AMRIT-DB: one db.war release goes everywhere. The change log and triggers are created only where STOPTB_VAN_ID > 0, i.e. on camp laptops (§7). Delete flags and indexes go to central and the laptops. Central gets no log.
FLW-API / Identity-API release with the new endpoints. Old apps are unaffected.
App release that uses the changes feed when the hub offers it, and old endpoints otherwise.
Field ownership and version checks are switched on per endpoint once every app in the field sends versions. Until then, a push without a version is accepted as it is today.
Change tracking: do you accept DB triggers plus a change log (recommended), or should we go with the fallback of fixing timestamps everywhere?
Scope: is it all villages assigned to the user (recommended), the camp catchment, or the whole service area?
Photos: lazy and Wi-Fi only, or none on the tablet unless the record is opened?
Field ownership map: confirm which role owns which fields, especially screening (registrar and nurse both edit it today) and death/outcome (counsellor).
Deletes on the tablet: remove the row, or keep it and hide it?
Retention: how long to keep the change log (30 days is suggested), and what a tablet that has been away longer gets (a fresh snapshot).
Old-data policy: what happens to data on a tablet when the user is no longer assigned to a village?
Logout with unsynced records: block logout until everything has synced, or allow it after a second confirmation? Logout wipes the tablet's database, so unsynced records are lost.
Change tracking: per-table assessment of mod columns, broken write paths, soft and hard deletes, and indexes, based on AMRIT-DB origin/main (no release-3.12.0 branch) and the API repos on release-3.12.0.
Key files: Identity RmnchDataSyncServiceImpl.java:187,410,485,524; TM BenVisitDetailRepo.java:64,107; FLW FormResponseItemSaver.java:349; V96 (tb_diagnostic_order); V120/V121 (downsync columns).
Hub↔central:
MMU UploadDataToServerImpl, GetDataFromVanAndSyncToDBImpl:810-864, DownSyncDataFromServerImpl:440-652, DataSyncRepositoryCentralDownload:245-354, AutoSyncService:125-135.
AMRIT-Deployment-Files/camp-laptop-setup/env.template (STOPTB_VAN_ID).
STOP-TB commit 1f19f944.
Scope:UserServiceImpl.java:31 (get(0)), BeneficiaryServiceImpl.java:143-178,229-280,603, StopTBServiceImpl getAll paths; app ServiceLocationActivity, RecordsRepo.allVillageIds, InAppDb BEN_BASIC_CACHE.
Station writes:RoleManager/RoleModuleConfig; AnthropometryViewModel.kt:84-101; BenDao.updateScreeningStatus; StopTBServiceImpl save paths (:239-301, :515-559); TBSuspectedServiceImpl:78-116; DiagnosticOrderServiceImpl:229-239,528-543,841; Identity RmnchDataSyncServiceImpl:280-418,892-948.