Getting diagnostic files off van-side disks and into S3 — decrypted, viewable, behind a durable URL — via a dedicated MMU-API push endpoint that runs alongside, not inside, the existing van-to-server sync engine.
FLW-API stores diagnostic-order documents — X-rays, lab PDFs, MTB reports delivered by the EMR Lite vendor — as AES-encrypted files on the van's local disk, at {storage-root}/{beneficiaryId}/{diagnosticOrderId}/{DOCUMENT_TYPE}.enc. Only metadata lives in the database: db_iemr.tb_diagnostic_document tracks the path, a SHA-256 hash, content type, and the usual van-sync bookkeeping columns — the same table shape on both the van's local DB and the central DB, owned by one shared AMRIT-DB migration set.
TM-API already runs a “central” deployment role — package dataSyncLayerCentral, reachable at whatever base URL ${MMU_API} resolves to (referred to below as MMU-API) — that receives the existing generic van-to-server sync traffic. That pipeline moves plain relational rows: the central INSERT/UPDATE is sized strictly off a configured column list, and row values are read positionally off the same list. A file's bytes don't fit that shape — an extra field desyncs the parameter count and fails the whole batch at the JDBC layer.
So documents get their own path: a small, purpose-built endpoint pair that mirrors the shape of van-to-server (batches, acks, a processed flag) without touching its machinery — ending in S3 rather than a second copy on central's own disk.
FLW-API picks up to 3 documents where docsProcessed = 'N' (oldest first), decrypts each .enc file and computes the SHA-256 of the plaintext, then POSTs them (sha256 + base64 plaintext) to MMU-API.
MMU-API verifies the hash and PUTs the plaintext to S3 under diagnostic-documents/{beneficiaryId}/{diagnosticOrderId}/{documentType}.<ext>, in a private bucket with server-side encryption at rest.
MMU-API records the permanent object URL (the deterministic virtual-hosted S3 URL for that key) and generates a presigned GET URL for immediate use.
MMU-API upserts the central tb_diagnostic_document row with that permanent s3Url and docsProcessed = 'P'.
MMU-API returns a per-document ack (with the presigned URL). FLW-API flips docsProcessed to 'P' only for rows actually acked; unacked rows stay 'N' and retry next call; permanently-failing rows move to 'E' (see section 8).
One AMRIT-DB migration adds two columns to tb_diagnostic_document. The schema is shared, so the same statement applies to both the van's local database and the central one.
AMRIT-DB/src/main/resources/db/migration/dbiemr/V102__DiagnosticDocument_S3Sync.sql
ALTER TABLE db_iemr.tb_diagnostic_document
ADD COLUMN docsProcessed VARCHAR(1) NOT NULL DEFAULT 'N',
ADD COLUMN s3Url VARCHAR(2048) NULL;
docsProcessed follows the same 'N'/'P' convention as the existing processed column elsewhere in this pipeline — van rows use it to mean “not yet pushed,” central rows to mean “written to S3.” A third value 'E' marks a row that failed too many times and should stop retrying.
Note. The NOT NULL DEFAULT 'N' backfills every existing central row to 'N'. That is harmless only if nothing on central selects rows by docsProcessed = 'N' — confirm before shipping, since on central that value is otherwise meaningless.
A new batch picker and pusher, independent of the file-ingest path that already exists. Decryption and hashing both happen here, on the van.
DiagnosticDocument.java gains docsProcessed and s3Url fields. DiagnosticDocumentRepo gains one derived query:
List<DiagnosticDocument> findByDocsProcessedOrderByIdAsc(
String docsProcessed, Pageable pageable);
Fetch up to batchSize (default 3) rows where docsProcessed = 'N', oldest first.
For each, read the .enc file at {storage-root}/{stored_path}, decrypt it to plaintext with the shared AES key, and compute the SHA-256 of the plaintext. Base64-encode the plaintext for transport.
POST the batch as a JSON array to mmuDiagnosticDocumentPushUrl (${MMU_API}/dataSync/diagnostic-documents).
Read back a per-document ack array. For each SUCCESS, set docsProcessed = 'P' and store the returned s3Url locally. Unacked rows stay 'N'; rows returned as a hard/permanent failure increment an attempt count and, past the cap, move to 'E'.
Request body — one entry per document
{
"diagnosticOrderId": 4821,
"beneficiaryId": 90234,
"documentType": "XRAY_CHEST",
"storedFileName": "XRAY_CHEST.enc",
"sha256Hash": "a3f9...", // of the plaintext
"contentType": "application/pdf",
"vanID": 12, "parkingPlaceID": 3, "vanSerialNo": 5510,
"fileContentBase64": "..." // decrypted plaintext bytes
}
Trigger. Manual for now, at POST /diagnostic/documents/pushToCentral — the same manual-trigger shape TM-API already uses for /dataSyncActivity/van-to-server. A @Scheduled variant is a drop-in later if polling turns out to be wanted.
New endpoint, sitting next to the existing sync receiver in the same controller and under the same role check.
POST /dataSync/diagnostic-documents
@PreAuthorize("hasRole('DATASYNC') || hasRole('DATA_SYNC')")
For each document in the batch, a new DiagnosticDocumentCentralIngestService:
Base64-decodes the file and recomputes its SHA-256 over the plaintext, comparing it to the hash FLW-API sent; a mismatch rejects that item only, not the batch — a transit-integrity check on the received bytes.
Writes the plaintext bytes to S3 via S3Client.putObject, key diagnostic-documents/{beneficiaryId}/{diagnosticOrderId}/{documentType}.<ext>, into a private bucket with SSE (SSE-S3 or SSE-KMS) enabled at rest.
Builds the permanent object URL for that key and stores it as s3Url. Separately, generates a presigned GET URL via S3Presigner to return in the ack for immediate viewing.
Upserts the central tb_diagnostic_document row — matched on the table's (diagnostic_order_id, document_type) unique constraint (confirm it exists; see section 8) — with the metadata, the permanent s3Url, and docsProcessed = 'P'.
Returns a per-document ack (status + s3Url + presigned URL) so the van only marks what actually landed.
Central handles no crypto: FLW-API has already decrypted, so MMU-API receives, verifies, and stores plaintext.
The requirement is a URL you can persist once and fetch the document from later, indefinitely. A presigned URL cannot satisfy that: it is signed with an expiry and S3's SigV4 scheme caps any presigned URL at 7 days, and because it would be generated at write time, the clock starts the moment the file lands. A link saved into s3Url would be dead long before anyone opened the row.
So the durable value and the access mechanism are separated:
s3Url stores the permanent object URL — the deterministic virtual-hosted path to the key (https://{bucket}.s3.{region}.amazonaws.com/diagnostic-documents/{beneficiaryId}/{diagnosticOrderId}/{documentType}.<ext>). The key is deterministic from IDs, so this reference never breaks and never needs regenerating.
Access is granted at view time: the read path re-presigns a short-lived GET URL on demand from that key. The stored value is forever; each fetched link is fresh.
Why not just make the object public so the plain URL works directly? That is the only way a stored URL is both permanent and directly clickable with no server step — but it exposes patient medical records to anyone who obtains the link. See section 7. The plan keeps the bucket private and re-presigns; if a truly public direct link is a hard requirement, treat it as a deliberate PHI-exposure decision, not a default.
Storing the file decrypted is what makes it directly viewable without the app-managed AES key. That is a normal pattern provided the surrounding controls hold:
Private bucket. Block Public Access on; no public ACLs; no bucket policy granting anonymous s3:GetObject.
Encryption at rest. SSE-S3 or SSE-KMS on the bucket, so “decrypted” means “no longer double-encrypted by the app,” not “stored in the clear on disk.”
Access only via presign. Reads go through a server endpoint that checks the caller's authorization, then presigns — the object is never reachable by URL alone.
Transport + key handling. The van holds the AES key — it already decrypts its own files — and now sends plaintext, so TLS on the push call is essential: the payload is plaintext PHI in transit, not ciphertext.
These are diagnostic records under health-data handling rules. Decrypting on the van and storing centrally in a private bucket is defensible; combining decrypted storage with a public permanent URL is the combination to avoid.
Retry cap / poison-document handling. Without an error state, a file that fails the central SHA-256 recheck stays 'N' and retries every call forever. The 'E' state plus an attempt counter stops that; failed rows are then visible for manual follow-up.
Payload size. Base64 inflates bytes by ~33%, and imaging files run to several MB. Three encoded files in one JSON body can reach tens of MB — verify request-body size limits and read/write timeouts on both the FLW-API client and the MMU-API endpoint, and keep the batch at 3 until measured.
Confirm the unique constraint. The central upsert relies on an existing (diagnostic_order_id, document_type) unique constraint as its conflict target. Confirm it is actually present on the table, or the upsert won't dedupe correctly.
Central docsProcessed = 'N' backfill. Re-check (from section 3) that nothing on central reads rows by docsProcessed = 'N', since the migration sets every existing row to it.
File extension in the S3 key. Because objects are decrypted, use the real content-type extension (.pdf, .jpg, …) rather than .enc, so a presigned link opens/renders correctly in a browser.