1. Ticket Details
Ticket ID | AMM-1402 |
Severity | Critical |
Category | Performance |
Affected Module / Feature | 104\ECD |
2. Issue Description
In Assam production not able to perform any operations. Unable to close the calls(both inbound and outbound). CPU utilization also more than 70-80%.When executing ecd monthly reports creating more cpu utilization&performance issues.
3. Root Cause Analysis (RCA)
Use one or more of the following techniques for RCA:
5 Whys
Fishbone (Ishikawa) Diagram
Fault Tree Analysis
Log/Trace review
Primary Root Cause Identified:Huge dataset extracted from production server and mysql sleep connections also not closing properly
Why it happened: Extraxted ecd monthly reports containing more than 1 lakh records
Why wasn't it caught earlier? Bulk report data extraction testing not happened in Dev&Preprod environments
4. Corrective Actions (Fixes for this instance)
Immediate actions to resolve the issue and restore service.
5. Preventive Actions (To prevent recurrence)
Systemic changes, monitoring, alerting, process updates, or automation.
6. Verification of Effectiveness
Describe how you'll verify that the corrective and preventive actions have worked (e.g., logs, metrics, test cases, recurrence check).
7. Lessons Learned
Summarize learnings and recommendations to share with team or org-wide.