Ticket ID | AMM-1402 |
Severity | Critical |
Category | Performance |
Affected Module / Feature | 104\ECD |
In Assam production not able to perform any operations. Unable to close the calls(both inbound and outbound). CPU utilization also more than 70-80%.When executing ecd monthly reports creating more cpu utilization&performance issues.
Use one or more of the following techniques for RCA:
5 Whys
Fishbone (Ishikawa) Diagram
Fault Tree Analysis
Log/Trace review
Primary Root Cause Identified:Huge dataset extracted from production server and mysql sleep connections also not closing properly
Why it happened: Extraxted ecd monthly reports containing more than 1 lakh records
Why wasn't it caught earlier? Bulk report data extraction testing not happened in Dev&Preprod environments
Immediate actions to resolve the issue and restore service.
Systemic changes, monitoring, alerting, process updates, or automation.
Describe how you'll verify that the corrective and preventive actions have worked (e.g., logs, metrics, test cases, recurrence check).
Summarize learnings and recommendations to share with team or org-wide.