I have been attempting to generate a PCI DSS compliance report for the last two audit cycles, and the process consistently fails during the data aggregation phase. The primary error logged indicates a "timestamp mismatch" between log sources, but the diagnostic output is insufficient to pinpoint the exact source of the discrepancy. Our environment is hybrid, with on-premises Windows Event Logs, syslog from Cisco ASA firewalls, and custom application logs from AWS EC2 instances being forwarded to our LogRhythm Data Processors.
From my analysis of the Data Indexer and Data Processor health metrics, all services appear operational. I have validated the following configuration points:
* **Time Synchronization:** All source systems, Data Processors, and the Platform Manager are synchronized to the same internal NTP stratum-1 source. Offsets are consistently under 10 milliseconds.
* **Log Source Configuration:** Each log source is defined with the correct timezone (UTC) and the timestamp format regex appears to match the incoming log samples. For example, our ASA syslog format is correctly parsed using:
```
^%{NUMBER:version} %{TIMESTAMP_ISO8601:timestamp} %{SYSLOG5424SD:structured_data} %{GREEDYDATA:message}
```
* **Report Parameters:** The PCI report's time window is set to a full calendar month, and I have verified the `ReportStart` and `ReportEnd` parameters align with the log source timestamps.
Despite these checks, the report engine fails with error code `LRR-1147`. The most granular detail I can extract from the AI Engine logs suggests the failure occurs when correlating Windows security event ID 4688 with corresponding network traffic logs from the ASA, implying the correlation engine is rejecting the paired events due to a perceived temporal misalignment.
My specific questions for the community are:
1. Has anyone encountered `LRR-1147` in the context of PCI reporting, and if so, what was the root cause beyond the generic "timestamp mismatch"?
2. Are there known issues with LogRhythm's internal time normalization when logs originate from mixed sources (e.g., Windows `EventTime` vs. syslog `timestamp`), particularly when the Data Processors are in different geographic regions but all set to UTC?
3. What is the most effective method to audit the *actual* timestamp being stored in the Data Index for a specific log, compared to the original log message? I suspect the issue may be in the metadata transformation phase rather than the initial parsing.
I am prepared to share anonymized log samples and configuration snippets if it aids in collaborative debugging. My next step is to write a custom SQL query against the LogRhythm database to directly compare timestamps, but I wanted to consult existing expertise before proceeding down that path.