Operational changes for Splunk users include the same index names and SPL queries, a host field that now reflects the SVM name instead of the EC2 hostname, a source field corresponding to the fsxn-observability path instead of the syslog path, and a delivery latency that shifts from near real-time to a default five-minute polling window.

Migration strategy is designed to avoid any data loss, starting with a parallel deployment where the serverless pipeline runs alongside the existing EC2 setup using a separate Splunk index for validation, followed by a cutover once event parity is verified and finally cleanup after production validation.

What’s next focuses on enabling a high-volume path via Kinesis Data Firehose with a built-in Splunk destination, ensuring HEC acknowledgment, mapping CIM fields, pre-creating indexes, and reinforcing production readiness with updated documentation and repositories.

Networking considerations cover where to place Splunk (cloud, private, or VPC) and ensuring end-to-end connectivity, including VPC, PrivateLink, and NAT configurations.

Intro: The piece highlights a serverless modernization that slashes AWS infrastructure costs for FSx for ONTAP audit log delivery to Splunk by about 90% while keeping Splunk licensing unchanged.

SPL query examples are provided to analyze failures, build an operations timeline, identify top users, and support targeted investigations.

Rollback plan: if issues arise, revert to the EC2 logging stack, delete the serverless stack, and use checkpointing in the Lambda pipeline to prevent data loss during overlap.

For sustained high throughput (over 1,000 events per second), a Firehose path is available using Kinesis Data Firehose with a built-in Splunk destination.

An HEC event example is provided, detailing the JSON structure with time, host, source, sourcetype, index, and the event payload fields.

Compared to the old flow (FSx for ONTAP → EC2 syslog-ng → EC2 Universal Forwarder → Splunk) which cost about $66 per month, the new flow (FSx for ONTAP → S3 Access Point → Lambda → Splunk HEC) costs about $6 per month and eliminates ongoing operational burdens.

A serverless modernization of the FSx for ONTAP audit log delivery to Splunk is deployed, where FSx writes to an S3 Access Point, an EventBridge rule triggers a Python 3.12 Lambda that reads from S3, parses JSON and EVTX, formats them as Splunk HEC events, and sends to Splunk with checkpoints stored in SSM Parameter Store; Splunk receives via the HEC endpoint at https://<splunk>:8088/services/collector/event.

The new serverless setup reduces infra costs to about six dollars per month versus roughly sixty-six dollars per month for the EC2-based stack, while maintaining Splunk licensing; EC2 costs vary by region and instance type.