Edge Dashboard: Turning CloudFront and WAF Logs into Actionable Insights
Author
Maxime Roth Fessler and Adithya Bodi
Date Published
Edge Dashboard: Turning CloudFront and WAF Logs into Actionable Insights
Amazon CloudFront logs contain a wealth of information about content delivery, from bandwidth consumption and cache hit rates to HTTP errors and traffic patterns. For teams managing websites and streaming platforms, these insights can help identify performance issues, understand traffic costs, and improve delivery efficiency.
Getting those insights, however, requires more than collecting logs. Teams need to process the data, query it efficiently, and build dashboards that make the results useful. Depending on the approach, this can mean maintaining custom data pipelines or paying recurring costs to query large volumes of raw logs.
Turning CloudFront Logs into a Ready-made Dashboard
The Edge Dashboard was developed by TrackIt to make log analysis simpler. It deploys a data processing pipeline and a preconfigured Grafana dashboard directly in the AWS account, turning raw CloudFront logs into actionable insights. The dashboard also supports AWS WAF logs and optional video metadata extraction, bringing delivery performance, security analytics, and video insights into one place.
Why we Built It
CloudFront logs can be analyzed using Amazon CloudWatch Logs and Logs Insights, but query costs can add up as log volumes grow and dashboards refresh. Logs Insights charges $0.005 per GB scanned, and a dashboard querying the previous 30 days of logs can scan that data again with each refresh. Based on our estimates, a 10-panel dashboard opened five times a day could cost around $107 per month at one million log lines per day. At 10 million lines per day, the estimated monthly cost rises to roughly $980.
Other approaches come with their own trade-offs. An AWS Security Lake and QuickSight can support WAF analytics, but CloudFront performance metrics and custom dashboard development require additional work. Streaming logs through Kinesis into OpenSearch provides near-real-time visibility, but also introduces the cost and operational overhead of running an OpenSearch cluster and maintaining a separate pipeline for WAF logs.
TrackIt developed the CloudFront & WAF Analytics Dashboard to bring CloudFront performance metrics, WAF security analytics, and bot detection into one place while keeping query costs predictable. Instead of repeatedly querying raw logs, the solution processes each log file into Parquet format, allowing Athena to scan only the columns required for each dashboard panel.
At an estimated one million log lines per day, the complete stack costs around $57 per month with the default daily processing schedule, or approximately $84 per month with hourly processing. The trade-off is data freshness: the hourly schedule can leave the dashboard up to an hour behind, unlike CloudWatch Logs, which can provide near-real-time visibility.
The Dashboard
The dashboard runs on Grafana 12 and comes preconfigured with more than 50 panels across eight sections, covering CloudFront traffic, delivery performance, and HTTP errors. Optional sections add AWS WAF security analytics, TLS fingerprint analysis for bot detection, and video asset metadata.
CloudFront Edge Analytics Dashboard in Grafana: requests, bandwidth, cache hit rate and error rate at a glance.
Section | Use it to | Example panels |
|---|---|---|
KPIs | Check overall health at a glance | Total requests, bandwidth, average response time, cache hit rate, error rate, bot traffic %, unique visitors |
Traffic | See what gets requested and through which edge locations | Requests and bandwidth over time, traffic by asset type, top edge locations, top requested URLs, top bandwidth-consuming assets |
Performance | Track latency and cache efficiency | Response time and time to first byte over time, bandwidth saved by cache, slowest assets, cache hit rate by asset type |
Top statistics | Break traffic down by country, IP, network and client | Top countries, IPs, paths, user agents, networks, world map |
HTTP errors | Spot error spikes | Status codes over time, status code distribution, 5xx error rate |
WAF security (optional) | See what WAF blocks and which rules fire | Blocked vs allowed requests, attack types, top blocked IPs, country threat analysis, top triggered rules |
TLS fingerprints (optional) | Find bot-like clients by TLS fingerprint | Top JA3/JA4 fingerprints by block rate, low-volume high-block fingerprints, fingerprint diversity by country |
Media (optional) | Inventory the videos you serve | Resolution distribution, video catalog, video vs non-video bandwidth |
Traffic Analytics: CloudFront requests and bandwidth over time, broken down by asset type, edge location and URL.
HTTP Error Analytics: 4xx and 5xx responses
When enabled, the video metadata extraction feature uses MediaInfo workers to extract resolution, bitrate, duration, and file size from video assets. The workers use a scale-to-zero approach, so compute instances shut down when the processing queue is empty, helping control costs.
How it Works
The architecture uses AWS Glue to process logs and Amazon Athena to query the results, keeping data processing and analytics serverless. The only continuously running compute instance is the Grafana server.
Serverless architecture: CloudFront and WAF logs in S3 are processed by AWS Glue, queried with Amazon Athena and displayed in Grafana.
On a schedule, AWS Step Functions orchestrates Glue jobs that process new CloudFront and WAF logs, aggregate metrics by day, asset, status code, and edge location, and store the results in Parquet format. Amazon Athena makes the aggregated data available to Grafana, which runs in an Auto Scaling Group behind an Application Load Balancer and is automatically replaced if a failure occurs.
When video metadata extraction is enabled, new videos in the origin S3 bucket are queued in Amazon SQS and processed by Spot instances running MediaInfo. The solution is deployed through nested AWS CloudFormation stacks, with optional email alerts for failed processing jobs.
Deployment
The CloudFront & WAF Analytics Dashboard is available through the AWS Marketplace and deploys directly into the AWS account using AWS CloudFormation.
- Subscribe and launch: Subscribe through the AWS Marketplace and launch the CloudFormation stack.
- Configure the deployment: Provide the S3 bucket and prefix containing CloudFront logs, the CloudFront origin bucket, a Grafana admin password, and the CIDR range permitted to access the dashboard.
- Configure optional features: Add a WAF logs bucket, enable video metadata extraction, configure an ACM certificate for HTTPS, or provide an email address for pipeline failure alerts.
- Access the dashboard: Once deployment completes, open the GrafanaDashboardUrl output and sign in as admin.
During the initial deployment, a bootstrap run processes existing logs and creates the Athena tables, making historical data available from the start. Afterward, an Amazon EventBridge schedule incrementally processes new logs. The default schedule runs daily, with the option to increase processing frequency to hourly.
Cost
For a deployment processing one million log lines per day in the us-east-1 region, the estimated monthly cost is approximately $57 with daily processing or $84 with hourly processing, based on on-demand pricing. The Grafana server and its Application Load Balancer account for most of the fixed cost, while more frequent processing increases AWS Glue costs.
Component | Daily pipeline | Hourly pipeline |
|---|---|---|
EC2 (t3.medium) + ALB for Grafana | ~$48 | ~$48 |
Glue ETL (Flex) | ~$1 | ~$28 |
S3 storage + KMS | ~$7 | ~$7 |
Secrets Manager, Athena, Step Functions | ~$1 | ~$1 |
Total | ~$57 | ~$84 |
Several optimizations help keep data processing costs low:
- Lower-cost processing: AWS Glue Flex jobs are billed at $0.29 per DPU-hour, compared with $0.44 for standard execution.
- No crawler overhead: Athena partition projection eliminates the need for Glue crawlers.
- Incremental processing: Glue job bookmarks ensure each run processes only newly added files.
- Efficient queries: Aggregated data is stored in Snappy-compressed Parquet format, allowing dashboard queries to scan only the required columns. A per-query scan limit provides an additional cost safeguard.
- On-demand video processing: MediaInfo workers scale to zero when the processing queue is empty.
For deployments that will run continuously, purchasing a one-year Reserved Instance for the Grafana server can reduce the estimated monthly cost by approximately $11.
Who it is for
- Media and streaming teams that want to understand which videos, resolutions, and edge locations account for the most CDN traffic.
- Platform and DevOps teams that need cache hit rates, error rates, and latency trends without building a custom log pipeline.
- Security teams using AWS WAF that want visibility into blocked requests, geographic attack patterns, and TLS fingerprints associated with bot activity.
- FinOps teams looking to identify the assets and URL paths that consume the most bandwidth.
The solution runs entirely in the customer’s AWS account, keeping logs within that environment. S3 buckets are encrypted with AWS KMS (Key Management Service), and access to Grafana is restricted to the configured CIDR range.
Next Steps
The CloudFront & WAF Analytics Dashboard is available for free on the AWS Marketplace. Only the AWS resources used by the deployment incur charges. For help adapting the dashboard to specific requirements or reviewing an existing CloudFront configuration, contact TrackIt.
