Overview
The reports provided by the cdn console are sometimes not detailed enough. By analyzing CDN access logs, you can pinpoint problems such as low cache hit rates and excessive back-to-origin requests. This article uses Upyun and Cloudflare as examples to introduce log analysis methods and common commands.
I. Why Analyze CDN Logs?
CDN reports only show the overall hit rate and cannot locate details for specific URLs, geographic regions, or time periods. Through logs, you can: find out which files have never been cached; discover excessively high back-to-origin rates in a certain region; troubleshoot cache issues caused by specific user agents; and analyze the distribution of HTTP status codes.
II. Obtaining CDN Logs
Upyun: Export from the "Log Analysis" section in the console, or configure log pushing to object storage or AWS S3, with support for hourly sharding.
Cloudflare: The free plan can collect logs via "Log Analysis" or Worker scripts; the Enterprise plan has detailed logs. Individual sites can use third-party tools such as Logflare for integration.
Alibaba Cloud/Tencent Cloud: Support custom log fields and push them to your own server for analysis.
III. Common Analysis Commands
Assume the log file is named cdn.log, with fields including: time, client IP, request method, URI, status code, bytes, cache status (HIT/MISS), Referer, and User-Agent. The following commands are used in a Linux environment.
1. Check cache hit rate
awk '{print $NF}' cdn.log | sort | uniq -c
Assume the last column is the cache status; count the proportion of HIT occurrences.
2. Find the files with the most back-to-origin requests (MISS)
grep "MISS" cdn.log | awk '{print $6}' | sort | uniq -c | sort -nr | head -20
$6 is the requested URI.
3. Count back-to-origin rate by geographic region (if you have an IP mapping database)
First use a script to convert IPs to country codes, then count. Use open-source tools like ip2location.
while read line; do ip=$(echo $line | awk '{print $2}'); country=$(geoiplookup $ip | awk -F: '{print $2}'); echo "$country $(echo $line | grep -q MISS && echo 1 || echo 0)"; done < cdn.log
4. Count cached status codes
awk '{print $9, $NF}' cdn.log | sort | uniq -c | sort -nr
Pay attention to non-200 responses that are cached (e.g., 404 may not need to be cached).
IV. Common Issues and Optimization Suggestions
Issue 1: The same resource URL with dynamic parameters causes a low hit rate
For example, style.css?version=1, style.css?version=2. Solution: Configure "Ignore specified parameters" (such as version) as the cache key in the CDN console.
Issue 2: High back-to-origin in certain regions
This may be because the CDN has few nodes in that region or there are routing issues. You can contact CDN customer service to adjust scheduling, or consider adding a dedicated regional acceleration domain.
Issue 3: Mobile User-Agent causes cache misses
Some CDNs cache different versions based on User-Agent by default. If mobile and desktop return the same content, you can disable the "Vary: User-Agent" header.
V. Example Automatic Analysis Script
Write a simple bash script to automatically analyze the previous day's logs every day and email a report summary.
#!/bin/bash
LOG_FILE="/var/log/cdn/$(date -d 'yesterday' +%Y%m%d).log"
HIT=$(grep -c "HIT" $LOG_FILE)
MISS=$(grep -c "MISS" $LOG_FILE)
TOTAL=$((HIT+MISS))
HIT_RATE=$(echo "scale=2; $HIT*100/$TOTAL" | bc)
echo "CDN Hit Rate: $HIT_RATE%" | mail -s "CDN Report" admin@example.com
Add it to cron to run daily.
Through log analysis, you can precisely optimize caching strategies and reduce CDN costs.
