Recently helped a few customers to do penetration testing, a turn over the CDN logs I'm happy - good guy, the attack traffic is almost overflowing, the protection strategy is still stuck in the “blocking IP” this primitive stage. These days, even the CDN have to “defense teammates”, you spend a lot of money to buy high defense services, if you will not look at the logs, basically equal to the hackers to spread money.
I've seen too many teams take the high-defense CDN as a “black box”, thinking that opening the protection will be able to rest easy. In fact, CDN logs are richer in attack clues than firewall alarms. But the question arises: hundreds of gigabytes of log files, in the end, which fields should be concerned about? What data is a real danger signal? Don't worry, today I'll use real-world experience to help you disassemble.
First of all, a classic misunderstanding: many people come up to stare at the bandwidth peak. Bandwidth spikes of course to be vigilant, but now the CC attack has long been refined - people with low-speed slow hit, each IP request only a few times per second, picking your dynamic interface to start. I have tested CDN5's logging system, their QPS distribution chart is much more sensitive than the bandwidth chart, often 20 minutes in advance to capture the abnormal climb.
The really damning metrics are hidden in the HTTP status code. Don't just look at the 404/500 error, that may be a program bug. focus on 499 (client active disconnect) and 429 (frequency limit). Last time there was a e-commerce station was woolgathering, the attacker specialized in using a large number of IP to launch the payment interface request and then immediately canceled, the log is full of 499. later we configured on the CDN07:
Directly let the exception request drop 90%.
URL aggregation analysis is the ace move in log mining. Even the most cunning hackers will leave patterns - such as a large number of requests for sensitive paths like `/wp-admin.php` and `/api/v1/user/login` at the same time. 08Host's logs panel has a great feature: it automatically tags the frequency of visits to similar URLs, and can cluster and display requests that have been coded in a way that is bypassed. Even coding bypassed variant requests can be clustered and displayed. I often recommend that customers set up threshold alerts to notify them when a single URL exceeds 2,000 requests per minute by SMS.
When it comes to IP behavior analysis, don't believe that “blocking a single IP” will solve the problem. Advanced attacks are all about IP pool polling, where each IP is used a few times. But a machine is still a machine, and there are always cracks. Look at this real case: in a DDoS, although all the IPs are new, I found that the User-Agents are all `Go-http-client/2.0`, which is obviously a Go script. Directly in the CDN5 background to add a rule:
Instantly cut out 70% of junk traffic.
Geographic distribution maps are also a treasure trove of tools. Normal user access has a geographical concentration (such as domestic business 90% traffic from the territory), but the attack traffic often global jump. Last week there was a business station suddenly appeared a large number of South African visits, a check is really blasting login. Later, they opened a geographical access control on CDN07, and the non-business area traffic was directly rejected, and the pressure on the server plummeted.
Finally, let's talk about an advanced technique: log timing correlation. Simply looking at the instantaneous peaks will miss the slow attacks, you have to stretch the timeline to see the trend. For example, the API interface visits are usually 100 per minute, suddenly become 500 and last 10 minutes - this is more dangerous than an instant burst of 5,000, because it is more like artificially controlled low-frequency penetration. A good CDN (such as 08Host) should support customized time window alerts, not just the default 5-minute statistics.
In fact, the defense strategy optimization on the three core: before the buried points (log fields to play all), in the matter of correlation (multi-dimensional cross-analysis), after the automation (alarm linkage ban). Many enterprises are stuck in the first step: even Referrer, X-Forwarded-For these basic fields are not recorded, and there is no way to start the later analysis.
To give you an example configuration, this is the format of the logs that need to be additionally logged on the Nginx side:
Remember, CDN logs are not databases for archiving, but real-time battle maps. Don't look at some vendors blowing up, really good logging system must support: original log download, API real-time query, customized dashboard. CDN5 in this regard to do quite hard, even the TCP handshake time can be taken out to do latency analysis, to help customers uncovered a lot of slow attacks.
In the end, the greatest value of a high-defense CDN is not to carry traffic hard, but to give you ammunition for analysis. Next time you look at the logs, try my set of combinations: first pull the URL frequency TOP100, and then cross-analysis of the state code and geographic distribution, and finally use User-Agent features to do the secondary filtering - to ensure that you can dig out those hidden deep “pseudo-normal” request. "Requests.
Security attack and defense is essentially a cost confrontation. You take out an attacker's labor costs with automated analysis, they can't win. Go check your CDN logging configuration now, and don't say I didn't warn you - there are some potholes that you have to wait until they fall apart before you think about filling them.
