Recently, I helped an e-commerce station to do emergency response, and I ran into an evil thing - the server did not go down, the bandwidth did not run full, but the website was so stuck that I could not even log in to the background. After half a day's investigation, I realized that hackers these days are starting to play "slow work": each connection with you to grind half an hour, thousands of fake users can take up the concurrent connection pool, serious users simply can not squeeze in.
This slow attack (Slowloris) targets the weak points of traditional firewalls. Ordinary WAFs focus on traffic peaks and request frequency, but this attack each request is disguised as legitimate traffic, like water dripping through the stone like slowly drain your resources. I have tested a mainstream firewall, the default configuration is actually a single client with a slow POST attack dragged down the back-end server.
High-defense CDN can prevent this thing, the key to look at two points: can accurately identify the "grinding" request, dare to take the initiative to cut off abnormal connections. The following share of the actual program integration of CDN5, CDN07 two vendors of the underlying logic, by the way, spit under some vendors fancy but no use of the function.
Let's start with the request timeout setting. This is not a simple set a global timeout on the end of the matter, it must be controlled at different levels. Front-end clients to the CDN edge nodes, edge nodes to the source station, the timeout policy for these two links must be adjusted separately. Don't believe those vendors bragged about "intelligent timeout", I've raked the default configuration of CDN07, the front-end default timeout is relaxed to 300 seconds, it is simply a backdoor to the slow attack.
This is the recommended way to segment the configuration (using Nginx as an example):
Pay special attention to the client_body_timeout parameter - slow POST attack is by slowly passing the request body to consume resources. The test found that more than 30 seconds has not finished transferring the body of the request, 99.9% is a malicious connection. There is a pit to remind: some CDN vendors console timeout settings hidden in the "advanced configuration", the default value is too high, remember to manually lower.
However, just rely on timeout interception will mistakenly hurt the normal large file upload. Last year, when we did penetration testing for 08Host, we found that they set the timeout too aggressively, which caused the designer to fail to upload PSD files all the time. Later, we added a dynamic timeout policy: the /content/upload path is relaxed to 120 seconds, while other paths maintain 30 seconds. This move CDN5 do quite smart, support for setting the timeout threshold by URL path.
A more advanced play is anomalous behavior identification. Slow attacks, while mimicking normal users, expose features at the protocol layer. For example, a normal user would not send a byte only every 30 seconds or keep 500 idle connections continuously. We have deployed a set of detection rules on CDN07, and the core logic is to count the "connection efficiency" of each IP:
This set of rules to catch slow attacks is particularly accurate, but you have to be careful to distinguish between crawlers and real users. Some search engine crawlers will deliberately slow down to avoid disturbing the site, we have suffered the loss of mistakenly blocked - later added a whitelist, the verified Googlebot to relax the policy.
What really saves me is CDN5's intelligent scheduling modules. They don't simply look at the timeout, but also analyze it in conjunction with the TCP stack state. For example, if there are suddenly a large number of TCP connections stuck in the LAST_ACK state, or the SYN_RECV queue continues to be full, the system will automatically trigger the slow attack cleaning mode. This feature is measured to reduce 70% manual intervention, is the price is a little beautiful.
Emergency response also found a tawdry operation: some attackers will use slow HTTP + low-speed CC attack combo. The first accounted for the connection pool does not let go, at the same time with low-speed CC attacks to consume the CPU. this time to CDN07 linkage defense mechanism came into play - when the detection of slow connections and low-frequency requests from the same IP segment, directly pull the black the entire C-segment, rather than wrongly killed can not be spared.
Finally give a real suggestion: do not expect to rely on CDN alone to prevent all slow attacks. We have done three layers of defense on the customer's side: CDN edge layer for timeout blocking, WAF layer for behavior analysis, and iptables for concurrent connection limitation on the source server. In particular, the Linux kernel parameter tuning can enhance the server's ability to resist slow attacks by three times:
Slow attacks can not prevent the essence of the resource consumption war. Last year, we tested three CDN vendors' ability to resist slow speeds, and CDN5 won by elastic resource pools - automatically expanding the connection limit when an attack is detected, and releasing resources after the attack stops.08Host is the most realistic, although the function is not so fancy, but the default configuration is safer than others, and is suitable for small and medium-sized websites that don't want to make a big fuss.
Now hackers are getting more and more sophisticated, and have started to use distributed slow attacks - each broiler only opens a few slow connections, and looks no different from normal users. In the future, I'm afraid we'll have to use machine learning to analyze user behavior patterns. At this stage, however, getting the timeout settings and anomaly detection right has been able to take out the 90% slow attack. Remember, the security policy is not the more complex the better, the key is to be able to implement in place.
Really want to assess the high defense CDN, it is recommended to build their own environment to test: slowhttptest tool to simulate the attack, to see if the console report can accurately alarm, cleaning policy is really effective. Do not believe that the sales blow "100% protection", I have tested a major high defense package, the default rules even basic slow attack did not stop - in the end, you have to rely on manual adjustment of the rules.
Web security is an iterative process of attack and defense. A strategy that works today may be bypassed tomorrow. Maintaining configuration transparency and regularly rehearsing emergency responses is much more realistic than piling up security products. After all, these days, even the CDN have to "prevent teammates" - some vendors in order to reduce the rate of false positives, intentionally relax the security policy, something happened instead of dumping the pot to the customer misconfiguration.
