Recently, I've been helping a few social apps migrate overseas, and the first thing the client always says is “Can you get Southeast Asian users to stop cursing?” The second sentence is “We got penetrated again last night, is there any way?”
These days to do social applications, have not been DDoS brush screen are embarrassed to say they are engaged in the Internet. Last month, an audio-video social platform has just caught fire, within three days suffered three consecutive more than 500G traffic attacks, North American users directly into the PPT card - the boss of the party hit me at 4:00 a.m. on the phone: “Brother, can you solve the problem by adding money?”
Overseas acceleration and high defense are supposed to be a combination of love for each other. You want fast speed, you have to spread the nodes to the user's doorstep; you want to prevent attacks, you have to centralize all the traffic cleaning. But when you put the North American user traffic around Tokyo cleaning and then pull back to Los Angeles, the delay is enough to make a cup of coffee - the user uninstalled the App.
I have tested the “Global Intelligent Scheduling” boasted by a big company, and the result is that the Australian user request was thrown to the Indian node, and the delay soared to 380ms. The technical document boasts of “dynamic path optimization”, which is actually the lowest cost scheduling according to the BGP routing table, and has nothing to do with the user experience.
A truly reliable solution must solve three problems at the same time: acceleration performance, defense capability, and cost control. Without one of them, it's all a joke.
Let's start with the acceleration piece. Social application traffic characteristics are too obvious: bursty, long connections, real-time requirements perverted. Pure HTTP caching program simply can not carry, have to start from the protocol layer.
The optimization solution we made for a live broadcast platform in Southeast Asia forced the deployment of the QUIC protocol to the edge nodes. Don't look at just such a change, weak network environment latency directly reduced by 40%. Especially in India, the network environment is comparable to the lottery countries, QUIC's 0-RTT handshake is much more reliable than the TCP handshake three times.
The configuration example is not really complicated (of course the actual deployment has to tune more parameters):
But don't believe in the “QUIC can speed up” nonsense. Some CDN vendors' QUIC implementations are not optimized at all, and I've tested one that claims to be globally-accelerated, and the QUIC performance is worse than TCP - and later found out that their kernel version is too old, and they don't even have a BBR to adapt to it.
Plus the defense piece. Social apps are most afraid of CC attacks, which look like normal requests but specifically dislike the interface. User login, friend list, message push, these APIs are broken in minutes.
Last year, I helped some overseas dating software to do protection, the other engineer swore that he had used “intelligent WAF”. As a result, I took the tool to simulate normal user behavior, and bypassed their rule base in ten minutes - because their JS challenge mechanism has not been updated for three years, and the crawlers can recognize it directly.
Now effective CC defense must be combined with behavioral analysis. For example, the detection of mouse trajectory, touch event frequency, and even device power status (I really have not seen the mobile crawler will simulate power changes).
This is the dynamic challenge rule we currently use on the CDN07 platform:
Interestingly, some attack sources now mimic human behavior. A bot caught last week actually swiped the screen randomly, but failed to mimic a phone's gyroscopic tilt - who in their right mind browses a social app with their phone on end like a level?
Node coverage is the true test of CDN vendor strength. Many vendors say “global coverage”, in fact, Europe and the United States nodes piled up, Latin America and Africa rely on two nodes to support. Especially South African users, often routed to Europe and then back around.
We've compared the actual performance of the three manufacturers:
CDN5's North American nodes are really strong, but Southeast Asia is basically renting bandwidth from local second-tier vendors, and the evening peak packet loss rate can reach 15%.
CDN07's European coverage is sickly dense, with pop points even in small Eastern European countries, but South American routes often go through cheap tunnels with too many latency fluctuations.
08Host this guy is interesting, specializing in deep cultivation of emerging markets. In Indonesia directly self-built server room, Jakarta user latency pressure to 20ms, but the North American node instead of the general.
So now the reliable practice is mixed scheduling. Static resources go CDN5, API traffic with CDN07 protection, live video on 08Host's dedicated line - although the management complexity is high, but the cost can be reduced 30% or more, the performance can also be improved.
One final note on cost pitfalls. Many vendors of “unlimited protection” is purely a word game. Once a customer bought an unlimited anti-DDoS package, the results were brushed 700G was told that “beyond the scale of the business” - it turns out that the contract is hidden in the daily 300G invisible upper limit.
Our contracts now explicitly require it to be spelled out:
- Whether the protection cap is based on peak or total volume
- Whether cleaning nodes are deployed nearby (otherwise latency spikes)
- Whether the statistical dimension of CC protection is the number of requests or the number of concurrent connections
- Whether overseas nodes support localized certificates (some countries require local SSL for data landing)
To be honest, this line of water is too deep. Some vendors sell Vietnam nodes as Singapore (routing around Hong Kong), there are vendors of “intelligent routing” is actually to find a node with the lowest latency regardless of defensive capabilities. Recently, I also encountered a strange case: a CDN of the Japanese node was knocked down, automatically cut the traffic to the United States node - Japanese users directly experience a delay of 400ms, this intelligent scheduling might as well be renamed as retarded scheduling.
A truly effective solution has to be tailored to the business form. Social applications must differentiate between business types:
Users take high bandwidth lines to upload videos
Low latency lines for message push
Prioritize friend requests to ensure stability
Finally, a real suggestion: don't believe the manufacturer's demo data, run real tests yourself. Use the global cloud testing platform to simulate user behavior around the world, and test for three consecutive days during the evening peak hours. Once a vendor showed me their Singapore node latency of 50ms, the actual test found that the measured latency to the server room rather than to the user's device - the real user to the node latency average of 170ms, the peak can be up to 300ms.
It is now standard practice for us to require vendors to open up their real-time monitoring interfaces and write their own scripts to count the real user experience everywhere. Especially in India and Brazil, which are areas with complex networks, it is important to look at the 95th percentile latency data rather than the average.
There is no once-and-for-all solution for overseas acceleration, only a continuous optimization process. Recently, we are experimenting with the edge computing program, putting some of the logic directly into the CDN nodes to execute - for example, light operations such as message liking, in the local node to return directly after processing, and even back to the source are saved. Tested Canadian user interaction delay from 220ms down to 80ms, the effect is much more obvious than simply accelerating.
In the end, technical solutions are for business services. Once a customer had to go on the most expensive global anycast network, and found that the main users in Southeast Asia - save 80% budget with local vendors, the experience is better. Doing technology is most likely to fall into the trap of “the pursuit of perfection”, and sometimes the most practical program instead of the brown fast and furious.
(Finished writing and looked at the word count, actually soared to 2500. The above is purely a history of actual blood and tears, such as the same, that you have stepped on the pit. Welcome to peer exchange trolling, but do not ask me specific customer name - to face have to keep secret.)
