DDoS attack on RDP services: recognize and fight. Successful experience from Tucha

How it all started

It all started on the morning of October 31, on the last day of the month, when many desperately need to have time to resolve urgent and important issues.

One of the partners, who keeps several virtual machines of the clients he serves in our cloud, reported that from 9:10 to 9:20 am several Windows servers operating at our Ukrainian site did not accept connections with the remote access service , users could not access their desktops, but after a few minutes the problem seemed to be resolved by itself.

We raised the statistics of the communication channels, but we did not find any traffic surges or dips. We looked at the statistics of the load on computing resources - no anomalies. What was it?

Then another partner, who places another hundred servers in our cloud, reported the same problems that some of their clients noted, and it turned out that, in general, the servers are available (they regularly respond to ping tests and other requests), but the service remote access on these servers then accepts new connections, then rejects them, while we were talking about servers at different sites, traffic to which comes from different data transmission channels.

A packet with a request to establish a connection arrives at the server: computer support jobs

xx:xx:xx.xxxxxx IP xxx.xxx.xxx.xxx.58355 > 192.168.xxx.xxx.3389: Flags [S], seq 467744439, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0

The server receives this packet, but the connection is rejected:

xx:xx:xx.xxxxxx IP 192.168.xxx.xxx.3389 > xxx.xxx.xxx.xxx.58355: Flags [R.], seq 0, ack 467744440, win 0, length 0

This means that the problem is clearly not caused by some kind of infrastructure malfunction, but by something else. Are all users having problems licensing Remote Desktop? Maybe some kind of malware has managed to penetrate their systems, but today it is activated, as it was with XData and Petya a couple of years ago ?

While we were sorting it out, we received similar requests from several more clients and partners.

What happens on these machines?

The event logs are full of messages about trying to guess the password:

DDoS-ataka na RDP-sluzhby raspoznat i poborot Uspeshnyj opyt ot Tucha_2

Typically, such attempts are logged on all servers where the standard port (3389) is used for the remote access service and access from anywhere is allowed. The Internet is full of bots that constantly scan all available connection points and try to guess the password (for this very reason, we strongly recommend using complex passwords instead of "123"). However, the intensity of these attempts that day was too high.

Recommend that customers devote a lot of time to changing settings for a huge number of end users in order to switch to a different port? Not a good idea, customers won't be happy. Recommend to only allow VPN access? In a hurry and panic, raising IPSec connections, for whom they are not raised, perhaps, such happiness does not smile at clients either. Although, I must say, this is in any case a godly business, we always recommend hiding the server in a private network and are ready to help with the settings, and for those who like to figure it out on their own, we share instructions for setting up IPSec / L2TP in our cloud in site-to-site mode or road-warrior, and if anyone wants to raise a VPN service on their own Windows server, we are always ready to share tips on how to raise standard RAS or OpenVPN . But, no matter how cool we were, this was not the best time to conduct educational work among clients, since it was necessary to fix the problem as quickly as possible with minimal stress for users.

DDoS-ataka na RDP-sluzhby raspoznat i poborot Uspeshnyj opyt ot Tucha_3

The solution that we implemented was as follows. We have adjusted the analysis of passing traffic in such a way as to track all attempts to establish a TCP connection to port 3389 and select addresses from it that try to establish connections with more than 16 different servers in our network for 150 seconds - these are the sources of the attack ( of course, if one of the clients or partners has a real need to establish connections with so many servers from the same source, you can always add such sources to the “white list.” Moreover, if in one class C network for these 150 seconds, more than 32 addresses are detected, it makes sense to block the entire network. The blocking is set for 3 days, and if during this time no attacks were made from this source, this source is automatically removed from the "black list".here .

We are ready to share the source code of such a system , there is nothing super complicated in it, and at the same time it can be adapted and used not only to protect against such an attack, but also to detect and block any attempts to scan the network: follow this link.

In addition, we made some changes to the settings of the monitoring system, which now more closely monitors the reaction of the control group of virtual servers in our cloud to an attempt to establish an RDP connection: if the reaction did not follow within a second, this is a reason to pay attention.

The solution turned out to be quite effective: there are no more complaints from both customers and partners, and from the monitoring system. New addresses and entire networks are regularly blacklisted, which indicates that the attack continues, but no longer affects the work of our clients.

There is safety in numbers

Today we learned that other operators have faced a similar problem. Someone still thinks that it was Microsoft that made some changes to the remote access service code (if you remember, we suspected the same on the first day, but we rejected this version very soon) and promises to do everything possible to find a solution rather.

Someone simply ignores the problem and advises clients to defend themselves on their own (change the connection port, hide the server in a private network, and so on).

DDoS-ataka na RDP-sluzhby raspoznat i poborot Uspeshnyj opyt ot Tucha_4

And on the very first day, we not only solved this problem, but also created some groundwork for a more global threat detection system that we plan to develop.