Denial-of-service: Diagnostics & Troubleshooting

When attackers render services unavailable

Denial-of-Service (DoS) attacks are a significant threat to Linux servers, aimed at making a service unavailable to its intended users.

What denial-of-service means

A Denial-of-Service attack occurs when a malicious actor attempts to disrupt the normal functioning of a service, typically by overwhelming it with a flood of illegitimate requests. The goal is to exhaust system resources, such as memory, CPU, or network bandwidth, rendering the service unavailable to legitimate users. When multiple systems are used in unison to launch an attack, it is referred to as a Distributed Denial-of-Service (DDoS) attack.

Possible causes of denial-of-service

There are several causes behind DoS attacks, including:

Malicious intent: Attackers may target a server for various reasons, such as revenge, financial gain, or to disrupt business operations.
Vulnerable applications: Applications that are poorly coded or have known vulnerabilities can be exploited to launch an attack.
Network configuration errors: Misconfigurations in firewalls, routers, or other network devices can inadvertently expose services to potential attacks.
Botnets: Attackers may utilize a network of compromised devices to execute large-scale attacks.

How to diagnose denial-of-service

Diagnosing a DoS attack involves monitoring server performance and analyzing network traffic. Here are steps to help identify an ongoing attack:

Check resource utilization: Use the top command to monitor CPU and memory usage.
```
top
```
Examine network traffic: Use the netstat command to check for unusual connections.
```
netstat -an | grep ESTABLISHED
```
Analyze logs: Review web server logs (e.g., /var/log/apache2/access.log for Apache) for unusual patterns, such as repeated requests from the same IP address.
```
tail -n 100 /var/log/apache2/access.log
```
Use tools: Utilize tools like iftop or tcpdump to monitor network traffic in real-time.
```
sudo iftop -i eth0
```

How to troubleshoot denial-of-service

Once a DoS attack is diagnosed, it’s essential to take action to mitigate its effects. Here are some troubleshooting steps:

Rate limiting: Implement rate limiting on your server to restrict the number of requests from a single IP address.
```
sudo iptables -A INPUT -p tcp --dport 80 -i eth0 -m connlimit --connlimit-above 20 -j REJECT
```
Block offending IPs: Identify and block IP addresses that are responsible for the attack.
```
sudo iptables -A INPUT -s <attacker_ip> -j DROP
```

Use nft for advanced filtering: You can use nft to create a more sophisticated firewall rule set.

sudo nft add table ip filter
sudo nft add chain ip filter input { type filter hook input priority 0; }
sudo nft add rule ip filter input ip saddr <attacker_ip> drop

Increase server resources: Temporarily increase server resources (CPU, memory, bandwidth) to handle increased traffic.
Enable a Web Application Firewall (WAF): A WAF can help filter out malicious traffic before it reaches your server.

Practical examples

In real-world scenarios, organizations have faced various types of DoS attacks. Consider the following:

HTTP Flood: An attacker sends a large number of HTTP requests to a web server, consuming all available resources. To mitigate this, you can use Nginx to limit the request rate:
```
http {
    limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s;

    server {
        location / {
            limit_req zone=mylimit burst=5;
        }
    }
}
```
SYN Flood: This attack exploits the TCP handshake by sending a flood of SYN packets, exhausting the connection table. You can configure iptables to protect against this:
```
sudo iptables -A INPUT -p tcp --syn -m limit --limit 1/s --limit-burst 4 -j ACCEPT
```

Applications that may cause denial-of-service

Certain applications are more prone to vulnerabilities that can lead to DoS attacks. Examples include:

Web servers: Misconfigured or outdated web servers can be targeted easily.
Database servers: Poorly optimized queries can lead to high resource usage.
APIs: Insecure APIs may not correctly validate requests, making them susceptible to abuse.

Preventing denial-of-service

Preventive measures are crucial in safeguarding against DoS attacks. Here are some strategies:

Keep software updated: Regularly update your server’s operating system and applications to patch vulnerabilities.
Implement firewall rules: Set up strict firewall rules to filter incoming traffic and block suspicious activity.
Use a content delivery network (CDN): A CDN can absorb and mitigate DDoS attacks by distributing traffic across multiple servers.
Regularly monitor logs: Keep an eye on logs for unusual activity that may indicate an impending attack.

Tips and best practices

Establish a response plan: Have a clear action plan in place for responding to DoS attacks.
Use monitoring tools: Employ server monitoring tools like Nagios or Zabbix to keep track of server health.
Educate staff: Train your team on recognizing early signs of attacks and how to respond effectively.

Real-world use cases

GitHub DDoS Attack (2018): GitHub experienced one of the largest DDoS attacks recorded, peaking at 1.3 Tbps. This attack utilized a technique called Memcached amplification, exploiting misconfigured Memcached servers to amplify the attack traffic.
Dyn DDoS Attack (2016): This attack targeted the DNS service provider Dyn, which resulted in major outages for several websites, including Twitter and Netflix. It utilized a botnet of IoT devices to generate massive amounts of traffic.