Need some linux help with the server

[Gryzor]

Hello everyone!

As you may have noticed for quite a while now the server has been occasionally slow and even not responding. The reason is, there has been (again) an influx of bots hitting the site.

I updated the blocking rules for Asian countries*, but until we see if this resolves the issue I'd like to investigate some more.

So here's the help I need, if there are any Linux experts in here:

I would like to monitor the incoming connections for a little while. That's easy to do, in a multitude of ways. BUT, I would like to export those connections in a easily readable format to parse in Excel and sort, so I can identify IP ranges. And I haven't found a solution to this.

So if anyone knows of a solution, pretty please, step forward

Thanks!


*I seem to remember there were a couple of persons having issues with this, so please get in touch

[ajcasado]

Hi @ikonsgr,
I'm a linux  user, but not an expert. Anyway I asked chatgpt and obtained the following response, I hope it fits your needs:

You can use

Code Select Expand

ss or

Code Select Expand

netstat in combination with a loop and some filtering to log incoming connections, then export to CSV for Excel. Here's a simple example using ss:

Code Select Expand

ss -ntu state established | awk '{print $5}' | cut -d':' -f1 | sort | uniq -c | sort -nr > connections.txt

This gives you a count of IPs. If you want a CSV format:

Code Select Expand

ss -ntu state established | awk '{print $5}' | cut -d':' -f1 | sort | uniq -c | awk '{print $2","$1}' > connections.csv

That outputs  IP,count — ready to open in Excel.
For continuous monitoring, wrap it in a loop or use

Code Select Expand

tcpdump with

Code Select Expand

-tttt and export the output for parsing.
Let me know if you need a more persistent/logging setup.

[Gryzor]

I'm not ikonsgr but thanks for your trouble, really appreciate it! It hadn't occurred to me to use AI for that.

I think netstat has been deprecated, though you can use it in a hackity way to get results. ss works quite better, tcpdump too if you can simplify the output. I tried your solutions, I got some interesting results but it needs some work to get exactly what I need. But it works.

I also tried asking Perplexity and it gave me this nice, simple script which I modified which also works fine (I didn't ask for unique IPs you may notice):

Code Select Expand

#!/bin/bash
output_file="connection_ips.txt"
echo "IP Address,Timestamp" > $output_file
monitoring_time=120 # 2 minutes

end=$((SECONDS+monitoring_time))
while [ $SECONDS -lt $end ]; do
    netstat -tn | grep ':80 ' | awk '{print $5}' | cut -d: -f1 | while read ip; do
        echo "$ip,$(date +"%Y-%m-%d %H:%M:%S")" >> $output_file
    done
    sleep 1
done

echo "Monitoring complete. Results saved to $output_file"

[Bryce]

You could add a log rule to your iptables firewall config. That would log the connections in the Kernel log file. You can then filter them out to analyse them.

Bryce.

[Gryzor]

You could add a log rule to your iptables firewall config. That would log the connections in the Kernel log file. You can then filter them out to analyse them.

Bryce.

Yeah, I've been resisting using iptables until I realize there's no simpler/better way😁

[ajcasado]

I'm not ikonsgr

Oops, my brain must've lagged harder than the server, apologies chief! , anyway, glad I could help!

I haven't tried Perplexity yet, but the script it generated looks solid, definitely worth a shot.

[Gryzor]

I'm not ikonsgr

Oops, my brain must've lagged harder than the server, apologies chief! , anyway, glad I could help!

I haven't tried Perplexity yet, but the script it generated looks solid, definitely worth a shot.

I think the original script had an error (didn't try it, I edited it before running, maybe it was my idea and it was fine).

Perplexity is pretty nice actually,I found a pro subscription on the cheap and I use it for some research, really loving it. Their own llm is pretty good (though it offers others, too).

The only drawback compared to ChatGTP is it doesn't have long-term memory (I think only ChatGTP does?).

[lmimmfn]

You can use:
lsof -i

It will show in columnar format the incomming connections to the server which can be imported to excel, heres a sample output:
COMMAND      PID    USER  FD  TYPE    DEVICE SIZE/OFF NODE NAME
node      667384 lmimmfn  63u  IPv4 197438208      0t0  TCP localhost:40022->localhost:27017 (ESTABLISHED)
node      667384 lmimmfn  64u  IPv4 197438209      0t0  TCP localhost:40036->localhost:27017 (ESTABLISHED)
node      667384 lmimmfn  65u  IPv4 197438210      0t0  TCP localhost:40048->localhost:27017 (ESTABLISHED)
node      667384 lmimmfn  66u  IPv4 197438211      0t0  TCP localhost:40052->localhost:27017 (ESTABLISHED)
node      667384 lmimmfn  67u  IPv4 197438230      0t0  TCP localhost:49218->localhost:6379 (ESTABLISHED)
node      667384 lmimmfn  68u  IPv4 197438231      0t0  TCP localhost:49232->localhost:6379 (ESTABLISHED)
node      667384 lmimmfn  69u  IPv4 197438232      0t0  TCP localhost:49236->localhost:6379 (ESTABLISHED)
node      667384 lmimmfn  70u  IPv6 197438874      0t0  TCP *:8585 (LISTEN)
node      667384 lmimmfn  71u  IPv4 197438875      0t0  TCP localhost:45638->localhost:50051 (ESTABLISHED)
node      667384 lmimmfn  72u  IPv4 197438879      0t0  TCP localhost:48272->localhost:27017 (ESTABLISHED)
node      667384 lmimmfn  73u  IPv4 197438884      0t0  TCP localhost:48282->localhost:27017 (ESTABLISHED)
node      667384 lmimmfn  74u  IPv6 197441725      0t0  TCP lmimmfn-dev:8585->100.120.44.137:52915 (ESTABLISHED)
node      667384 lmimmfn  76u  IPv6 197441727      0t0  TCP lmimmfn-dev:8585->100.120.44.137:52916 (ESTABLISHED)
node      667384 lmimmfn  77u  IPv6 197441728      0t0  TCP lmimmfn-dev:8585->100.120.44.137:52917 (ESTABLISHED)
node      667384 lmimmfn  78u  IPv6 197441730      0t0  TCP lmimmfn-dev:8585->100.120.44.137:52919 (ESTABLISHED)
node      667384 lmimmfn  79u  IPv6 197441729      0t0  TCP lmimmfn-dev:8585->100.120.44.137:52918 (ESTABLISHED)
node      667384 lmimmfn  80u  IPv6 197441731      0t0  TCP lmimmfn-dev:8585->100.120.44.137:52920 (ESTABLISHED)
node      667384 lmimmfn  81u  IPv6 197441732      0t0  TCP lmimmfn-dev:8585->100.120.44.137:52933 (ESTABLISHED)

Here i have web server ruinning on port 8585 and i have sessions open from 100.120.44.137, so you probably have the server running on port 443 so you can just pipe to grep for just those connections:
lsof -i | grep ":443->"

If you want just the connections you can | it to awk as connections are 9th column:
lsof -i | awk '{print $9}'

If you want just the list of connected incomming ip's you can use:
lsof -i | awk '{print $9}' | grep ":443->" | sed 's/.*->//'

Heres sample output filtering from the above example and on port 8585
lsof -i  | awk '{print $9}' | grep ":8585->" | sed 's/.*->//'
100.120.44.137:52915
100.120.44.137:52916
100.120.44.137:52917
100.120.44.137:52919
100.120.44.137:52918
100.120.44.137:52920
100.120.44.137:52933

You could crontab the above(running say every 5 minutes) to concatanate(use >> instead for > for concatenation) to a file e.g.(and dont use /tmp for your server, put it somewhere else) /tmp/ip_addresses.log
lsof -i  | awk '{print $9}' | grep ":443->" | sed 's/.*->//' >/tmp/ip_addresses.log

to add to crontab:
crontab -e
will open the crontab file for editing, add the line:
*/5 * * * * lsof -i  | awk '{print $9}' | grep ":443->" | sed 's/.*->//' >/tmp/ip_addresses.log
save it( escape then ":" then "wq"

Then to get the unique ones just :
cat /tmp/ip_addresses.log | sort | uniq

you can also aggregate the count of connections from the same ipaddress with:
cat /tmp/ip_addresses.log | sort | uniq -c

[Gryzor]

I *think* lsof was the first I tried, but the output format was not readable if directed to a file. But will try it again to double check, thanks! 

[genesis8]

I am using a debian 12 VM on OVH.

For just stats of Apache visitors, I am using Awstats

I am also using Fail2ban with the following personal rules to ban :

- too many 404 errors (trying to use known web pages of wordpress by example)
- SSH access not in a few authorized IP ranges (even if only one user is authorized for SSH access)
- ignoring error from my IP ranges, because I already got banned from my own rules :-)

Access to phpmyadmin is limited in Apache configuration with the same IP ranges authorized (my IP provider from home and for the phone)

[lmimmfn]

@genisis8 - i dont think its a good idea to post the types of security rules of your public server even if they are higher level, better to just PM the info.