On Sun 06 Jul 2025 at 22:55:22 (-0400), Rick Macdonald wrote:
After running Debian for nearly 30 years (and other distros prior to that), my Linux server has been hit by a ransomware attack about 11 days ago.
I have backups, so nothing important has been lost at this point.
However, I can't figure out how it got in, how it works, if there are executables on my computer that need to be cleaned, etc.
After running Debian for nearly 30 years (and other distros prior to
that), my Linux server has been hit by a ransomware attack about 11
days ago.
I don't know the attack method, but I'd suspect smb first
On Sun, Jul 06, 2025 at 08:47:22PM -0600, Rick Macdonald wrote:
After running Debian for nearly 30 years (and other distros prior to
that), my Linux server has been hit by a ransomware attack about 11 days ago.
Another machine running firewall sofware is cheap [...]
insurance.
I apologize for the length of this question.
After running Debian for nearly 30 years (and other distros prior to
that), my Linux server has been hit by a ransomware attack about 11
days ago. I have backups, so nothing important has been lost at this
point. However, I can't figure out how it got in, how it works, if
On Sun 06 Jul 2025 at 22:55:22 (-0400), Rick Macdonald wrote:
After running Debian for nearly 30 years (and other distros prior to that), >> my Linux server has been hit by a ransomware attack about 11 days ago.
I have backups, so nothing important has been lost at this point.
That's the most important thing.
However, I can't figure out how it got in, how it works, if there are
executables on my computer that need to be cleaned, etc.
You should consider the entire system compromised beyond repair. Nuke and
pave -- do a complete reinstall from scratch, restore from a known good
backup, and re-enable services one at a time.
I apologize for the length of this question.
After running Debian for nearly 30 years (and other distros prior to
that), my Linux server has been hit by a ransomware attack about 11 days
ago.
I have backups, so nothing important has been lost at this point.
However, I can't figure out how it got in, how it works, if there are executables on my computer that need to be cleaned, etc. I believe I
have been able to stop the attack, by simply fixing permissions on directories and files.
However, that obviously doesn't remove or block
the attack from my machine.
When I search for this malware on the web, I find Windows-specific discussions. If I'm unable to learn what to do from the folks here, suggestions about where to go for information and help would be most
welcome.
Here's what I have observed and done, which might have some clues:
- I first noticed it because of the rattling of the hard drive and the
hard drive activity light on solid.
- Looking at iotop and top, I expected to see some process pegging the
CPU and the disk I/O, but nothing seemed to stand out. I may have seen a Chromium thread doing a lot of I/O, but not for long.
- I unplugged the network ethernet cable and it stopped. Later that day,
I reconnected it and it started up again, but it seemed like it wasn't
until an hour or three later. Then I unplugged it again.
- At first I thought it was related to my media servers, Plex and Kodi, because the only files that I found to be encrypted were videos, audio
files, and image files. Then I found 1 encrypted file that was
different: my procmail rules file. This lead me to notice that all of
the encrypted files had "other" write permissions (666, 777). These were pretty much all old files from various sources. For example, photos from
up to 20 years ago from other people's cameras, etc.
- Because I suspected Kodi, I powered off the 3 android boxes I have in
the house that run Kodi to access my server (using MariaDB and smb). I haven't yet turned on any of these boxes again.
- The attack left a text file in every directory where it encrypted
files, with the name "5a067ee9_3a53aaff_1aedfa64___READ_THIS___5a067ee9_3a53aaff_1aedfa64.txt", with owner/group "nobody/nogroup". I've quoted the ransom file text below.
- No files outside of my home directory have been touched. I believe
that only files writable by "other" were encrypted. After encryption,
the files have a timestamp of the time of encryption, and are still
owned by me. The encrypted files have names like "0H1JsqXEw5.fse_5a067ee9_3a53aaff_1aedfa64", where the characters after
the dot (the extension, so to speak) are always the same.
- I have found and changed the permissions of every file and directory (except for /tmp) writable by "other". When I connect the ethernet
network cable now, there seems to be no further encrypting by the
malware. I check this by the lack of disk activity, and using the find command to search for files newer than the time I last connected to the network, I run "updatedb" and "locate" for filenames containing
"READ_THIS" and "fse_". I disconnect the network overnight though, just
in case.
- I eventually realized that some files that appeared to be encrypted
had not been renamed. I don't know what to think about this, other than
maybe the malware program doesn't rename file until a directory is
completed, and I disconnected the network cable while it appeared to be active.
- During all this, there was a power outage. After that, one Windows PC
that belonged to my mother has not been powered back on. I think I've
read this such malware can jump from Windows to Linux.
Some thoughts:
I read that files created by NFS or smb can be owned by nobody/nogroup.
The 2 running process owned by nobody are /usr/bin/memcached and /usr/ sbin/smbd. The remote kodi boxes access the server files using smb.
I don't know what it means that only files owned by me have been hit,
but only files with 777/666 permissions. Given that the new files are
created by nobody, it seems like they aren't able to actually log into
my account?
The ransomeware notification file:
The main point is to find out which system was hit.
According to the description it looks like the Linux server itself
wasn't hit, but a different system that can access files on the server
via network...
After running Debian for nearly 30 years (and other distros prior to
that), my Linux server has been hit by a ransomware attack about 11
days ago.
Please boot live media in the server, open a root terminal, mount the
server file systems under /mnt/server/, run the following commands (I
have assumed your username is "rick"; please substitute the correct
name), and copy/paste the console session into your reply. Document any changes that you have made since the attack:
On Mon, Jul 07, 2025 at 09:44:11PM +0200, Detlef Vollmann wrote:
[...]
The main point is to find out which system was hit.
According to the description it looks like the Linux server itself
wasn't hit, but a different system that can access files on the
server via network...
Yes. The guess put forward elsewhere in this thread that it was
perhaps a Windows client over Samba is pretty compelling. Especially
the observation that only world-writable files were hit is a finger
pointing in this direction.
In this case, a perimeter firewall will not help.
You likely got compromised by downloading something from the internet
or via e-mail.
On Mon, Jul 07, 2025 at 07:17:36AM +0200, john doe wrote:
In this case, a perimeter firewall will not help.
You likely got compromised by downloading something from the internet or via e-mail.
That is unlikely if the generated files were owned by nobody rather than the user.
On Mon, Jul 07, 2025 at 09:44:11PM +0200, Detlef Vollmann wrote:
[...]
The main point is to find out which system was hit.Yes. The guess put forward elsewhere in this thread that it was perhaps
According to the description it looks like the Linux server itself
wasn't hit, but a different system that can access files on the server
via network...
a Windows client over Samba is pretty compelling. Especially the observation that only world-writable files were hit is a finger pointing in this direction.
On 7/9/25 1:39 PM, Rick Macdonald wrote:
...
I checked, and sure enough, smb.conf had world-writeable permissions.
I've seen where some Kodi web pages suggest this. I've had it this
way for many years, but now I have made it read-only.
In samba logs you might be able to see which hosts did what and when
on which shares.
I t seems something is opening every file in my Media share:
I had a question that I forgot to add to my initial long post. This was
since "top" didn't show any great CPU usage, could the encryption have
been performed on another machine (Windows or one of my 3 Android Kodi boxes)? A number of you suggested exactly this.
I checked, and sure enough, smb.conf had world-writeable permissions.
I've seen where some Kodi web pages suggest this. I've had it this way
for many years, but now I have made it read-only.
So far, I booted up the Windows machine. I don't see any sign of an
attack on it. This is my mother's PC. She passed away at age 100 a year
ago. The PC is on and connected to the network, but I don't do much on it.
I also booted up 1 of my 3 Android Kodi boxes. No new attacks on my
Linux server. I'll look at the other 2 next.
The only Kodi addon I remember updating recently is opentitles, which
seems to have switched from opentitles.org to opentitles.com.
On 7/9/25 10:39, Rick Macdonald wrote:
I had a question that I forgot to add to my initial long post. This
was since "top" didn't show any great CPU usage, could the encryption
have been performed on another machine (Windows or one of my 3
Android Kodi boxes)? A number of you suggested exactly this.
If you want to identify the source of the attack, one idea is to put
the server on an isolated network segment, restore it to the
configuration it had when the attacks occurred, and wait to see if the attacks resume. If so, find the source. If not, add a suspect
computer to the isolated network segment and repeat.
If you want to remove malware from the Windows computer, run Windows
Update, run a Windows Defender full scan, and run a Windows Defender
offline scan.
On 2025-07-09 18:43, David Christensen wrote:
On 7/9/25 10:39, Rick Macdonald wrote:
I had a question that I forgot to add to my initial long post. This
was since "top" didn't show any great CPU usage, could the encryption
have been performed on another machine (Windows or one of my 3
Android Kodi boxes)? A number of you suggested exactly this.
If you want to identify the source of the attack, one idea is to put
the server on an isolated network segment, restore it to the
configuration it had when the attacks occurred, and wait to see if the
attacks resume. If so, find the source. If not, add a suspect
computer to the isolated network segment and repeat.
In 30 years I've never seen an isolated network. May I ask how this
might be done?
If you want to remove malware from the Windows computer, run Windows
Update, run a Windows Defender full scan, and run a Windows Defender
offline scan.
Will do, thanks.
On 7/9/25 22:14, Rick Macdonald wrote:
In 30 years I've never seen an isolated network. May I ask how this
might be done?
Assuming an Internet gateway with 4 LAN ports and Wi-Fi, and a server with 1 LAN port, turn off everything except the gateway, connect the server LAN
port to a gateway a LAN port (via switches, if needed), and boot the server. Add wired hosts by connecting their LAN port to a gateway LAN port (via switches, if needed). Add Wi-Fi hosts by booting them.
On Wed, Jul 09, 2025 at 23:23:29 -0700, David Christensen wrote:
On 7/9/25 22:14, Rick Macdonald wrote:
In 30 years I've never seen an isolated network. May I ask how
this might be done?
Assuming an Internet gateway with 4 LAN ports and Wi-Fi, and a
server with 1 LAN port, turn off everything except the gateway,
connect the server LAN port to a gateway a LAN port (via switches,
if needed), and boot the server. Add wired hosts by connecting
their LAN port to a gateway LAN port (via switches, if needed).
Add Wi-Fi hosts by booting them.
An alternative example (with no Wi-Fi):
* One switch or hub. Connect to power.
* Two or more computers. Connect to the switch/hub, and to power.
* On each computer, set an appropriate address manually, so they can
talk to each other.
Use any non-routable IP addressing you like. 192.168.1.* is a common
choice.
As an even simpler example, if you only have *two* computers, you can
connect them directly to each other, without needing a switch/hub.
Back in the olden days (before gigabit ethernet adapters), you would
have needed a special crossover ethernet cable for this. Now, on
modern devices, you should be able to use a regular ethernet cable.
On Wed, Jul 09, 2025 at 23:23:29 -0700, David Christensen wrote:
On 7/9/25 22:14, Rick Macdonald wrote:An alternative example (with no Wi-Fi):
In 30 years I've never seen an isolated network. May I ask how thisAssuming an Internet gateway with 4 LAN ports and Wi-Fi, and a server with 1 >> LAN port, turn off everything except the gateway, connect the server LAN
might be done?
port to a gateway a LAN port (via switches, if needed), and boot the server. >> Add wired hosts by connecting their LAN port to a gateway LAN port (via
switches, if needed). Add Wi-Fi hosts by booting them.
* One switch or hub. Connect to power.
* Two or more computers. Connect to the switch/hub, and to power.
* On each computer, set an appropriate address manually, so they can
talk to each other.
Use any non-routable IP addressing you like. 192.168.1.* is a common
choice.
As an even simpler example, if you only have *two* computers, you can
connect them directly to each other, without needing a switch/hub.
Back in the olden days (before gigabit ethernet adapters), you would
have needed a special crossover ethernet cable for this. Now, on modern devices, you should be able to use a regular ethernet cable.
<br>Thanks to all for the continuing help.</div><div dir="auto"><br>Rick</div></body></html>------BGSVE3DJF323JNLV9NRSW5093NNBRG--
On 2025-07-10 04:57, Greg Wooledge wrote:
On 7/9/25 22:14, Rick Macdonald wrote:An alternative example (with no Wi-Fi):
In 30 years I've never seen an isolated network. May I ask how this
might be done?
* One switch or hub. Connect to power.
* Two or more computers. Connect to the switch/hub, and to power.
* On each computer, set an appropriate address manually, so they can
talk to each other.
Use any non-routable IP addressing you like. 192.168.1.* is a common
choice.
OK, this I can understand and do. I had actually typed up this as a "guess", but then erased it thinking it might need something more complicated.
As an even simpler example, if you only have *two* computers, you can
connect them directly to each other, without needing a switch/hub.
Back in the olden days (before gigabit ethernet adapters), you would
have needed a special crossover ethernet cable for this. Now, on modern
devices, you should be able to use a regular ethernet cable.
I have an extra switch siting on the shelf, so I'm good.
Thanks to all for the continuing help.
Rick
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 546 |
Nodes: | 16 (2 / 14) |
Uptime: | 151:47:52 |
Calls: | 10,383 |
Files: | 14,054 |
Messages: | 6,417,815 |