Beware Fedora updates of April 28, 2024

It breaks iptables with no warning

I’m rather angry right now but I promise to remain calm — at least in this post.

I just finished installing Fedora updates today and a couple packages relating to IPTables were installed. Actually the iptables packages were removed and iptables-legacy packages were installed. In combination with the merge of /sbin and /usr/sbin, and some other file location changes, all of my IPTables firewalls stopped working when I rebooted my firewall and server systems.

The problems manifested as inability to communicate with the outside world — and presumably for the outside world to be unable to send email to Both.org or access the Both.org website. It only took me a couple minutes to determine that the firewall was not running because I got an error message indicating that the iptables-save and iptables commands weren’t present.

WHAT!!??

The firewall is responsible for determining packet routing on the system I use as the firewall and router. Without the firewall rules, routing in or out doesn’t work. That’s one of the clues I used to determine the source of the problem. But that wasn’t the root cause.

A quick look at the /sbin directory showed me the following. This is what the iptables symlinks looked like after the morning updates. The links all contain the “-legacy” insertion so the systemd service unit couldn’t find the tools needed to start iptables. I did check /etc/sysconfig/iptables to ensure it was still there and unchanged — it was OK.

# cd /sbin
[root@yorktown sbin]# ll ipt*
lrwxrwxrwx. 1 root root     21 Apr 16 11:24 iptables-apply -> ../bin/iptables-apply
lrwxrwxrwx. 1 root root     22 Apr 16 11:24 iptables-legacy -> ../bin/iptables-legacy
lrwxrwxrwx. 1 root root     30 Apr 16 11:24 iptables-legacy-restore -> ../bin/iptables-legacy-restore
lrwxrwxrwx. 1 root root     27 Apr 16 11:24 iptables-legacy-save -> ../bin/iptables-legacy-save
lrwxrwxrwx. 1 root root     19 Apr 16 11:22 iptables-nft -> ../bin/iptables-nft
lrwxrwxrwx. 1 root root     27 Apr 16 11:22 iptables-nft-restore -> ../bin/iptables-nft-restore
lrwxrwxrwx. 1 root root     24 Apr 16 11:22 iptables-nft-save -> ../bin/iptables-nft-save
lrwxrwxrwx. 1 root root     33 Apr 16 11:22 iptables-restore-translate -> ../bin/iptables-restore-translate
lrwxrwxrwx. 1 root root     25 Apr 16 11:22 iptables-translate -> ../bin/iptables-translate
-rwxr-xr-x. 1 root root 203360 Jul 17  2024 iptraf-ng
lrwxrwxrwx. 1 root root     15 Apr 16 11:22 iptstate -> ../bin/iptstate
lrwxrwxrwx. 1 root root     15 Apr 16 11:24 iptunnel -> ../bin/iptunnel

Compare the above to this look at the previous set of links in /sbin. There’s quite a bit of difference.

# ll ipt*
lrwxrwxrwx. 1 root root    26 Apr  9 08:06 iptables -> /etc/alternatives/iptables
lrwxrwxrwx. 1 root root    17 Jan 16 19:00 iptables-nft -> xtables-nft-multi
lrwxrwxrwx. 1 root root    17 Jan 16 19:00 iptables-nft-restore -> xtables-nft-multi
lrwxrwxrwx. 1 root root    17 Jan 16 19:00 iptables-nft-save -> xtables-nft-multi
lrwxrwxrwx. 1 root root    34 Apr  9 08:06 iptables-restore -> /etc/alternatives/iptables-restore
lrwxrwxrwx. 1 root root    17 Jan 16 19:00 iptables-restore-translate -> xtables-nft-multi
lrwxrwxrwx. 1 root root    31 Apr  9 08:06 iptables-save -> /etc/alternatives/iptables-save
lrwxrwxrwx. 1 root root    17 Jan 16 19:00 iptables-translate -> xtables-nft-multi
-rwxr-xr-x. 1 root root 34248 Jan 16 19:00 iptc
-rwxr-xr-x. 1 root root 87456 Mar 17 20:00 iptstate
-rwxr-xr-x. 1 root root 24264 Jan 16 19:00 iptunnel

The fix is rather easy. I just used the following commands to create the links I needed.

# cd /sbin
# ln -s iptables-legacy iptables
# ln -s iptables-legacy-restore iptables-restore
# ln -s iptables-legacy-save iptables-save
# systemctl restart iptables.service

The new links are now available for the SysAdmin (me) and systemd to use. I also restarted fail2ban to ensure that all the blocking entries were added to the firewall. If you’re not using fail2ban you don’t need to do that.

# ll ipt*
lrwxrwxrwx. 1 root root     15 Apr 28 15:06 iptables -> iptables-legacy
lrwxrwxrwx. 1 root root     21 Apr 17 15:53 iptables-apply -> ../bin/iptables-apply
lrwxrwxrwx. 1 root root     22 Apr 17 15:53 iptables-legacy -> ../bin/iptables-legacy
lrwxrwxrwx. 1 root root     30 Apr 17 15:53 iptables-legacy-restore -> ../bin/iptables-legacy-restore
lrwxrwxrwx. 1 root root     27 Apr 17 15:53 iptables-legacy-save -> ../bin/iptables-legacy-save
lrwxrwxrwx. 1 root root     19 Apr 17 15:50 iptables-nft -> ../bin/iptables-nft
lrwxrwxrwx. 1 root root     27 Apr 17 15:50 iptables-nft-restore -> ../bin/iptables-nft-restore
lrwxrwxrwx. 1 root root     24 Apr 17 15:50 iptables-nft-save -> ../bin/iptables-nft-save
lrwxrwxrwx. 1 root root     23 Apr 28 15:06 iptables-restore -> iptables-legacy-restore
lrwxrwxrwx. 1 root root     33 Apr 17 15:50 iptables-restore-translate -> ../bin/iptables-restore-translate
lrwxrwxrwx. 1 root root     20 Apr 28 15:06 iptables-save -> iptables-legacy-save
lrwxrwxrwx. 1 root root     25 Apr 17 15:50 iptables-translate -> ../bin/iptables-translate
-rwxr-xr-x. 1 root root 203360 Jul 17  2024 iptraf-ng
lrwxrwxrwx. 1 root root     15 Apr 17 15:50 iptstate -> ../bin/iptstate
lrwxrwxrwx. 1 root root     15 Apr 17 15:51 iptunnel -> ../bin/iptunnel

A bit of testing verified that all was once again working as it should.

A short rant

I am furious that this occurred. There is absolutely no reason for it. Fortunately, my systems weren’t open without a firewall for very long, but it takes only a few microseconds to infect a computer that’s unprotected.

I much prefer IPTables for my use case and it’s easier and faster to work with than NFTables or firewalld, neither of which I like in the least. It’s not that they don’t work, but both are obscure and firewalld is way too complex — for my use cases.

Although IPTables is no longer the default firewall front-end, I never expected that it was going to be disabled by incompetence (the nicest word I can think of at the moment). Clearly it’s intended to continue as a legacy tool but apparently someone failed to perform simple tests to see if it would still work after the updates were installed. Changes like this should be transparent in their application but loud in the announcement of their coming and that they’ve been implemented.

And they should damn well be thoroughly tested.