It seems I accidentaly retrace03.rdu-cc.fedoraproject.org. It's been refusing connections for about 15 minutes now. I was debugging an issue with GDB dumping core in containers and I think I accidentally rebooted the server thinking it was staging.
~~For what it's worth, we had retrace stuck before due to misconfiguration.~~ (Actually, that was staging.)
As soon as viable.
I have put an outage note on status.fp.o https://status.fedoraproject.org/
I can reach the mgmt console but can't get shell access to the server. The server is up but unreachable over ssh
Metadata Update from @mobrien: - Issue tagged with: outage
I think at this point, the server will need another reboot and if that fails to bring up its network, then it will need to be booted into single user to see if disks are full/dead/etc.
It looks like it's interfaces moved around so it didn't bring any up. Looking more now.
So yeah, there was a ifcfg-eth1, but it had eth2 mac address in it.
I did:
mv ifcfg-eth1 ifcfg-eth2 vi ifcfg-eth2 (and changed all eth1 to eth2) nmcli c reload
and it came up. It doesn't look like this config is in ansible, so not sure why it changed.
Anyhow, it's back up now.
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Thanks for your investigation and the fix.