Fedora Copr hypervisors

Fedora Copr hosts several hypervisors within the Fedora Infrastructure lab. These hypervisors are monitored using Nagios probes.

Running playbooks

Warning

Running playbooks:

$ ssh batcave01.iad2.fedoraproject.org
[yourname@batcave01 ~][PROD-IAD2]$ sudo rbac-playbook groups/copr-hypervisor.yml
...

Reboot

If a hypervisor becomes inconsistent, you may need to reboot it. The Resalloc server (starting VMs on the hypervisors) is capable of recovering from the reboot.

Warning

Please note that any running builds will be rescheduled by Copr Backend.

Typically, executing the same hypervisor’s playbook with the -t trigger_reboot option is sufficient. However, to avoid rebooting all hypervisors, you can limit the playbook for specific hosts using the -l pattern, e.g.:

one-box $ sudo rbac-playbook groups//copr-hypervisor.yml -t trigger_reboot -l 'vmhost-x86-copr02.rdu-cc.fedoraproject.org'
all-x86 $ sudo rbac-playbook groups//copr-hypervisor.yml -t trigger_reboot -l '*x86*'
...

Access to KVM and cold rebooting

Warning

  • ssh access to cloud-noc-os01.rdu-cc.fedoraproject.org is needed

  • access to the Team’s Bitwaarden account is needed

To access the management consoles of our hypervisors (which are only available within the local management network), you need to use the cloud-noc-os01.rdu-cc.fedoraproject.org hop-box. You can either use elinks over SSH, or set up a SOCKS proxy for local browsing:

$ ssh -ND 9999 cloud-noc-os01.rdu-cc.fedoraproject.org

Then, configure Firefox settings by searching for socks, opening Network Settings, and selecting Manual proxy configuration. Specify SOCKS Host: localhost, and Port: 9999. Afterward, you can visit the management console IP within the management network range, such as http://172.X.Y.Z.

The information regarding IPs and passwords for specific hosts can be found in the team’s Bitwaarden account. Refer to the Secret Note Fedora Copr Hypervisors.

Adding a new hypervisor

For instructions on adding a new hypervisor, please refer to the separate section on how to install hypervisors in the Fedora Infra ansible repository