freeipa

#7743 Create automation to ensure that all integration tests are executed
Opened by rcritten. Modified

Open
Close Issue

Request for enhancement

As developer, I want all integration tests executed at some point (nightly, gating, etc) so that I can ensure the quality of a release.

Some integration tests have been written but not added to nightly or some other pr-ci group of tests which means they never get executed.

A quickie search found these not executed:

test_dns_locations.py
test_customized_ds_config_install.py
test_idviews.py
test_user_permissions.py
test_legacy_clients.py

And test_trust.py but we don't have AD generally available for PR-CI so that makes sense for now.

There are a lot of tests only executed in gating. I'm not sure if that is expected or desired.

There is a whole bunch of gating tests that are missing in nightly. It's certainly unexpected for me. I was under the impression that nightly tests are a superset of gating tests.

  • test_integration/test_simple_replication.py
  • test_integration/test_external_ca.py
  • test_integration/test_sudo.py
  • test_integration/test_commands.py
  • test_integration/test_kerberos_flags.py
  • test_integration/test_http_kdc_proxy.py
  • test_integration/test_forced_client_reenrollment.py
  • test_integration/test_advise.py
  • test_integration/test_testconfig.py
  • test_integration/test_service_permissions.py
  • test_integration/test_netgroup.py
  • test_integration/test_vault.py
  • test_integration/test_authselect.py

Note, test_idviews also needs AD.

Is test_legacy_clients.py still useful? The tests are from 2013 and check compatibility with even older legacy systems.
Edited by cheimes

PS: test_customized_ds_config_install and test_dns_locations are stable and still working, https://github.com/freeipa/freeipa/pull/2500

Good effort. Sometime in March or so I had created an ugly script which helps with finding the missing tests, let me dig that up.

Not running gating in nightlies is known fact. I don't remember when it was discussed last time, but the general assumption was: gating is run several times a day, last merged item ran it. This it is not necessary to run in nightlies as it was basically run.

Adding the tool output what is the situation today

Nightly and Gating overlap:
test_integration/test_caless.py::TestServerReplicaCALessToCAFull::test_install_caless_server_replica
test_integration/test_caless.py::TestServerReplicaCALessToCAFull::test_replica_ipa_ca_install
test_integration/test_caless.py::TestServerReplicaCALessToCAFull::test_server_ipa_ca_install
test_integration/test_dnssec.py::TestInstallDNSSECFirst::test_chain_of_trust
test_integration/test_dnssec.py::TestInstallDNSSECFirst::test_sign_root_zone
test_integration/test_replica_promotion.py::TestSubCAkeyReplication::test_sign_with_subca_on_replica
test_integration/test_replica_promotion.py::TestSubCAkeyReplication::test_subca_master
test_integration/test_replica_promotion.py::TestSubCAkeyReplication::test_subca_replica
Tests not in CI
test_integration/test_backup_and_restore.py::TestBackupAndRestoreDMPassword::test_restore_bad_dm_password
test_integration/test_backup_and_restore.py::TestBackupAndRestoreDMPassword::test_restore_dirsrv_not_running
test_integration/test_backup_and_restore.py::TestBackupAndRestoreWithReplica::test_full_backup_and_restore_with_replica
test_integration/test_backup_and_restore.py::TestReplicaInstallAfterRestore::test_replica_install_after_restore
test_integration/test_caless.py::TestReplicaCALessToCAFull::test_install_caless_server_replica
test_integration/test_caless.py::TestReplicaCALessToCAFull::test_replica_ipa_ca_install
test_integration/test_caless.py::TestServerCALessToExternalCA::test_install_caless_server
test_integration/test_caless.py::TestServerCALessToExternalCA::test_server_ipa_ca_install_external
test_integration/test_customized_ds_config_install.py::TestCustomInstallMaster::test_customized_ds_install_master
test_integration/test_customized_ds_config_install.py::TestCustomInstallReplica::test_customized_ds_install_replica
test_integration/test_dns_locations.py::TestDNSLocations::test_adtrust_system_records
test_integration/test_dns_locations.py::TestDNSLocations::test_all_servers_in_location
test_integration/test_dns_locations.py::TestDNSLocations::test_change_weight
test_integration/test_dns_locations.py::TestDNSLocations::test_ipa_ca_records
test_integration/test_dns_locations.py::TestDNSLocations::test_nsupdate_without_locations
test_integration/test_dns_locations.py::TestDNSLocations::test_one_replica_in_location
test_integration/test_dns_locations.py::TestDNSLocations::test_remove_replica_with_ca
test_integration/test_dns_locations.py::TestDNSLocations::test_restore_locations_and_weight
test_integration/test_dns_locations.py::TestDNSLocations::test_two_replicas_in_location
test_integration/test_dns_locations.py::TestDNSLocations::test_without_locations
test_integration/test_external_ca.py::TestExternalCAInstall::test_install_external_ca
test_integration/test_external_ca.py::TestExternalCAInstall::test_install_master
test_integration/test_external_ca.py::TestExternalCAInvalidCert::test_external_ca
test_integration/test_external_ca.py::TestExternalCAdirsrvStop::test_external_ca_dirsrv_stop
test_integration/test_external_ca.py::TestMultipleExternalCA::test_master_install_ca1
test_integration/test_external_ca.py::TestMultipleExternalCA::test_master_install_ca2
test_integration/test_idviews.py::TestCertsInIDOverrides::test_certs_in_idoverrides_ad_users
test_integration/test_idviews.py::TestCertsInIDOverrides::test_dbus_user_lookup
test_integration/test_idviews.py::TestRulesWithServicePrincipals::test_rules_with_service_principals
test_integration/test_installation.py::TestADTrustInstall::test_replica0_only_adtrust
test_integration/test_installation.py::TestADTrustInstall::test_replica1_all_components_adtrust
test_integration/test_installation.py::TestADTrustInstallWithDNS_KRA_ADTrust::test_replica0_only_adtrust
test_integration/test_installation.py::TestADTrustInstallWithDNS_KRA_ADTrust::test_replica1_all_components_adtrust
test_integration/test_installation.py::TestInstallWithCA_DNS3::test_number_of_zones
test_integration/test_installation.py::TestInstallWithCA_DNS4::test_number_of_zones
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_all_trustdomains_found
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_apply_advice
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_establish_trust
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_getent_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_getent_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_getent_ipa_group
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_getent_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_getent_subdomain_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_getent_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_getent_treedomain_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_getent_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_id_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_id_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_id_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_id_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_ipauser_authentication
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_login_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_login_disabled_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_login_disabled_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_login_disabled_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_login_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_login_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_login_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_range_properties_in_nonposix_trust
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_remove_nonposix_trust
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatNonPosix::test_user_gid_uid_resolution_in_nonposix_trust
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_all_trustdomains_found
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_apply_advice
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_establish_trust
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_getent_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_getent_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_getent_ipa_group
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_getent_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_getent_subdomain_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_getent_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_getent_treedomain_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_getent_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_id_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_id_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_id_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_id_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_login_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_login_disabled_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_login_disabled_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_login_disabled_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_login_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_login_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_login_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_range_properties_in_posix_trust
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_remove_trust_with_posix_attributes
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_user_uid_gid_resolution_in_posix_trust
test_integration/test_legacy_clients.py::TestLegacyNssLdapRedHatPosix::test_user_without_posix_attributes_not_visible
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_all_trustdomains_found
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_apply_advice
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_establish_trust
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_getent_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_getent_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_getent_ipa_group
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_getent_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_getent_subdomain_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_getent_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_getent_treedomain_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_getent_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_id_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_id_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_id_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_id_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_ipauser_authentication
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_login_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_login_disabled_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_login_disabled_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_login_disabled_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_login_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_login_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_login_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_range_properties_in_nonposix_trust
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_remove_nonposix_trust
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatNonPosix::test_user_gid_uid_resolution_in_nonposix_trust
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_all_trustdomains_found
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_apply_advice
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_establish_trust
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_getent_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_getent_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_getent_ipa_group
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_getent_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_getent_subdomain_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_getent_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_getent_treedomain_ad_group
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_getent_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_id_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_id_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_id_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_id_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_login_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_login_disabled_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_login_disabled_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_login_disabled_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_login_ipa_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_login_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_login_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_range_properties_in_posix_trust
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_remove_trust_with_posix_attributes
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_user_uid_gid_resolution_in_posix_trust
test_integration/test_legacy_clients.py::TestLegacyNssPamLdapdRedHatPosix::test_user_without_posix_attributes_not_visible
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_all_trustdomains_found
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_apply_advice
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_establish_trust
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_getent_ad_group
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_getent_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_getent_ipa_group
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_getent_ipa_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_getent_subdomain_ad_group
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_getent_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_getent_treedomain_ad_group
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_getent_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_id_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_id_ipa_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_id_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_id_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_ipauser_authentication
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_login_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_login_disabled_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_login_disabled_ipa_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_login_disabled_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_login_ipa_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_login_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_login_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_range_properties_in_nonposix_trust
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_remove_nonposix_trust
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatNonPosix::test_user_gid_uid_resolution_in_nonposix_trust
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_all_trustdomains_found
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_apply_advice
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_establish_trust
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_getent_ad_group
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_getent_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_getent_ipa_group
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_getent_ipa_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_getent_subdomain_ad_group
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_getent_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_getent_treedomain_ad_group
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_getent_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_id_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_id_ipa_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_id_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_id_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_login_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_login_disabled_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_login_disabled_ipa_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_login_disabled_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_login_ipa_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_login_subdomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_login_treedomain_ad_user
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_range_properties_in_posix_trust
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_remove_trust_with_posix_attributes
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_user_uid_gid_resolution_in_posix_trust
test_integration/test_legacy_clients.py::TestLegacySSSDBefore19RedHatPosix::test_user_without_posix_attributes_not_visible
test_integration/test_ordering.py::TestBase::test_c_second
test_integration/test_ordering.py::TestBase::test_d_first
test_integration/test_ordering.py::TestChild::test_a_fourth
test_integration/test_ordering.py::TestChild::test_b_third
test_integration/test_ordering.py::TestChild::test_c_second
test_integration/test_ordering.py::TestChild::test_d_first
test_integration/test_replica_promotion.py::TestReplicaInstallCustodia::test_replica_install_for_custodia
test_integration/test_replication_layouts.py::Test2ConnectedTopologyWithCA::test_2_connected_topology_with_ca
test_integration/test_replication_layouts.py::Test2ConnectedTopologyWithCAKRA::test_2_connected_topology_with_ca_kra
test_integration/test_replication_layouts.py::Test2ConnectedTopologyWithoutCA::test_2_connected_topology_without_ca
test_integration/test_replication_layouts.py::TestDoubleCircleTopologyWithCA::test_2_connected_topology_with_ca
test_integration/test_replication_layouts.py::TestDoubleCircleTopologyWithCAKRA::test_2_connected_topology_with_ca_kra
test_integration/test_replication_layouts.py::TestDoubleCircleTopologyWithoutCA::test_2_connected_topology_with_ca
test_integration/test_trust.py::TestBasicADTrust::test_all_trustdomains_found
test_integration/test_trust.py::TestBasicADTrust::test_establish_trust
test_integration/test_trust.py::TestBasicADTrust::test_ipauser_authentication
test_integration/test_trust.py::TestBasicADTrust::test_range_properties_in_nonposix_trust
test_integration/test_trust.py::TestBasicADTrust::test_remove_nonposix_trust
test_integration/test_trust.py::TestBasicADTrust::test_user_gid_uid_resolution_in_nonposix_trust
test_integration/test_trust.py::TestEnforcedPosixADTrust::test_all_trustdomains_found
test_integration/test_trust.py::TestEnforcedPosixADTrust::test_establish_trust
test_integration/test_trust.py::TestEnforcedPosixADTrust::test_range_properties_in_posix_trust
test_integration/test_trust.py::TestEnforcedPosixADTrust::test_remove_trust_with_posix_attributes
test_integration/test_trust.py::TestEnforcedPosixADTrust::test_user_uid_gid_resolution_in_posix_trust
test_integration/test_trust.py::TestEnforcedPosixADTrust::test_user_without_posix_attributes_not_visible
test_integration/test_trust.py::TestExternalTrustWithRootDomain::test_all_trustdomains_found
test_integration/test_trust.py::TestExternalTrustWithRootDomain::test_establish_trust
test_integration/test_trust.py::TestExternalTrustWithRootDomain::test_remove_nonposix_trust
test_integration/test_trust.py::TestExternalTrustWithSubdomain::test_all_trustdomains_found
test_integration/test_trust.py::TestExternalTrustWithSubdomain::test_establish_trust
test_integration/test_trust.py::TestExternalTrustWithSubdomain::test_remove_nonposix_trust
test_integration/test_trust.py::TestExternalTrustWithSubdomain::test_user_gid_uid_resolution_in_nonposix_trust
test_integration/test_trust.py::TestExternalTrustWithTreedomain::test_all_trustdomains_found
test_integration/test_trust.py::TestExternalTrustWithTreedomain::test_establish_trust
test_integration/test_trust.py::TestExternalTrustWithTreedomain::test_remove_nonposix_trust
test_integration/test_trust.py::TestExternalTrustWithTreedomain::test_user_gid_uid_resolution_in_nonposix_trust
test_integration/test_trust.py::TestInvalidRangeTypes::test_all_trustdomains_found
test_integration/test_trust.py::TestInvalidRangeTypes::test_establish_trust
test_integration/test_trust.py::TestInvalidRangeTypes::test_invalid_range_types
test_integration/test_trust.py::TestNonexternalTrustWithSubdomain::test_all_trustdomains_found
test_integration/test_trust.py::TestNonexternalTrustWithSubdomain::test_establish_trust
test_integration/test_trust.py::TestNonexternalTrustWithTreedomain::test_all_trustdomains_found
test_integration/test_trust.py::TestNonexternalTrustWithTreedomain::test_establish_trust
test_integration/test_trust.py::TestPosixADTrust::test_all_trustdomains_found
test_integration/test_trust.py::TestPosixADTrust::test_establish_trust
test_integration/test_trust.py::TestPosixADTrust::test_range_properties_in_posix_trust
test_integration/test_trust.py::TestPosixADTrust::test_remove_trust_with_posix_attributes
test_integration/test_trust.py::TestPosixADTrust::test_user_uid_gid_resolution_in_posix_trust
test_integration/test_trust.py::TestPosixADTrust::test_user_without_posix_attributes_not_visible
test_integration/test_trust.py::TestTrustWithUPN::test_all_trustdomains_found
test_integration/test_trust.py::TestTrustWithUPN::test_establish_trust
test_integration/test_trust.py::TestTrustWithUPN::test_remove_nonposix_trust
test_integration/test_trust.py::TestTrustWithUPN::test_upn_in_nonposix_trust
test_integration/test_trust.py::TestTrustWithUPN::test_upn_user_authentication
test_integration/test_trust.py::TestTrustWithUPN::test_upn_user_resolution_in_nonposix_trust
test_integration/test_user_permissions.py::TestInstallClientNoAdmin::test_installclient_as_user_admin
test_integration/test_user_permissions.py::TestUserPermissions::test_delete_preserve_as_alternate_admin
test_integration/test_user_permissions.py::TestUserPermissions::test_stageuser_show_as_alternate_admin

ipa-check-included-tests.py

It makes sense to me not running gating tests in nightly, we just need to be consistent.

The main ask of this ticket is to prevent missing tests in the future. A new introspection test should be added that looks at the PR to see if it is adding new tests and ensure that it is added in one of the yaml config files.

Fixing the overlap and missing tests would be a nice bonus too.

master:

  • e64ae1d043fb00f04e1c8c77cd214a806ce308b0 Add missing tests to nighly runs
  • 4cede86673e33bd2e8c9b6dd5eec2ccdb98c3a1a Speed up test_customized_ds_config_install

master:

  • 60a31d3f0ebca54ae21edaeb86b8480ffafe6e23 Nightly tests: add test_user_permissions.py

master:

  • 1be415cd04a0fcadaa12ecf5dd2c34a7e741a504 ipatests: add missing tests for test_caless

master:

  • 14ad844bb164b4b83b8462fe6e8fa2273acef07f ipatests: add missing tests in test_backup_and_restore.py

master:

  • 1e7be6fb3d1b50e6d5f398ea385f4fec6fad6598 ipatests: add missing tests for test_external_ca.py
  • 19211257c64ac95d1b0cc0d83729383dea7a98d1 ipatests: add missing tests for test_installation.py
  • d18b0d558ba56f3da16c6240d46ed9809c37b256 ipatests: add missing tests for test_replica_promotion.py

Please, what is missing here for closing this ticket?

The ticket is not only about adding the missing tests, but also creating a check to ensure that all the existing tests are part of either gating or nightly. See Rob's comment above.
Metadata
None
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.