From 274ec8dcf3d42281381940fe5a576da8bb708079 Mon Sep 17 00:00:00 2001 From: Tomas Hrcka Date: Oct 20 2023 07:25:37 +0000 Subject: [PATCH 1/2] [releng] Add RC compose page Signed-off-by: Tomas Hrcka --- diff --git a/modules/release_guide/pages/final_RC_compose.adoc b/modules/release_guide/pages/final_RC_compose.adoc index 58daad8..c19f238 100644 --- a/modules/release_guide/pages/final_RC_compose.adoc +++ b/modules/release_guide/pages/final_RC_compose.adoc @@ -2,3 +2,216 @@ include::_partials/attributes.adoc[] = Final Release candidate +When Quality Engineering (QE) requests a Release Candidate (RC) they do +so by opening an issue in the releng repository on pagure. Release +candidate composes are not currently automated. + +[cols=",,",options="header",] +|=== +|Compose Name |Configuration File |Compose Script +|GA |fedora-final.conf |release-candidate.sh +|=== + +== Action + +=== Make pungi config changes + +Create fedora-final.conf on the branched release branch, For inspiration look at https://pagure.io/pungi-fedora/pull-request/1148[This PR] + +make sure to set runroot_tag in general.conf to current branched +make sure all ostree releases have config_branch set to the branched release +make sure that if new artifacts are introduced in the release, they are part of the final config. +use grep to find any references to rawhide. + +=== Review Compose Tags + +. List any pre-existing builds in the current compose tag ++ +[source,subs="attributes+"] +.... +$ koji list-tagged f{branched}-compose +.... +. Verify pre-existing builds are in compose tags ++ +The tagged builds from the previous composes should all be present in +the output from the previous step. Consult the request ticket for the +list of builds expected in this output. ++ +[NOTE] +==== +The very first run of an GA compose should have no builds +listed under the compose tag. It is important to clear pre-existing +builds from the compose tag when moving between the Beta and RC +composes. Verify that these builds were removed. + +[source,subs="attributes+"] +.... +$ koji list-tagged f{branched}-compose +$ koji untag-build --all f{branched}-compose [build1 build2 ...] +.... +==== ++ +[NOTE] +==== +The order in which packages are added into the +f{branched}-compose tag matter. If the builds are untagged +erroneously then special attention should be given to adding them back +correctly. +==== +. Add builds specified by QE to the current compose tag ++ +[source,subs="attributes+"] +.... +$ koji tag-build f{branched}-compose [build1 build2 ...] +.... ++ +[NOTE] +==== +These steps may be completed on a local machine as long as the user has +appropriate permissions in the koji tool. +==== + +=== Running the Compose + +. Update the pungi-fedora config file Composes use a configuration file +to construct the compose. Each compose uses its own configuration. The +`global_release` variable should start from 1.1 and the second number +should increment each time a new compose is created. + +* GA - `fedora-final.conf` +. Log into the compose backend ++ +.... +$ ssh compose-x86-01.iad2.fedoraproject.org +.... +. Open a screen session ++ +.... +$ screen +.... +. Obtain the pungi-fedora branch for the current compose ++ +The first time any user account executes a compose the pungi-fedora git +repository must be cloned. The compose candidate script that invokes +pungi should be run from `compose-x86-01.iad2.fedoraproject.org`. ++ +.... +$ git clone ssh://git@pagure.io/pungi-fedora.git +.... ++ +Enter the pungi-fedora directory. ++ +.... +$ cd pungi-fedora +.... ++ +If the clone step above was not required then fully update the existing +repository checkout from pagure. ++ +[source,subs="attributes+"] +.... +$ git fetch origin +$ git checkout f{branched} +$ git pull origin f{branched} +.... +. Run the compose ++ +.... +$ sudo ./release-candidate.sh RC-#.# +.... ++ +The numbering scheme begins with 1.1 and the second number is +incremented after each compose. ++ +[NOTE] +==== +Pungi requires numbers in the format #.# as an argument. It is because +of this that composes always start with the number 1 and the second +number is incremented with each compose. +==== ++ +[NOTE] +==== +If the compose fails with a directory missing error, then create the +compose directory with `mkdir /mnt/koji/compose/{branched}` +==== + +=== Syncing the Compose + +We sync the compose to `/pub/alt/stage` to enable faster access to new +content for QA and the larger Fedora community. + +. Log into the compose backend ++ +.... +$ ssh compose-x86-01.iad2.fedoraproject.org +.... +. Open a screen session ++ +.... +$ screen +.... +. Check the status of the compose ++ +[source,subs="attributes+"] +.... +$ cat /mnt/koji/compose/{branched}/[compose_id]/STATUS +.... ++ +Do not continue with any further steps if the output above is `DOOMED`. +. Create the directory targeted for the copy : ++ +[source,subs="attributes+"] +.... +$ sudo -u ftpsync mkdir -m 750 -p /pub/alt/stage/{branched}_RC-[#.#] +.... +. Locate the compose directory that will be the copy source : ++ +[source,subs="attributes+"] +.... +$ ls /mnt/koji/compose/{branched}/[compose_id] +.... ++ +[NOTE] +==== +Take care executing the synchronization if the next compose is already +running. Be sure to grab the correct directory. + +If in doubt, check +/mnt/koji/compose/{branched}/[compose_id]/STATUS to be sure it is +finished. +==== + +. Run the synchronization one-liner ++ +The synchronization of the completed compose to the public domain is +currently a one-liner shell script. Pay close attention to what needs +replaced in the example below. ++ +[source,subs="attributes+"] +.... +$ sudo -u ftpsync sh -c 'for dir in Everything Cloud Container Kinoite Labs Modular Server Silverblue Spins Workstation metadata; do rsync -avhH /mnt/koji/compose/{branched}/Fedora-{branched}-20190911.0/compose/$dir/ /pub/alt/stage/{branched}_RC-1.1/$dir/ --link-dest=/pub/fedora/linux/development/{branched}/Everything/ --link-dest=/pub/alt/stage/{branched}_RC-1.1/Everything/; done' +.... ++ +[NOTE] +==== +If multiple composes are run like 1.2, 1.3, add multiple --link-dest +arguments above with multiple composes +==== +. Set the permissions of the synced compose : ++ +[source,subs="attributes+"] +.... +$ sudo -u ftpsync chmod 755 /pub/alt/stage/{branched}_RC-[#.#] +.... +. Update the issue in the releng pagure repository ++ +Once the compose and sync is complete the issue in pagure should be +updated and closed. + +=== Verification + +The method for verifying a compose has completed is checking +`/mnt/koji/compose/{branched}/[compose_dir]/STATUS`. Any status +other than DOOMED is OK. + From cdc052c49fe4d6d9d08eb9dc90b6db22a5cf29d4 Mon Sep 17 00:00:00 2001 From: Tomas Hrcka Date: Oct 20 2023 07:31:18 +0000 Subject: [PATCH 2/2] Cleanup unused pages Signed-off-by: Tomas Hrcka --- diff --git a/modules/release_guide/pages/sop_adding_build_targets.adoc b/modules/release_guide/pages/sop_adding_build_targets.adoc deleted file mode 100644 index fed2fa8..0000000 --- a/modules/release_guide/pages/sop_adding_build_targets.adoc +++ /dev/null @@ -1,179 +0,0 @@ -== Adding Build Targets - -=== Description - -Each new release we create a build target for the next release. This SOP -will describe the steps necessary to prepare the new build target. - -=== Action - -Adding a build target is a complicated task. It involves updating koji, -git, and fedora-release package. - -[[adding_build_targets_koji]] -==== Koji - -In koji a couple collection tags need to be made, and a target created -to tie them together. We create a base collection tag named after the -release, and a build tag to hold a few things we use in the buildroots -that are not part of the distribution (glibc32/glibc64). Inheritance to -the previous release is used for ownership and package data, as well as -buildroot content data. - -The `add-tag`, `add-tag-inheritance`, `edit-tag`, and `add-target` -commands are used. - -.... -$ koji add-tag --help -Usage: koji add-tag [options] name -(Specify the --help global option for a list of other help options) - -Options: --h, --help show this help message and exit ---parent=PARENT Specify parent ---arches=ARCHES Specify arches - - -$ koji add-tag-inheritance --help -Usage: koji add-tag-inheritance [options] tag parent-tag -(Specify the --help global option for a list of other help options) - -Options: --h, --help show this help message and exit ---priority=PRIORITY Specify priority ---maxdepth=MAXDEPTH Specify max depth ---intransitive Set intransitive ---noconfig Set to packages only ---pkg-filter=PKG_FILTER -Specify the package filter ---force=FORCE Force adding a parent to a tag that already has that -parent tag - -$ koji edit-tag --help -Usage: koji edit-tag [options] name -(Specify the --help global option for a list of other help options) - -Options: - -h, --help show this help message and exit - --arches=ARCHES Specify arches - --perm=PERM Specify permission requirement - --no-perm Remove permission requirement - --lock Lock the tag - --unlock Unlock the tag - --rename=RENAME Rename the tag - -$ koji add-target --help -Usage: koji add-target name build-tag -(Specify the --help global option for a list of other help options) - -Options: --h, --help show this help message and exit -.... - -For example if we wanted to create the Fedora 17 tags, we would do the -following: - -.... -koji add-tag --parent f16-updates f17 -koji add-tag --parent f17 f17-updates -koji add-tag --parent f17-updates f17-candidate -koji add-tag --parent f17-updates f17-updates-testing -koji add-tag --parent f17-updates-testing f17-updates-testing-pending -koji add-tag --parent f17-updates f17-updates-pending -koji add-tag --parent f17-updates f17-override -koji add-tag --parent f17-override --arches=i686,x86_64 f17-build -koji add-tag-inheritance --priority 1 f17-build f16-build -koji edit-tag --perm=fedora-override f17-override -koji edit-tag --lock f17-updates -koji add-target f17 f17-build -.... - -[NOTE] -==== -The `-pending` tags are used by -https://fedoraproject.org/wiki/Bodhi[Bodhi] and -https://fedoraproject.org/wiki/Taskotron[Taskotron] to track and test -proposed updates. These tags are not build targets and they don't get -made into repos, so proper inheritance isn't vital. -==== - -==== Git - -The pkgdb_sync_git_branches.py file which is hosted in Fedora -Infrastructure ansible -(roles/distgit/templates/pkgdb_sync_git_branches.py) needs to be updated -for the new target for making branches. - -Update `BRANCHES` with the new branch information. The branch name maps -to the branch that is its parent. - -.... -BRANCHES = {'el4': 'rawhide', 'el5': 'rawhide', 'el6': 'f12', - 'OLPC-2': 'f7', - 'rawhide': None, - 'fc6': 'rawhide', - 'f7': 'rawhide', - 'f8': 'rawhide', - 'f9': 'rawhide', - 'f10': 'rawhide', - 'f11': 'rawhide', - 'f12': 'rawhide', - 'f13': 'rawhide', 'f14': 'rawhide'} -.... - -and update `GITBRANCHES` with the translation from pkgdb branch string -to git branch string: - -.... -GITBRANCHES = {'EL-4': 'el4', 'EL-5': 'el5', 'EL-6': 'el6', 'OLPC-2': 'olpc2', - 'FC-6': 'fc6', 'F-7': 'f7', 'F-8': 'f8', 'F-9': 'f9', 'F-10': 'f10', - 'F-11': 'f11', 'F-12': 'f12', 'F-13': 'f13', 'F-14': 'f14', 'devel': 'rawhide'} -.... - -The genacls.pkgdb file also needs to be updated for active branches to -generate ACLs for. Update the `ACTIVE` variable. genacls.pkgdb lives in -puppet (modules/gitolite/files/distgit/genacls.pkgdb). The format is -pkgdb branch string to git branch string (until pkgdb uses git branch -strings): - -.... -ACTIVE = {'OLPC-2': 'olpc2/', 'EL-4': 'el4/', 'EL-5': 'el5/', - 'EL-6': 'el6/', 'F-11': 'f11/', 'F-12': 'f12/', 'F-13': 'f13/', - 'F-14': 'f14/', 'devel': 'rawhide'} -.... - -==== fedora-release - -Currently the fedora-release package provides the `%{?dist}` definitions -used during building of packages. When a new target is created, -fedora-release must be built for the collection with new dist -definitions. - -==== Comps - -* In the comps module in Fedora Hosted git -(ssh://git.fedorarhosted.org/git/comps.git), create and add a new comps -file based on the previous release. (Just copying it should be fine.) -Add the new file to `po/POTFILES.in`. -* When rawhide is retargeted in koji to point to the new release, update -the `Makefile` to target comps-rawhide.xml at the new version. -* Don't forget to `git push` your changes after committing. - -=== Verification - -Given the complexity of all the changes necessary to create a new build -target, the best way to verify is to attempt a build. Given that -fedora-release has to be built before anything else so that dist tags -translate correctly it is a good test case. For example, this was used -to test the new Fedora 15 target: - -* Use pkgdb to request an F-15 branch of fedora-release -* Use pkgdb2branch.py to actually make the branch -* Update fedora-release clone -* Adjust .spec file in rawhide for new dist defines -* commit/build -* Track build in koji to ensure proper tagging is used - -What this won't test is translations of dist at tag time given that -`fedora-release` doesn't use dist in it's Release. Verifying with a -second package that uses dist is a good idea. diff --git a/modules/release_guide/pages/sop_adding_content_generator.adoc b/modules/release_guide/pages/sop_adding_content_generator.adoc deleted file mode 100644 index 2563151..0000000 --- a/modules/release_guide/pages/sop_adding_content_generator.adoc +++ /dev/null @@ -1,58 +0,0 @@ -== Adding new koji content generator - -=== Description - -Koji added support for content generators some time ago. Basic premise -of content generators is that it lets us create build systems for new -types of content without affecting or changing core Koji code and in -some way simplify integration with rest of the release toolchain. More -information about content generators, background, requirements and more -can be found in Koji -https://docs.pagure.org/koji/content_generators/[content generator -documentation] - -For content generator to be able to create/import builds into Koji -following prerequisites have to be met: - -* Koji has to recognize the content generator type -* User doing the content generator import has to have permissions for -this action -* Any new content types have to be defined in Koji - -=== Questions to ask - -There are some questions that should be answered before the content -generator is enabled/added to Koji - -* Where is the content generator service running, what is its support -status etc? -* Is new type of content being added or is the content generator -providing different way to build content Koji already knows about? -* What is the expected size of content that will be imported into Koji? -* Does the content generator follow each of the requirements for writing -it from Koji documentation referenced above? - -=== Adding a new content generator into koji - -First we create the content generator and give a user permission to do -imports for it: - -.... -koji grant-cg-access --new -.... - -In many cases the content generator will be adding content with new -content type. This can be achieved simply by running: - -.... -koji call addBType -.... - -==== Explanation - -* username - is a name of user which will be doing the imports. In most -cases this will be a service-level account -* content generator name - this name has to be provided by the content -generator development team -* --new - this switch ensures we create the content generator if it -doesn't exist diff --git a/modules/release_guide/pages/sop_adding_new_branch_sla.adoc b/modules/release_guide/pages/sop_adding_new_branch_sla.adoc deleted file mode 100644 index efade52..0000000 --- a/modules/release_guide/pages/sop_adding_new_branch_sla.adoc +++ /dev/null @@ -1,28 +0,0 @@ -== Adding New Branch SLAs - -=== Description - -In the ArbitraryBranching model, packagers can choose whatever SLAs they -want for the branches of their packages, but they must choose from a -subset of pre-defined SLAs stored in PDC, maintained by releng. - -This SOP describes the steps necessary for a release engineer to create -a new SLA. - -=== Action - -Adding a new SLA is simple. It involves running a script in the releng -repo, with an authorized token. There is a token available on -[.title-ref]#pdc-backend01# in the [.title-ref]#/etc/pdc.d/# directory. - -.... -$ ./scripts/pdc/insert-sla.py -Name of the SLA: wild_and_cavalier -Description of the SLA: Anything goes! This branch may rebase at any time. No stability guarantees provided. -.... - -=== Verification - -Verifying that the SLA is present is simple: visit the -https://pdc.fedoraproject.org/rest_api/v1/component-branch-slas/[appropriate -PDC endpoint] and verify that your newly-added SLA is present. diff --git a/modules/release_guide/pages/sop_adding_new_release_engineer.adoc b/modules/release_guide/pages/sop_adding_new_release_engineer.adoc deleted file mode 100644 index 2cf2c5a..0000000 --- a/modules/release_guide/pages/sop_adding_new_release_engineer.adoc +++ /dev/null @@ -1,160 +0,0 @@ -== Adding a New Release Engineer - -=== Description - -People volunteer (or get assigned) to doing Fedora release engineering -from time to time. This SOP seeks to describe the process to add a new -release engineer so that they have the rights to accomplish their tasks, -know where to find the tasks, and are introduced to the existing -members. There are several groups that manage access to the respective -systems: - -* `cvsadmin`: Admin group for pkgdb2 (allows to retire/orphan all -packages etc), allows git write access via SSH to all packages in -dist-git -* `gitreleng`: Allows write access to release engineering git repo -* `signers`: Membership is necessary to use -https://fedoraproject.org/wiki/Sigul_Client_Setup_SOP[sigul]. -* `sysadmin`: Allows SSH access to bastion, the SSH gateway to the PHX2 -data center. SSH access to several other internal systems is only -possible from here. -* `sysadmin-cvs`: Allows shell access to pkgs01 (pkgs.fedoraproject.org) -* `sysadmin-releng`: Allows SSH access to autosign01, koji03, koji04, -releng04, relepel01 from bastion - -=== Action - -A new release engineer will access rights in a variety of systems, as -well as be introduced to the releng group. - -==== Git - -Fedora Release Engineering maintains a git repo of scripts. This can be -found in https://pagure.io/pagure[Pagure] at -ssh://git@pagure.io/releng.git. Write access to this group is controlled -by access to the 'gitreleng' FAS group. The new member's FAS username -will need to be added to this group. - -https://pagure.io/releng - -`FIXME: walkthrough group addition` - -==== FAS - -There is a releng group in FAS that release engineers are added to in -order to grant them various rights within the Fedora infrastructure. The -new member's FAS username will need to be added to this group. - -`FIXME: walkthrough group addition` - -==== Koji - -In order to perform certain (un)tagging actions a release engineer must -be an admin in koji. To grant a user admin rights one uses the -`grant-permission` command in koji. - -.... -$ koji grant-permission --help -Usage: koji grant-permission [ ...] -(Specify the --help global option for a list of other help options) - -Options: - -h, --help show this help message and exit -.... - -For example if we wanted to grant npetrov admin rights we would issue: - -.... -$ koji grant-permission admin npetrov -.... - -==== Sigul - -Sigul is our signing server system. They need to bet setup as a signer -if they are going to be signing packages for a release. - -For information on how to setup Sigul, please see: -https://fedoraproject.org/wiki/Sigul_Client_Setup_SOP[sigul] - -==== RelEng Docs Page - -The new release engineer should be added to the -ref:link:index-team-composition[Release Engineering membership list] - -==== rel-eng email list - -Release engineering ticket spam and discussion happens on our -https://admin.fedoraproject.org/mailman/listinfo/rel-eng[Mailing List]. -New releng people need to subscribe. - -==== IRC - -We ask that release engineers idle in [.title-ref]##fedora-releng# on -Libera to be available for queries by other infrastructure admins. -Idling on [.title-ref]##fedora-admin# on Libera is optional. It is noisy -little bit but people sometimes ask releng stuff. - -==== New member announcement - -When a new releng member starts, we announce it to the email list. This -lets the other admins know to expect a new name to show up in tickets -and on IRC. - -=== Verification - -==== Git - -You can verify group membership by sshing to a system that is setup with -FAS and using `getent` to verify membership in the gitreleng group: - -.... -$ ssh fedorapeople.org getent group gitreleng -gitreleng:x:101647:ausil,dwa,jwboyer,kevin,notting,pbabinca,sharkcz,skvidal,spot -.... - -You can verify that the new user is in the above list. - -==== FAS - -You can verify group membership by sshing to a system that is setup with -FAS and using `getent` to verify membership in the releng group: - -.... -$ ssh fedorapeople.org getent group releng -releng:x:101737:atowns,ausil,dwa,jwboyer,kevin,lmacken,notting,pbabinca,spot -.... - -You can verify that the new user is in the above list. - -==== Koji - -To verify that the releng member is an admin koji use the -`list-permissions` koji command: - -.... -$ koji list-permissions --user npetrov -admin -.... - -This shows that npetrov is an admin. - -==== Sigul - -* `FIXME` - -==== Wiki Page - -Verification is easy. Just look at the page. - -==== releng mailing list - -Verify by asking the user if they got the announcement email - -==== Announcement email - -See above - -=== Consider Before Running - -* Make sure the releng person has a solid grasp of the tasks we do and -where to get help if stuck diff --git a/modules/release_guide/pages/sop_adding_packages_compose_artifact.adoc b/modules/release_guide/pages/sop_adding_packages_compose_artifact.adoc deleted file mode 100644 index f6ba618..0000000 --- a/modules/release_guide/pages/sop_adding_packages_compose_artifact.adoc +++ /dev/null @@ -1,81 +0,0 @@ -== Adding a package to a Release Artifact - -=== Description - -In the event that a Fedora contributor would like to have a package -added to an Artifact of a Compose (such as an installer ISO image, a -liveCD, Cloud Image, Vagrant, Docker, etc.) that is slated for Release, -the following procedures must be followed due to the interdependence of -different components within the distro layout. - -==== Background - -First, some information on where this all comes from and how things fit -together. - -There is the concept of the "Install Tree" which is the collection of -packages available at install time. This is a vast sub-set of the whole -of the Fedora Package Collection and it is the pool of possible packages -that is available to end users who choose to customize their install -from the https://fedoraproject.org/wiki/Anaconda[Anaconda] installer. It -is also the pool of possible packages that is available to the -https://pagure.io/fedora-kickstarts[fedora-kickstarts] kickstart files -that are used to generate various components of the compose via -https://pagure.io/pungi[pungi] which then produces the Release -Artifacts. - -The Install Tree itself is defined by the -https://pagure.io/fedora-comps[comps] groups so in order to add a net -new package to one of the Release Artifacts, the package must be placed -in an appropriate https://pagure.io/fedora-comps[comps] xml file. For -more information on what specifically defines the "appropriate -[.title-ref]#comps_# xml file" and what kinds of approvals or review -might be needed for adding new packages, please see -https://fedoraproject.org/wiki/How_to_use_and_edit_comps.xml_for_package_groups[this -HowTo]. - -=== Action - -We will need to edit the comps file specific to the Fedora release we -would like to target. For example, if we were to target Fedora 25 we -would edit `comps-f25.xml.in` found within the -https://pagure.io/fedora-comps[comps] git repository and this should be -modified based on the -https://fedoraproject.org/wiki/How_to_use_and_edit_comps.xml_for_package_groups#How_to_edit_comps[How -to edit comps] procedure. - -If the package that was added is a part of a pre-existing comps group -that is already used in the target Release Artifact's -https://pagure.io/fedora-kickstarts[fedora-kickstarts] kickstart file -then we are done. - -However, if there is a new comps group added then we need to include -that new comps group in the respective -https://pagure.io/fedora-kickstarts[fedora-kickstarts] kickstart file -similar to the following. - -.... -%packages -@mynewcompsgroup -.... - -Next we will need to tell https://pagure.io/pungi[pungi] Variants data -about the new group and it's relationship to the corresponding -https://sgallagh.wordpress.com/2016/03/18/sausage-factory-multiple-edition-handling-in-fedora/[Variant]. -This information is held in the https://pagure.io/pungi-fedora[Fedora -Pungi Configs] https://pagure.io/[pagure] git forge repository. The file -needed to be edited is `variants-fedora.xml` and can be viewed from a -web browser -https://pagure.io/pungi-fedora/blob/master/f/variants-fedora.xml[here]. - -Once this has been completed, we're all done. - -=== Verification - -Verify that the next compose is successful and that the change made -didn't cause any issues. This can be done from the -https://pdc.fedoraproject.org/compose/[Fedora Product Definition Center] -which is a central store of information about Composes and their -resulting artifacts. - -=== Consider Before Running diff --git a/modules/release_guide/pages/sop_adding_side_build_targets.adoc b/modules/release_guide/pages/sop_adding_side_build_targets.adoc deleted file mode 100644 index 712261f..0000000 --- a/modules/release_guide/pages/sop_adding_side_build_targets.adoc +++ /dev/null @@ -1,205 +0,0 @@ -== Adding Side Build Tags - -=== Description - -Bigger Features can take a while to stabilize and land or need a large -number of packages to be built against each other, this is easiest -served by having a separate build tag for the development work. This SOP -will describe the steps necessary to prepare the new build target. - -=== Action - -Engineers should be aware that adding side build targets incurs extra -newRepo tasks in the koji. - -==== Research Tag - -. Verify whether a tag already exists. -+ -The typical tag format is _PRODUCT_-_DESCRIPTOR_. The _DESCRIPTOR_ -should be something brief that clearly shows why the tag exists. -+ -[NOTE] -==== -Don't think too hard about what makes a good descriptor. The descriptor -for the XFCE version 4.8 side-build target was _xfce48_. KDE often -simply uses _kde_ as its descriptor. Use best judgement and if in doubt -ask in IRC on `#fedora-releng`. -==== -+ -EPEL6 - -.... -$ koji taginfo epel6-kde -.... -+ -EPEL7 - -.... -$ koji taginfo epel7-kde -.... -+ -Fedora - -.... -$ koji taginfo f28-kde -.... -+ -[NOTE] -==== -If the tag already exists, continue searching for an available tag by -appending `-2` and incrementing the number by one until an available tag -is found. For example, if `f28-kde` already exists then search for -`f28-kde-2`, `f28-kde-3`, and so on until a suitable tag is found. -==== -. Determine the appropriate architectures. -+ -EPEL6 - -.... -$ koji taginfo dist-6E-epel-build -.... -+ -EPEL7 - -.... -$ koji taginfo epel7-build -.... -+ -Fedora - -.... -$ koji taginfo f28-build -.... - -==== Create Side Build Target - -. Create the proper tag. -+ -Note the slightly different syntax depending on which product needs the -side-build target and the comma delimited list of architectures based on -the information from a previous step. -+ -EPEL6 - -.... -$ koji add-tag epel6-kde --parent=dist-6E-epel-build --arches=i686,x86_64,ppc64 -.... -+ -EPEL7 - -.... -$ koji add-tag epel7-kde --parent=epel7-build --arches=aarch64,x86_64,ppc64,ppc64le -.... -+ -Fedora - -.... -$ koji add-tag f28-kde --parent=f28-build --arches=armv7hl,i686,x86_64,aarch64,ppc64,ppc64le,s390x -.... -. Create the target. -+ -EPEL6 - -.... -$ koji add-target epel6-kde epel6-kde -.... -+ -EPEL7 - -.... -$ koji add-target epel7-kde epel7-kde -.... -+ -Fedora - -.... -$ koji add-target f28-kde f28-kde -.... -. Find the taskID for the newRepo task associated with the newly created -target. -+ -.... -$ koji list-tasks --method=newRepo -ID Pri Owner State Arch Name -25101143 15 kojira OPEN noarch newRepo (f28-kde) -.... -. Ensure the newRepo task is being run across all architectures. -+ -.... -$ koji watch-task 25101143 -Watching tasks (this may be safely interrupted)... -25101143 newRepo (f28-kde): open (buildvm-14.phx2.fedoraproject.org) -25101154 createrepo (i386): closed -25101150 createrepo (ppc64le): closed -25101152 createrepo (ppc64): closed -25101151 createrepo (aarch64): closed -25101149 createrepo (armhfp): closed -25101153 createrepo (s390x): open (buildvm-ppc64le-04.ppc.fedoraproject.org) -25101148 createrepo (x86_64): open (buildvm-aarch64-21.arm.fedoraproject.org) -.... -. Request Package Auto-Signing for New Tag -+ -File a ticket in https://pagure.io/fedora-infrastructure/issues[pagure -infrastructure] requesting the new tag be enabled for package -auto-signing. -. Update the Pagure Issue -+ -Update the issue according to the following template which assumes a -side target was made for KDE under Fedora 28. _TAG_NAME_ has been -created: -+ -____ -$ koji add-tag f28-kde --parent=f28-build ---arches=armv7hl,i686,x86_64,aarch64,ppc64,ppc64le,s390x - -$ koji add-target f28-kde f28-kde - -You can do builds with: - -$ fedpkg build --target=f28-kde - -Let us know when you are done and we will move all the builds into f28. -____ - -=== Cleanup - -Fedora Release Engineering is responsible for merging the packages from -the side-build target and tag back into the main tag. The requestor will -update the original ticket when ready for the procedure outlined below. - -. Remove the target -+ -.... -$ koji remove-target -.... -. Merge side build back to main target. -+ -Get the latest checkout from https://pagure.io/releng/[Fedora Release -Engineering Repository] and run the [.title-ref]#mass-tag.py# from the -scripts directory. -+ -.... -$ ./mass-tag.py --source --target > mass_tag.txt -.... -+ -[NOTE] -==== -The _MAIN_TAG_NAME_ for Fedora is typically the pending subtag, e.g. -`f28-pending` when bodhi is not managing updates. After bodhi is enabled -and managing updates then merge into `f28-updates-candidate`. -==== -. Paste Output to the Original Ticket -+ -Paste the output from mass-tag.py into the pagure/releng ticket to show -what packages were merged and what packages need rebuilding for those -who work on the buildroot. - -Tags are *never* removed. - -=== Consider Before Running - -* Is the amount of work to be done worth the cost of newRepo tasks. -* If there is only a small number of packages overrides may be better. -* Is there a mass-rebuild going on? no side tags are allowed while a -mass rebuild is underway diff --git a/modules/release_guide/pages/sop_branch_freeze.adoc b/modules/release_guide/pages/sop_branch_freeze.adoc deleted file mode 100644 index 9305884..0000000 --- a/modules/release_guide/pages/sop_branch_freeze.adoc +++ /dev/null @@ -1,31 +0,0 @@ -== Branching Freeze - -=== Introduction/Background - -When the next release is branched from rawhide, it initially composes -much like rawhide with nightly composes and no updates process. - -Once the Bodhi is activated, we will push updates to the branched and -the nightly composes will start to differ. But two weeks before the -scheduled release of either Beta or GA, we will start the freeze for -that release and stop pushing updates. - -* Send announcement to devel-announce mailing list noting that the alpha -change freeze is going to happen at least one day in advance. - -[NOTE] -==== -For updates pushers: In Change freeze only updates that fix accepted -blockers or Freeze break bugs are allowed into the main tree. Please -coordinate with QA for any stable updates pushes. Otherwise ONLY push -updates-testing. -==== - -[NOTE] -==== -For Final/GA release: During Final freeze, we dont want to block any -packages in koji as it will effect the RC composes. So, please update -the -https://pagure.io/releng/blob/master/f/scripts/block_retired.py[block_retired.py] -script and remove the branched release reference. -==== diff --git a/modules/release_guide/pages/sop_branching.adoc b/modules/release_guide/pages/sop_branching.adoc deleted file mode 100644 index 6804dc9..0000000 --- a/modules/release_guide/pages/sop_branching.adoc +++ /dev/null @@ -1,33 +0,0 @@ -== Branching - -=== Description - -This SOP covers how to make git and pkgdb branches for packages, either -for new packages that have passed review, or for existing packages that -need a new branch (e.g. EPEL). Release Engineering has written a script -to automate this process. - -=== Normal Action (automated) - -. On your local system (not on an infrastructure hosted system), be sure -you have the following packages installed: -* python-bugzilla -* python-configobj -* python-fedora -. Run "bugzilla login" and successfully receive an Authorization cookie. -. {blank} -+ -Clone the fedora-infrastructure tools repository::: -.... -git clone https://pagure.io/releng.git -.... -. In scripts/process-git-requests, run "process-git-requests". Answer -the prompts. - -=== Manual Action - -==== Creating a new branch for an existing package - -. ssh into `pkgs.fedoraproject.org` -. `pkgdb-client edit -u $YOURUSERNAME -b $NEWBRANCH --master=devel $NAMEOFPACKAGE` -. `pkgdb2branch.py $NAMEOFPACKAGE` diff --git a/modules/release_guide/pages/sop_breaking_development_freeze.adoc b/modules/release_guide/pages/sop_breaking_development_freeze.adoc deleted file mode 100644 index ee40471..0000000 --- a/modules/release_guide/pages/sop_breaking_development_freeze.adoc +++ /dev/null @@ -1,53 +0,0 @@ -include::_partials/attributes.adoc[] - -== Breaking Development Freeze - -=== Description - -Packages which require an exception to freeze policies must be run -through this SOP. - -The following freeze policies are set for the following significant -release milestones: - -* https://fedoraproject.org/wiki/Software_String_Freeze_Policy[String -Freeze] -* https://fedoraproject.org/wiki/Milestone_freezes[Beta Freeze] -* https://fedoraproject.org/wiki/Milestone_freezes[Final Freeze] - -See https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle[Fedora -Release Life Cycle] for a summary of all the freezes, dates, and -exception handling, or the release engineering -[https://fedorapeople.org/groups/schedule/f-{rawhide_next}-releng-tasks.html -calendar for the current release]. - -=== Action - -The commands to tag a package properly once it has been accepted: - -[source,subs="attributes+"] -.... -$ koji move-pkg --force dist-f{rawhide_next}-updates-candidate dist-f{rawhide_next} -$ koji tag-pkg --force f{rawhide_next}- -.... - -Where is the package name, and is the first release -in which the package should land (e.g. alpha, beta, final). - -=== Verification - -The `koji` client reports success or failure. For secondary -verification, run these commands: - -[source,subs="attributes+"] -.... -$ koji latest-pkg dist-f{rawhide_next} -$ koji latest-pkg dist-f{rawhide_next}-updates-candidate -.... - -=== Consider Before Running - -* Change agrees with stated policies (see links above) -* Approval for change has been granted under -https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process[Blocker Bug -Process] or [.title-ref]#Freeze Exception Bug Process# diff --git a/modules/release_guide/pages/sop_composing_fedora.adoc b/modules/release_guide/pages/sop_composing_fedora.adoc deleted file mode 100644 index b310bfd..0000000 --- a/modules/release_guide/pages/sop_composing_fedora.adoc +++ /dev/null @@ -1,612 +0,0 @@ -== Composing Fedora - -=== Description - -All composes are defined by configuration files kept in the -https://pagure.io/pungi-fedora[pungi-fedora repository]. - -Composes fall into two categories. They may be release candidates -created on demand or nightly composes set to run at a scheduled time -each day. - -[cols=",,",options="header",] -|=== -|Compose Name |Configuration File |Compose Script -|Docker |fedora-docker.conf |docker-nightly.sh -|Cloud |fedora-cloud.conf |cloud-nightly.sh -|Modular |fedora-modular.conf |modular-nightly.sh -|Nightly |fedora.conf |nightly.sh -|=== - -When Quality Engineering (QE) requests a Release Candidate (RC) they do -so by opening an issue in the releng repository on pagure. Release -candidate composes are not currently automated. - -[cols=",,",options="header",] -|=== -|Compose Name |Configuration File |Compose Script -|Beta |fedora-beta.conf |release-candidate.sh -|GA |fedora-final.conf |release-candidate.sh -|=== - -=== Action - -The following procedures are for release candidates only. They do not -apply to the scheduled nightly composes. - -==== Review Compose Tags - -. List any pre-existing builds in the current compose tag -+ -.... -$ koji list-tagged f[release_version]-compose -.... -. Verify pre-existing builds are in compose tags -+ -The tagged builds from the previous composes should all be present in -the output from the previous step. Consult the request ticket for the -list of builds expected in this output. -+ -[NOTE] -==== -The very first run of an Beta, or GA compose should have no builds -listed under the compose tag. It is important to clear pre-existing -builds from the compose tag when moving between the Beta and RC -composes. Verify that these builds were removed. - -.... -$ koji list-tagged f[release_version]-compose -$ koji untag-build --all f[release_version]-compose [build1 build2 ...] -.... -==== -+ -[NOTE] -==== -The order in which packages are added into the -f[release_version]-compose tag matter. If the builds are untagged -erroneously then special attention should be given to adding them back -correctly. -==== -. Add builds specified by QE to the current compose tag -+ -.... -$ koji tag-build f[release_version]-compose [build1 build2 ...] -.... -+ -[NOTE] -==== -These steps may be completed on a local machine as long as the user has -appropriate permissions in the koji tool. -==== - -==== Package Signing before the Compose - -. Check for unsigned packages -+ -.... -$ koji list-tagged f[release_version]-signing-pending -.... -+ -[NOTE] -==== -If there are unsigned builds then wait for the automated queue to pick -them up and sign them. Contact a member of the Fedora infrastructure -team if the package signing has taken more than thirty minutes. -==== - -==== Running the Compose - -. Update the pungi-fedora config file Composes use a configuration file -to construct the compose. Each compose uses its own configuration. The -`global_release` variable should start from 1.1 and the second number -should increment each time a new compose is created. -* Beta - `fedora-beta.conf` -* RC - `fedora-final.conf` -. Log into the compose backend -+ -.... -$ ssh compose-x86-01.phx2.fedoraproject.org -.... -. Open a screen session -+ -.... -$ screen -.... -. Obtain the pungi-fedora branch for the current compose -+ -The first time any user account executes a compose the pungi-fedora git -repository must be cloned. The compose candidate script that invokes -pungi should be run from `compose-x86-01.iad2.fedoraproject.org`. -+ -.... -$ git clone ssh://git@pagure.io/pungi-fedora.git -.... -+ -Enter the pungi-fedora directory. -+ -.... -$ cd pungi-fedora -.... -+ -If the clone step above was not required then fully update the existing -repository checkout from pagure. -+ -.... -$ git fetch origin -$ git checkout f[release_version] -$ git pull origin f[release_version] -.... -. Run the compose -+ -.... -$ sudo ./release-candidate.sh [Beta|RC]-#.# -.... -+ -The numbering scheme begins with 1.1 and the second number is -incremented after each compose. -+ -[NOTE] -==== -Pungi requires numbers in the format #.# as an argument. It is because -of this that composes always start with the number 1 and the second -number is incremented with each compose. -==== -+ -[NOTE] -==== -If the compose fails with a directory missing error, then create the -compose directory with `mkdir /mnt/koji/compose/[release_version]` -==== - -==== Syncing the Compose - -We sync the compose to `/pub/alt/stage` to enable faster access to new -content for QA and the larger Fedora community. - -. Log into the compose backend -+ -.... -$ ssh compose-x86-01.iad2.fedoraproject.org -.... -. Open a screen session -+ -.... -$ screen -.... -. Check the status of the compose -+ -.... -$ cat /mnt/koji/compose/[release_version]/[compose_id]/STATUS -.... -+ -Do not continue with any further steps if the output above is `DOOMED`. -. Create the directory targeted for the copy : -+ -.... -$ sudo -u ftpsync mkdir -m 750 -p /pub/alt/stage/[release_version]_[release_label]-[#.#] -.... -. Locate the compose directory that will be the copy source : -+ -.... -$ ls /mnt/koji/compose/[release_version]/[compose_id] -.... -+ -[NOTE] -==== -Take care executing the synchronization if the next compose is already -running. Be sure to grab the correct directory. - -If in doubt, check -/mnt/koji/compose/[release_version]/[compose_id]/STATUS to be sure it is -finished. -==== -. Run the synchronization one-liner -+ -The synchronization of the completed compose to the public domain is -currently a one-liner shell script. Pay close attention to what needs -replaced in the example below. -+ -.... -$ sudo -u ftpsync sh -c 'for dir in Everything Cloud Container Kinoite Labs Modular Server Silverblue Spins Workstation metadata; do rsync -avhH /mnt/koji/compose/31/Fedora-31-20190911.0/compose/$dir/ /pub/alt/stage/31_Beta-1.1/$dir/ --link-dest=/pub/fedora/linux/development/31/Everything/ --link-dest=/pub/alt/stage/31_Beta-1.1/Everything/; done' -.... -+ -[NOTE] -==== -If multiple composes are run like 1.2, 1.3, add multiple --link-dest -arguments above with multiple composes -==== -. Set the permissions of the synced compose : -+ -.... -$ sudo -u ftpsync chmod 755 /pub/alt/stage/[release_version]_[release_label]-[#.#] -.... -. Update the issue in the releng pagure repository -+ -Once the compose and sync is complete the issue in pagure should be -updated and closed. -+ -Standard Ticket Verbage - -Compose is done and available from -https://kojipkgs.fedoraproject.org/compose/26/Fedora-26-20170328.0/compose/ -it has been synced to -http://dl.fedoraproject.org/pub/alt/stage/26_Alpha-1.4/ rpms have all be -hardlinked to /pub/fedora/linux/development/26/ - -===== Verification - -The method for verifying a compose has completed is checking -`/mnt/koji/compose/[release_version]/[compose_dir]/STATUS`. Any status -other than DOOMED is OK. - -=== Pre-Release Work - -==== Pushing Updates to Stable - -When the release is signed off on Thursday after the Go/No-Go meeting, -push the freeze and blocker to stable updates - -Generally the updates are requested stable by QA. If they are not -available, you can request them by following - -.... -$ bodhi updates request stable -.... - -Once the updates are requested stable, please push them to stable by -following the -https://docs.pagure.org/releng/sop_pushing_updates.html#pushing-stable-updates-during-freeze[bodhi -push to stable sop] - -==== koji tag changes - -Once the updates are pushed stable, we need to clone the koji tag for -beta release or lock the koji tag for final release. - -===== For Beta Release - -.... -$ koji clone-tag --all --latest-only f31 f31-Beta -$ koji clone-tag --all --latest-only f31-modular f31-Beta-modular -.... - -===== For Final Release - -.... -$ koji edit-tag --lock f31 -$ koji edit-tag --lock f31-modular -.... - -==== Bodhi Changes - -Set the bodhi release to `current` - -.... -$ bodhi releases edit --name F31 --state current -.... - -=== Changes for Final Release - -Once Final is GO, we need to perform different changes as that of Beta -release. - -==== Last Branched Compose - -Manually run a branched compose so that the GOLD content is same as the -nightly compose. This also helps in updating the silverblue refs as that -of the GOLD content. - -==== Update silverblue refs - -Please update the refs as per the following commands on -[.title-ref]#bodhi-backend01.phx2.fedoraproject.org# - -Run the following commands from -[.title-ref]#/mnt/koji/compose/ostree/repo# and -[.title-ref]#/mnt/koji/ostree/repo/# - -.... -$ sudo -u ftpsync ostree refs --create=fedora/31/x86_64/updates/silverblue fedora/31/x86_64/silverblue -$ sudo -u ftpsync ostree refs --create=fedora/31/aarch64/updates/silverblue fedora/31/aarch64/silverblue -$ sudo -u ftpsync ostree refs --create=fedora/31/ppc64le/updates/silverblue fedora/31/ppc64le/silverblue - -$ sudo ostree refs --delete fedora/31/x86_64/silverblue -$ sudo ostree refs --delete fedora/31/aarch64/silverblue -$ sudo ostree refs --delete fedora/31/ppc64le/silverblue - -$ sudo -u ftpsync ostree refs --alias --create=fedora/31/x86_64/silverblue fedora/31/x86_64/updates/silverblue -$ sudo -u ftpsync ostree refs --alias --create=fedora/31/aarch64/silverblue fedora/31/aarch64/updates/silverblue -$ sudo -u ftpsync ostree refs --alias --create=fedora/31/ppc64le/silverblue fedora/31/ppc64le/updates/silverblue -.... - -Run the following command only from [.title-ref]#/mnt/koji/ostree/repo/# - -.... -$ sudo ostree summary -u -.... - -[NOTE] -==== -Before pushing the updates to fxx-updates, run the last branched compose -so that both branched and rc composes have the same content. Once the -branched compose is done, then update the silverblue refs as mentioned -above. If the order is changed, that will screw up the refs -==== - -==== Disable Branched Compose - -Now that we have a final GOLD compose, we dont need nightly branched -composes anymore. This is disabled in -https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/releng[releng -role] in infra ansible repo and then running the playbook. - -.... -$ sudo rbac-playbook groups/releng-compose.yml -.... - -==== Lift RelEng freeze - -Lift the RelEng Freeze so that the updates will be pushed to stable. -This is done by editing -https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/vars/all/RelEngFrozen.yaml[RelEngFrozen -variable] in infra ansible repo and then run the bodhi playbook. - -.... -$ sudo rbac-playbook groups/bodhi-backend.yml -.... - -==== Other Changes - -These changes include enabling nightly container and cloud composes, -other variable changes in infra ansible repo, bodhi pungi config -changes, updates sync changes and others. - -Run the appropriate playbooks after the following changes - -.... -diff --git a/roles/releng/files/branched b/roles/releng/files/branched - index 966f5c3..1c0454f 100644 - --- a/roles/releng/files/branched - +++ b/roles/releng/files/branched - @@ -1,3 +1,3 @@ - # branched compose - -MAILTO=releng-cron@lists.fedoraproject.org - -15 7 * * * root TMPDIR=`mktemp -d /tmp/branched.XXXXXX` && cd $TMPDIR && git clone https://pagure.io/pungi-fedora.git && cd pungi-fedora && git checkout f31 && /usr/local/bin/lock-wrapper branched-compose "PYTHONMALLOC=debug LANG=en_US.UTF-8 ./nightly.sh" && sudo -u ftpsync /usr/local/bin/update-fullfiletimelist -l /pub/fedora-secondary/update-fullfiletimelist.lock -t /pub fedora fedora-secondary - +#MAILTO=releng-cron@lists.fedoraproject.org - +#15 7 * * * root TMPDIR=`mktemp -d /tmp/branched.XXXXXX` && cd $TMPDIR && git clone https://pagure.io/pungi-fedora.git && cd pungi-fedora && git checkout f31 && /usr/local/bin/lock-wrapper branched-compose "PYTHONMALLOC=debug LANG=en_US.UTF-8 ./nightly.sh" && sudo -u ftpsync /usr/local/bin/update-fullfiletimelist -l /pub/fedora-secondary/update-fullfiletimelist.lock -t /pub fedora fedora-secondary - diff --git a/roles/releng/files/cloud-updates b/roles/releng/files/cloud-updates - index a0ffbe8..287d57d 100644 - --- a/roles/releng/files/cloud-updates - +++ b/roles/releng/files/cloud-updates - @@ -6,6 +6,6 @@ MAILTO=releng-cron@lists.fedoraproject.org - MAILTO=releng-cron@lists.fedoraproject.org - 15 7 * * * root TMPDIR=`mktemp -d /tmp/CloudF29.XXXXXX` && pushd $TMPDIR && git clone -n https://pagure.io/pungi-fedora.git && cd pungi-fedora && git checkout f29 && LANG=en_US.UTF-8 ./cloud-nightly.sh RC-$(date "+\%Y\%m\%d").0 && popd && rm -rf $TMPDIR - - -#Fedora 28 Cloud nightly compose - -#MAILTO=releng-cron@lists.fedoraproject.org - -#15 8 * * * root TMPDIR=`mktemp -d /tmp/CloudF28.XXXXXX` && pushd $TMPDIR && git clone -n https://pagure.io/pungi-fedora.git && cd pungi-fedora && git checkout f28 && LANG=en_US.UTF-8 ./cloud-nightly.sh RC-$(date "+\%Y\%m\%d").0 && popd && rm -rf $TMPDIR - +#Fedora 31 Cloud nightly compose - +MAILTO=releng-cron@lists.fedoraproject.org - +15 8 * * * root TMPDIR=`mktemp -d /tmp/CloudF31.XXXXXX` && pushd $TMPDIR && git clone -n https://pagure.io/pungi-fedora.git && cd pungi-fedora && git checkout f31 && LANG=en_US.UTF-8 ./cloud-nightly.sh RC-$(date "+\%Y\%m\%d").0 && popd && rm -rf $TMPDIR - diff --git a/roles/releng/files/container-updates b/roles/releng/files/container-updates - index d763149..5446840 100644 - --- a/roles/releng/files/container-updates - +++ b/roles/releng/files/container-updates - @@ -1,6 +1,6 @@ - -#Fedora 28 Container Updates nightly compose - -#MAILTO=releng-cron@lists.fedoraproject.org - -#45 5 * * * root TMPDIR=`mktemp -d /tmp/containerF28.XXXXXX` && pushd $TMPDIR && git clone -n https://pagure.io/pungi-fedora.git && cd pungi-fedora && git checkout f28 && LANG=en_US.UTF-8 ./container-nightly.sh RC-$(date "+\%Y\%m\%d").0 && popd && rm -rf $TMPDIR - +#Fedora 31 Container Updates nightly compose - +MAILTO=releng-cron@lists.fedoraproject.org - +45 5 * * * root TMPDIR=`mktemp -d /tmp/containerF31.XXXXXX` && pushd $TMPDIR && git clone -n https://pagure.io/pungi-fedora.git && cd pungi-fedora && git checkout f31 && LANG=en_US.UTF-8 ./container-nightly.sh RC-$(date "+\%Y\%m\%d").0 && popd && rm -rf $TMPDIR - - # Fedora 30 Container Updates nightly compose - MAILTO=releng-cron@lists.fedoraproject.org - diff --git a/vars/all/00-FedoraCycleNumber.yaml b/vars/all/00-FedoraCycleNumber.yaml - index 22476b0..4bd0d46 100644 - --- a/vars/all/00-FedoraCycleNumber.yaml - +++ b/vars/all/00-FedoraCycleNumber.yaml - @@ -1 +1 @@ - -FedoraCycleNumber: 30 - +FedoraCycleNumber: 31 - - diff --git a/vars/all/FedoraBranched.yaml b/vars/all/FedoraBranched.yaml - index 42ac534..0bbcc1d 100644 - --- a/vars/all/FedoraBranched.yaml - +++ b/vars/all/FedoraBranched.yaml - @@ -1 +1 @@ - -FedoraBranched: True - +FedoraBranched: False - - diff --git a/vars/all/FedoraPreviousPrevious.yaml b/vars/all/FedoraPreviousPrevious.yaml - index a8e3d3b..a061e04 100644 - --- a/vars/all/FedoraPreviousPrevious.yaml - +++ b/vars/all/FedoraPreviousPrevious.yaml - @@ -1 +1 @@ - -FedoraPreviousPrevious: False - +FedoraPreviousPrevious: True - diff --git a/vars/all/Frozen.yaml b/vars/all/Frozen.yaml - index 97d3bc3..7578a88 100644 - --- a/vars/all/Frozen.yaml - +++ b/vars/all/Frozen.yaml - @@ -1 +1 @@ - -Frozen: True - +Frozen: False - - - diff --git a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 - index 688bade..28b524a 100644 - --- a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 - +++ b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 - @@ -179,8 +179,8 @@ ostree = { - # In the case of testing, also inject the last stable updates - "https://kojipkgs{{ env_suffix }}.fedoraproject.org/compose/updates/f[[ release.version_int ]]-updates/compose/Everything/$basearch/os/", - [% endif %] - - # For f31 the compose location is going to be under /compose/branched/ - - [% if release.version_int == 31 %] - + # For F32 the compose location is going to be under /compose/branched/ - + [% if release.version_int == 32 %] - "https://kojipkgs{{ env_suffix }}.fedoraproject.org/compose/branched/latest-Fedora-[[ release.version_int ]]/compose/Everything/$basearch/os/" - [% else %] - "https://kojipkgs{{ env_suffix }}.fedoraproject.org/compose/[[ release.version_int ]]/latest-Fedora-[[ release.version_int ]]/compose/Everything/$basearch/os/" - - diff --git a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 - index 28b524a..640ddf0 100644 - --- a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 - +++ b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 - @@ -193,8 +193,8 @@ ostree = { - "ostree_ref": "fedora/[[ release.version_int ]]/${basearch}/testing/silverblue", - [% endif %] - "tag_ref": False, - - "arches": ["x86_64"], - - "failable": ["x86_64"] - + "arches": ["x86_64", "ppc64le", "aarch64" ], - + "failable": ["x86_64", "ppc64le", "aarch64" ] - }, - ] - } - - - diff --git a/roles/bodhi2/backend/files/new-updates-sync b/roles/bodhi2/backend/files/new-updates-sync - index d08c893..2d0fb4d 100755 - --- a/roles/bodhi2/backend/files/new-updates-sync - +++ b/roles/bodhi2/backend/files/new-updates-sync - @@ -25,8 +25,9 @@ RELEASES = {'f31': {'topic': 'fedora', - 'modules': ['fedora', 'fedora-secondary'], - 'repos': {'updates': { - 'from': 'f31-updates', - - 'ostrees': [{'ref': 'fedora/31/x86_64/updates/silverblue', - - 'dest': OSTREEDEST}], - + 'ostrees': [{'ref': 'fedora/31/%(arch)s/updates/silverblue', - + 'dest': OSTREEDEST, - + 'arches': ['x86_64', 'ppc64le', 'aarch64']}], - 'to': [{'arches': ['x86_64', 'armhfp', 'aarch64', 'source'], - 'dest': os.path.join(FEDORADEST, '31', 'Everything')}, - {'arches': ['ppc64le', 's390x'], - @@ -34,8 +35,9 @@ RELEASES = {'f31': {'topic': 'fedora', - ]}, - 'updates-testing': { - 'from': 'f31-updates-testing', - - 'ostrees': [{'ref': 'fedora/31/x86_64/testing/silverblue', - - 'dest': OSTREEDEST}], - + 'ostrees': [{'ref': 'fedora/31/%(arch)s/testing/silverblue', - + 'dest': OSTREEDEST, - + 'arches': ['x86_64', 'ppc64le', 'aarch64']}], - 'to': [{'arches': ['x86_64', 'aarch64', 'armhfp', 'source'], - 'dest': os.path.join(FEDORADEST, 'testing', '31', 'Everything')}, - {'arches': ['ppc64le', 's390x'], - - - diff --git a/roles/pkgdb-proxy/files/pkgdb-gnome-software-collections.json b/roles/pkgdb-proxy/files/pkgdb-gnome-software-collections.json - index aac977e..9e0cbf2 100644 - --- a/roles/pkgdb-proxy/files/pkgdb-gnome-software-collections.json - +++ b/roles/pkgdb-proxy/files/pkgdb-gnome-software-collections.json - @@ -12,14 +12,14 @@ - "version": "devel" - }, - { - - "allow_retire": true, - + "allow_retire": false, - "branchname": "f31", - "date_created": "2014-05-14 12:36:15", - "date_updated": "2018-08-14 17:07:23", - "dist_tag": ".fc31", - "koji_name": "f31", - "name": "Fedora", - - "status": "Under Development", - + "status": "Active", - "version": "31" - }, - { -.... - -==== Bodhi config - -==== After Beta - -.... -diff --git a/vars/all/FedoraBranchedBodhi.yaml b/vars/all/FedoraBranchedBodhi.yaml -index 606eb2e..ca2ba61 100644 ---- a/vars/all/FedoraBranchedBodhi.yaml -+++ b/vars/all/FedoraBranchedBodhi.yaml -@@ -3,4 +3,4 @@ -#prebeta: After bodhi enablement/beta freeze and before beta release -#postbeta: After beta release and before final release -#current: After final release --FedoraBranchedBodhi: prebeta -+FedoraBranchedBodhi: postbeta -.... - -==== After Final - -.... -diff --git a/vars/all/FedoraBranchedBodhi.yaml b/vars/all/FedoraBranchedBodhi.yaml -index 380f61d..76ba14d 100644 ---- a/vars/all/FedoraBranchedBodhi.yaml -+++ b/vars/all/FedoraBranchedBodhi.yaml -@@ -1,2 +1,2 @@ -#options are: prebeta, postbeta, current --FedoraBranchedBodhi: postbeta -+FedoraBranchedBodhi: current -.... - -==== Mirroring - -Run [.title-ref]#stage-release.sh# script from -https://pagure.io/releng[releng repo] in pagure on -[.title-ref]#bodhi-backend01.phx2.fedoraproject.org#, this will sign the -checksums and will put the content on mirrors. - -===== For Beta - -.... -$ sh scripts/stage-release.sh 32_Beta Fedora-32-20200312.0 32_Beta-1.2 fedora-32 1 -.... - -===== For Final - -.... -$ sh scripts/stage-release.sh 32 Fedora-32-20200415.0 32_RC-1.1 fedora-32 0 -.... - -[NOTE] -==== -Make sure to grab the directory size usage numbers which is used to send -an email to [.title-ref]#mirror-admin@lists.fedoraproject.org# list. -==== - -==== Sync the signed checksums to stage - -We need to sync the signed checksums to /pub/alt/stage/ by running the -following command - -.... -$ for dir in Everything Cloud Container Labs Server Spins Workstation Silverblue Kinoite metadata; do sudo -u ftpsync rsync -avhH /mnt/koji/compose/37/Fedora-37-20221105.0/compose/$dir/ /pub/alt/stage/37_RC-1.7/$dir/ --link-dest=/pub/fedora/linux/releases/37/Everything/ --link-dest=/pub/alt/stage/37_RC-1.2/Everything/ --link-dest=/pub/alt/stage/37_RC-1.3/Everything --link-dest=/pub/alt/stage/37_RC-1.4/Everything --link-dest=/pub/alt/stage/37_RC-1.5/Everything --link-dest=/pub/alt/stage/37_RC-1.6/Everything --link-dest=/pub/alt/stage/37_RC-1.7/Everything; done -.... - -=== Move development to release folder with mirrormanager - -Two weeks after the release move bits from development to release -directory - -. {blank} -+ -ssh to the mm-backend01.iad2.fedoraproject.org:: - ::;; - $ ssh mm-backend01.iad2.fedoraproject.org -. {blank} -+ -get root:: - ::;; - $ sudo su -. {blank} -+ -run the mm2_move-devel-to-release:: - ::;; - $ mm2_move-devel-to-release --version=35 --category='Fedora Linux' - -=== Consider before Running - -Composes and file synchronizations should be run in a screen session on -a remote machine. This enables the operator to withstand network -connection issues. diff --git a/modules/release_guide/pages/sop_create_release_signing_key.adoc b/modules/release_guide/pages/sop_create_release_signing_key.adoc deleted file mode 100644 index 3623010..0000000 --- a/modules/release_guide/pages/sop_create_release_signing_key.adoc +++ /dev/null @@ -1,543 +0,0 @@ -include::_partials/attributes.adoc[] - -== Create Release Signing Key - -=== Description - -At the beginning of each release under development a new package signing -key is created for it. This key is used to prove the authenticity of -packages built by Fedora and distributed by Fedora. This key will be -used to sign all packages for the public test and final releases. - -=== Action - -==== Sigul - -Sigul is the signing server which holds our keys. In order to make use -of a new key, the key will have to be created and access to the key will -have to be granted. The `new-key`, `grant-key-access`, and -`change-passphrase` commands are used. - -.... -$ sigul new-key --help -usage: client.py new-key [options] key - -Add a key - -options: - -h, --help show this help message and exit - --key-admin=USER Initial key administrator - --name-real=NAME_REAL - Real name of key subject - --name-comment=NAME_COMMENT - A comment about of key subject - --name-email=NAME_EMAIL - E-mail of key subject - --expire-date=YYYY-MM-DD - Key expiration date - -$ sigul grant-key-access --help -usage: client.py grant-key-access key user - -Grant key access to a user - -options: - -h, --help show this help message and exit - -$ sigul change-passphrase --help -usage: client.py change-passphrase key - -Change key passphrase - -options: - -h, --help show this help message and exit -.... - -For example if we wanted to create the Fedora {branched} signing key, we would -do the following: - -. Log into a system configured to run sigul client. -. Create the key using a strong passphrase when prompted -+ -[source,subs="attributes+"] -.... -$ sigul new-key --key-admin ausil --key-type gnupg \ - --gnupg-name-real Fedora \ - --gnupg-name-comment {branched} \ - --gnupg-name-email fedora-23-primary@fedoraproject.org fedora-23 -.... -+ -For EPEL -+ -.... -$ sigul new-key --key-admin ausil --key-type gnupg \ - --gnupg-name-real "Fedora EPEL" \ - --gnupg-name-comment 7 \ - --gnupg-name-email epel@fedoraproject.org epel-7 -.... -. Wait a while for entropy. This can take several minutes. -. For Fedora, also create the IMA signing key -+ -[source,subs="attributes+"] -.... -$ sigul new-key --key-admin ausil --key-type ECC fedora-{branched}-ima -.... -. Grant key access to Fedora Account holders who will be signing -packages and protect it with a temporary a passphrase. For example, -`CHANGEME.`. Do the same with the -ima key for Fedora. -+ -[source,subs="attributes+"] -.... -$ sigul grant-key-access fedora-{branched} kevin -.... - -[NOTE] -==== -*IMPORTANT:* Grant the access to autopen user as it's required for -robosignatory autosigning and then restart robosignatory service -==== - -. Provide the key name and temporary passphrase to signers. If they -don't respond, revoke access until they are ready to change their -passphrase. Signers can change their passphrase using the -`change-passphrase` command: -+ -[source,subs="attributes+"] -.... -$ sigul change-passphrase fedora-{branched} -.... -. When your sigul cert expires, you will need to run: -+ -.... -certutil -d ~/.sigul -D -n sigul-client-cert -.... -+ -to remove the old cert, then -+ -.... -sigul_setup_client -.... -+ -to add a new one. - -==== fedora-repos - -The fedora-repos package houses a copy of the public key information. -This is used by rpm to verify the signature on files encountered. -Currently the fedora-repos package has a single key file named after the -version of the key and the arch the key is for. To continue our example, -the file would be named `RPM-GPG-KEY-fedora-{branched}-primary` which is the -primary arch key for Fedora {branched}. To create this file, use the -`get-public-key` command from sigul: - -[source,subs="attributes+"] -.... -$ sigul get-public-key fedora-{branched} > RPM-GPG-KEY-fedora-{branched}-primary -.... - -Add this file to the repo and update the archmap file for the new -release. - -[source,subs="attributes+"] -.... -$ git add RPM-GPG-KEY-fedora-{branched}-primary -.... - -Then make a new fedora-repos build for rawhide -(`FIXME: this should be its own SOP`) - -==== getfedora.org - -getfedora.org/keys lists information about all of our keys. We need to -let the websites team know we have created a new key so that they can -add it to the list. - -We do this by filing an issues in their pagure instance -https://pagure.io/fedora-websites/ we should point them at this SOP - -===== Web team SOP - -.... -# from git repo root -cd fedoraproject.org/ -curl $KEYURL > /tmp/newkey -$EDITOR update-gpg-keys # Add key ID of recently EOL'd version to obsolete_keys -./update-gpg-key /tmp/newkey -gpg static/fedora.gpg # used to verify the new keyring -# it should look something like this: -# pub 4096R/57BBCCBA 2009-07-29 Fedora (12) -# pub 4096R/E8E40FDE 2010-01-19 Fedora (13) -# pub 4096R/97A1071F 2010-07-23 Fedora (14) -# pub 1024D/217521F6 2007-03-02 Fedora EPEL -# sub 2048g/B6610DAF 2007-03-02 [expires: 2017-02-27] -# it must only have the two supported versions of fedora, rawhide and EPEL -# also verify that static/$NEWKEY.txt exists -$EDITOR data/content/{keys,verify}.html # see git diff 1840f96~ 1840f96 -.... - -==== sigulsign_unsigned - -`sigulsign_unsigned.py` is the script Release Engineers use to sign -content in koji. This script has a hardcoded list of keys and aliases to -the keys that needs to be updated when we create new keys. - -Add the key details to the `KEYS` dictionary near the top of the -`sigulsign_unsigned.py` script. It lives in Release Engineering's git -repo at `ssh://git@pagure.io/releng.git` in the `scripts` directory. You -will need to know the key ID to insert the correct information: - -.... -$ gpg -.... - -==== Public Keyservers - -We upload the key to the public key servers when we create the keys. To -do this, we need to get the ascii key block from sigul, determine the -key ID, import they key into our local keyring, and then upload it to -the key servers. - -.... -$ sigul get-public-key fedora-13 > fedora-13 -$ gpg fedora-13 (The ID is the "E8E40FDE" part of 4096R/E8E40FDE) -$ gpg --import fedora-13 -$ gpg --send-keys E8E40FDE -.... - -==== pungi-fedora - -The nightly compose configs come from the pungi-fedora project on -https://pagure.io We need to create a pull request to pull in the new -key. - -.... -$ git clone ssh://git@pagure.io//pungi-fedora.git -$ cd pungi-fedora -$ vim *conf - -$ git commit -m 'Add new key' -$ git push -$ file a Pull Request -.... - -==== Koji - -Koji has a garbage collection utility that will find builds that meet -criteria to be removed to save space. Part of that criteria has to do -with whether or not the build has been signed with a key. If the -collection utility doesn't know about a key it will ignore the build. -Thus as we create new keys we need to inform the utility of these keys -or else builds can pile up. The configuration for the garbage collection -lives within ansible. - -On a clone of the infrastructure ansible git repo edit the -roles/koji_hub/templates/koji-gc.conf.j2 file: - -.... -diff --git a/roles/koji_hub/templates/koji-gc.conf.j2 b/roles/koji_hub/templates/koji-gc.conf.j2 -index 9ecb750..9c48a8e 100644 ---- a/roles/koji_hub/templates/koji-gc.conf.j2 -+++ b/roles/koji_hub/templates/koji-gc.conf.j2 -@@ -35,6 +35,7 @@ key_aliases = - 81B46521 fedora-24 - FDB19C98 fedora-25 - 64DAB85D fedora-26 -+ F5282EE4 fedora-27 - 217521F6 fedora-epel - 0608B895 fedora-epel-6 - 352C64E5 fedora-epel-7 -@@ -52,6 +53,7 @@ unprotected_keys = - fedora-24 - fedora-25 - fedora-26 -+ fedora-27 - fedora-extras - redhat-beta - fedora-epel -@@ -91,6 +93,7 @@ policy = - sig fedora-24 && age < 12 weeks :: keep - sig fedora-25 && age < 12 weeks :: keep - sig fedora-26 && age < 12 weeks :: keep -+ sig fedora-27 && age < 12 weeks :: keep - sig fedora-epel && age < 12 weeks :: keep - sig fedora-epel-6 && age < 12 weeks :: keep - sig fedora-epel-7 && age < 12 weeks :: keep -.... - -In this case the fedora-epel key was added to the list of key aliases, -then referenced in the list of unprotected_keys, and finally a policy -was created for how long to keep builds signed with this key. - -Once you've made your change commit and push. The buildsystem will pick -up this change the next time puppet refreshes. - -=== Verification - -We can verify that the key was created in sigul, the correct users have -access to the key, the key was added to the fedora-release package, that -the website was updated with the right key, that sigulsign_unsigned was -properly updated, and that the key was successfully updated to the -public key servers. - -==== sigul - -Use the `list-keys` command to verify that the key was indeed added to -sigul: - -.... -$ sigul list-keys -Administrator's password: -fedora-10 -fedora-10-testing -fedora-11 -fedora-12 -fedora-13 -.... - -Our new key should be on the list. This command expects *your* -administrative password. - -Use the `list-key-users` command to verify all the signers have access: - -.... -$ sigul list-key-users fedora-13 -Key passphrase: -jkeating -jwboyer -.... - -This command expects *your* key passphrase for the key in question. - -==== fedora-release - -To verify that the key was added to this package correctly, download the -latest build from koji and run rpm2cpio on it, then run gpg on the key -file: - -.... -$ koji download-build --arch noarch --latest f27 fedora-repos -fedora-repos-rawhide-27-0.1.noarch.rpm | 7.3 kB 00:00:00 -fedora-repos-27-0.1.noarch.rpm | 87 kB 00:00:00 -$ rpmdev-extract fedora-repos-27-0.1.noarch.rpm -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-27-fedora -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-10-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-10-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-10-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-10-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-10-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-11-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-11-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-11-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-11-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-11-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-12-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-12-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-12-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-12-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-12-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-13-arm -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-13-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-13-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-13-mips -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-13-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-13-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-13-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-14-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-14-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-14-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-arm -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-s390 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-15-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-arm -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-s390 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-16-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-arm -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-s390 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-arm -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-s390 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-18-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-19-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-19-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-19-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-19-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-19-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-19-s390 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-19-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-19-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-19-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-s390 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-aarch64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-ppc64le -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-s390 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-21-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-aarch64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-ppc64le -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-s390 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-22-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-aarch64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-ppc64le -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-s390 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-23-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-24-aarch64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-24-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-24-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-24-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-24-ppc64le -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-24-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-24-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-24-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-24-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-25-aarch64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-25-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-25-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-25-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-25-ppc64le -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-25-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-25-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-25-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-25-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-aarch64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-ppc64le -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-aarch64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-armhfp -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-ppc64le -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-s390x -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-7-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-8-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-8-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-8-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-8-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-8-primary-original -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-8-x86_64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-9-i386 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-9-ia64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-9-ppc -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-9-ppc64 -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-9-primary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-9-primary-original -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-9-secondary -fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-9-x86_64 -fedora-repos-27-0.1.noarch/etc/yum.repos.d -fedora-repos-27-0.1.noarch/etc/yum.repos.d/fedora-cisco-openh264.repo -fedora-repos-27-0.1.noarch/etc/yum.repos.d/fedora-updates-testing.repo -fedora-repos-27-0.1.noarch/etc/yum.repos.d/fedora-updates.repo -fedora-repos-27-0.1.noarch/etc/yum.repos.d/fedora.repo - -$ gpg2 fedora-repos-27-0.1.noarch/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-primary -pub rsa4096 2017-02-21 [SCE] - 860E19B0AFA800A1751881A6F55E7430F5282EE4 -uid Fedora 27 (27) - pub 4096R/E8E40FDE 2010-01-19 Fedora (13) -.... - -You may wish to do this in a tempoary directory to make cleaning it up -easy. - -==== getfedora.org - -One can simply browse to https://getfedora.org/keys to verify that the -key has been uploaded. - -==== sigulsign_unsigned - -The best way to test whether or not the key has been added correctly is -to sign a package using the key, like our newly built fedora-repos -package. - -.... -$ ./sigulsign_unsigned.py fedora-13 fedora-release-13-0.3 -Passphrase for fedora-13: -.... - -The command should exit cleanly. - -==== Public key servers - -One can use the search-keys command from gpg to locate the -key on the public server: - -.... -$ gpg2 --search-keys "Fedora (13)" -gpg: searching for "Fedora (13)" from hkp server subkeys.pgp.net -(1) Fedora (13) - 4096 bit RSA key E8E40FDE, created: 2010-01-19 -... -.... - -==== Koji - -Log into koji02.phx2.fedoraproject.org by way of -bastion.fedoraproject.org. - -Verify that `/etc/koji-gc/koji-gc.conf` has the new key in it. - -=== Consider Before Running - -Nothing at this time. diff --git a/modules/release_guide/pages/sop_deprecate_ftbfs_packages.adoc b/modules/release_guide/pages/sop_deprecate_ftbfs_packages.adoc deleted file mode 100644 index 1b01625..0000000 --- a/modules/release_guide/pages/sop_deprecate_ftbfs_packages.adoc +++ /dev/null @@ -1,122 +0,0 @@ -== Deprecate FTBFS Packages - -=== Description - -[NOTE] -==== -FTBFS = "Fails To Build From Source" -==== - -Every release prior to the Feature Freeze we deprecate all packages that -https://fedoraproject.org/wiki/Fails_to_build_from_source[FTBFS]. This -keeps out software that no longer builds from source, and prevents -future problems down the road. - -=== Action - -The FTBFS process takes place in stages: - -. Detecting a list of FTBFS packages and the dependencies that will be -broken if they are removed. -. Sending the list of potential deprecated FTBFS packages to -devel@lists.fedoraproject.org for community review and removal from the -FTBFS list by fixing the package. -. Removing packages confirmed as FTBFS from the Fedora package -repositories. - -==== Detecting FTBFS - -We will remove packages that have failed to build for at least two -release cycles. For example, in preparation for Fedora 21 branching, -packages which FTBFS since the Fedora 19 cycle (i.e. packages that have -a dist tag of fc18 or earlier) will be considered candidates for -removal. Adjust -https://pagure.io/releng/blob/main/f/scripts/find_FTBFS.py[find_FTBFS.py] -and run it to get a list of candidate packages. - -Given a candidate list from above, rel-eng should attempt to build each -of the candidate packages using koji. Should package building now -succeed, the package may be removed from the candidate list. - -==== Announcing Packages to be Deprecated - -Email the output to the development list -(`devel@lists.fedodraproject.org`) at least a week before the feature -freeze. This gives maintainers an opportunity to fix packages that are -important to them. Follow-up on the list where necessary. - -==== Retiring FTBFS packages - -Once maintainers have been given an opportunity to pick up and fix FTBFS -packages, the remaining packages are `retired` by blocking them, and -creating the `dead.package` file in git. - -===== GIT and Package DB - -Required permissions: provenpackage for GIT, cvsadmin for Package DB. - -We just have to remove the existing files from the `rawhide` branch and -replace them with a `dead.package` file whose contents describe why the -package is dead. Also the package needs to be marked as retired in -PackageDB. Fedpkg takes care of this: - -For example, if we wished to clean up git for the roxterm package we -would: - -.... -$ fedpkg clone roxterm -$ cd roxterm -$ fedpkg retire "Retired on $(date -I), because it failed to build for two releases (FTBFS Cleanup)." -.... - -===== Koji - -Required permissions: admin in koji if the automatic blocking fails. - -Blocking should happen automatically a few minutes after the packags was -retired in PackageDB. If it does not, use the `block-pkg` `koji` command -is used to do the blocking. - -Koji accepts multiple package names as input and thus we can use the -FTBFS package list as input. Deprecated packages are only blocked from -the latest `f##` tag. For example, if we wanted to `deprecate` (block) -`sbackup, roxterm,` and `uisp` from rawhide during the development of -Fedora 21 we would run the following command: - -.... -$ koji block-pkg f21 sbackup roxterm uisp -.... - -===== Bugs - -This procedure probably leaves open bugs for the deprecated packages -behind. It is not within the scope of releng to take care of these. If -bugs are closed, only bugs targeted at Rawhide should be affected, since -other branches might still be maintained. - -=== Verification - -To verify that the packages were blocked correctly we can use the -`latest-pkg` `koji` action. - -.... -$ koji latest-pkg dist-f16 wdm -.... - -This should return nothing, as the `wdm` package is blocked. - -Also check that package DB shows that the package is retired and that -the rawhide branch contains only a dead.package file. - -=== Consider Before Running - -Generally we block anything that doesn't leave broken dependencies. If -there are packages whose removal would result in broken dependencies a -second warning should be sent to devel@lists.fedoraproject.org and to --owner@fedoraproject.org for each dependent package. - -Allow another couple of days for maintainers to take notice and fix -these package so the package repository can be maintained without broken -dependencies or needing to deprecate the package. It is not good to have -broken package dependencies in our package repositories so every effort -should be made to find owners or to fix the broken dependencies. diff --git a/modules/release_guide/pages/sop_mass_rebuild_modules.adoc b/modules/release_guide/pages/sop_mass_rebuild_modules.adoc deleted file mode 100644 index 8b183a8..0000000 --- a/modules/release_guide/pages/sop_mass_rebuild_modules.adoc +++ /dev/null @@ -1,120 +0,0 @@ -== Mass Rebuild of Modules - -=== Description - -Periodically we do mass rebuilds of modules in rawhide during the -development cycle. This SOP will outline the steps necessary to do this. - -=== Assumptions - -This assumes that the mass rebuild has already been approved and -scheduled via release engineering and FESCo. Coordinate with -infrastructure as well for any needed updates. - -=== Considerations - -* The most important thing to keep in mind while doing a mass rebuild is -to communicate clearly what actions are being performed and the status -of the rebuild. -* Check in on scripts frequently to avoid a long stalled command from -adding significant delays in completing the rebuild. - -=== Actions - -==== Preparatory Steps - -The following steps should be completed after the -https://docs.pagure.org/releng/sop_mass_rebuild_packages.html[mass -rebuild of packages] is done. - -. Update Scripts - -The mass rebuild depends on two main scripts from the -https://pagure.io/releng[releng git repository]. Each one requires some -changes in variables for each new mass rebuild cycle. - -____ -* {blank} -+ -_mass-rebuild-modules.py_:: - ** rebuildid -* {blank} -+ -_massrebuildsinfo.py_:: - ** module_mass_rebuild_epoch - ** module_mass_rebuild_platform -____ - -Change the following items: - -* the `rebuildid` to match the release for which you are mass rebuilding -modules as per in massrebuildsinfo.py -* `module_mass_rebuild_epoch` mostly will be the epoch of mass rebuild -of packages -* `module_mass_rebuild_platform` should be the rawhide module platform - -==== Starting the Mass Rebuild of Modules - -The `mass-rebuild-modules.py` script takes care of: - -* Discovering available available modules from PDC -* Find the module info from mbs and check if a module build is submitted -after the epoch date -* Checking out modules from dist-git -* Switching to appropriate stream -* Find modulemd file -* Use libmodulemd to determine if this module stream applies to this -platform version -* If needs rebuilding, committing the change -* Push the commit -* Submitting the build request through mbs - -. Connect to the mass-rebuild Machine -+ -____ -.... -$ ssh compose-branched01.iad2.fedoraproject.org -.... -____ -. Start a terminal multiplexer -+ -____ -.... -$ tmux -.... -____ -. Clone or checkout the latest copy of the -https://pagure.io/releng[releng git repository]. -. Run the [.title-ref]#mass-rebuild-modules.py# script from -_releng/scripts_ -+ -____ -.... -$ cd path/to/releng_repo/scripts -$ ./mass-rebuild-modules.py build --wait 2>&1 | tee ~/massbuildmodules.out -.... -____ - -[NOTE] -==== -The token file should be located in infra's private ansible repo, or ask -infra to get it to you using this -https://pagure.io/fedora-infrastructure/issue/8048#comment-587789[process]. -==== - -[NOTE] -==== -The [.title-ref]#build# option is really important to pay attention, -since the mass branching of modules will also use the same script, just -changing the option to [.title-ref]#branch# and -[.title-ref]#module_mass_branching_platform# in -[.title-ref]#massrebuildsinfo.py# -==== - -==== Post Mass Rebuild Tasks - -Once the module mass rebuild is done, send an email to the -devel-announce@ list - -. Send the final notification to the -_devel-announce@lists.fedoraproject.org_ list diff --git a/modules/release_guide/pages/sop_mass_rebuild_packages.adoc b/modules/release_guide/pages/sop_mass_rebuild_packages.adoc deleted file mode 100644 index 9cf3ca8..0000000 --- a/modules/release_guide/pages/sop_mass_rebuild_packages.adoc +++ /dev/null @@ -1,286 +0,0 @@ -== Mass Rebuild - -=== Description - -Periodically we do mass rebuilds of rawhide during the development -cycle. This SOP will outline the steps necessary to do this. - -=== Assumptions - -This assumes that the mass rebuild has already been approved and -scheduled via release engineering and FESCo. Coordinate with -infrastructure as well for any needed koji updates. - -This also assumes that the mass rebuild does not need to be done in -dependency order, and that the mass rebuild does not involve a ABI -change. - -=== Considerations - -* The most important thing to keep in mind while doing a mass rebuild is -to communicate clearly what actions are being performed and the status -of the rebuild. -* Check in on scripts frequently to avoid a long stalled command from -adding significant delays in completing the rebuild. -* Check with secondary arches, whether they up-to-date enough with -primary, create rebuild tag and target when they are. It will then take -care of rebuilds of the arch specific packages in appropriate kojis. - -=== Actions - -==== Preparatory Steps - -The following steps may be completed in the weeks leading up to the -scheduled mass rebuild. - -. Create the Mass Rebuild Pagure Issue -+ -____ -Create an issue on the https://pagure.io/releng/issues[Release -Engineering issues page] that points at the schedule for the current -release. - -See https://pagure.io/releng/issue/6898[the Fedora 27 mass rebuild issue -example]. -____ -. Set up the Mass Rebuild Wiki Page -+ -____ -The mass rebuild wiki page should answer the following questions for -maintainers: - -* Why the mass rebuild is happening -* How to opt out of the mass rebuild - -[NOTE] -==== -See https://fedoraproject.org/wiki/Fedora_26_Mass_Rebuild[the Fedora 26 -Wiki example]. -==== -____ -. Send out the Mass Rebuild Notice -+ -____ -Send out the same information posted on the wiki to the -[.title-ref]#devel-announce@lists.fedoraproject.org# mailing list. - -[NOTE] -==== -See -https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org/message/QAMEEWUG7ND5E7LQYXQSQLRUDQPSBINA/[the -Fedora 26 e-mail example]. -==== -____ -. Create a Tag to Contain the Mass Rebuild -+ -____ -Mass rebuilds require their own tag to contain all related builds. The -example assumes we are doing a rebuild for Fedora 26. - -.... -$ koji add-tag f26-rebuild --parent f26 -.... -____ -. Request Package Auto-Signing for New Mass-Rebuild Tag -+ -____ -File a ticket with https://pagure.io/fedora-infrastructure/issues[Fedora -Infrastructure] requesting the new mass-rebuild tag be enabled for -package auto-signing. -____ -. Create the Koji Target for the Mass Rebuild -+ -____ -Using the same [.title-ref]#f26-rebuild# tag created in the previous -example: - -.... -$ koji add-target f26-rebuild f26-build -.... - -[NOTE] -==== -*koji add-target* _target-name_ _buildroot-tag_ _destination-tag_ -describes the syntax format above. If the _destination-tag_ is not -specified then it will be the same as the _target-name_. -==== -____ -. Update Scripts -+ -____ -The mass rebuild depends on four main scripts from the -https://pagure.io/releng[releng git repository]. Each one requires some -changes in variables for each new mass rebuild cycle. - -* {blank} -+ -mass-rebuild.py:: - ** buildtag - ** targets - ** epoch - ** comment - ** target -* {blank} -+ -find-failures.py:: - ** buildtag - ** desttag - ** epoch -* mass-tag.py -* {blank} -+ -need-rebuild.py:: - ** buildtag - ** target - ** updates - ** epoch -____ - -Change the following items: - -* the build tag, holding tag, and target tag should be updated to -reflect the Fedora release you're building for -* the `epoch` should be updated to the point at which all features that -the mass rebuild is for have landed in the build system (and a newRepo -task completed with those features) -* the comment which is inserted into spec changelogs - -==== Starting the Mass Rebuild - -The `mass-rebuild.py` script takes care of: - -* Discovering available packages in koji -* Trimming out packages which have already been rebuilt -* Checking out packages from git -* Bumping the spec file -* Committing the change -* git tagging the change -* Submitting the build request to Koji - -. Connect to the mass-rebuild Machine -+ -____ -.... -$ ssh branched-composer.phx2.fedoraproject.org -.... -____ -. Start a terminal multiplexer -+ -____ -.... -$ tmux -.... -____ -. Clone or checkout the latest copy of the -https://pagure.io/releng[releng git repository]. -. Run the mass-rebuild.py script from _releng/scripts_ -+ -____ -.... -$ cd path/to/releng_repo/scripts -$ ./mass-rebuild.py 2>&1 | tee ~/massbuild.out -.... -____ - -==== Monitoring Mass Rebuilds - -The community has a very high interest in the status of rebuilds and -many maintainers will want to know if their build failed right away. The -`find-failures.py` and `need-rebuild.py` scripts are designed to update -publicly available URLs for stakeholders to monitor. - -. Connect to a Compose Machine -+ -____ -.... -$ ssh compose-x86-02.phx2.fedoraproject.org -.... -____ -. Start a terminal multiplexer -+ -____ -.... -$ tmux -.... -____ -. Clone or checkout the latest copy of the -https://pagure.io/releng[releng git repository] -. {blank} -+ -Set Up the Rebuild Failures Notification Web Site:: - The `find_failures.py` script discovers attempted builds that have - failed. It lists those failed builds and sorts them by package owner. - + -.... -$ while true; do ./find_failures.py > f26-failures.html && cp f26-failures.html /mnt/koji/mass-rebuild/f26-failures.html; sleep 600; done -.... -. Start a second pane in the terminal emulator -. {blank} -+ -Set up the Site for Packages that Need Rebuilt:: - The `need-rebuild.py` script discovers packages that have not yet been - rebuilt and generates an html file listing them sorted by package - owner. This gives external stakeholders a rough idea of how much work - is remaining in the mass rebuild. - + -.... -$ while true; do ./need-rebuild.py > f26-need-rebuild.html && cp f26-need-rebuild.html /mnt/koji/mass-rebuild/f26-need-rebuild.html; sleep 600; done -.... - -==== Testing the Mass Rebuild - -Fedora openQA's update test suite can be run on the mass rebuild tag -before it is merged, to find any major bugs it may cause. The command -to trigger tests is `fedora-openqa tag `, -e.g. `fedora-openqa tag f39-rebuild 39`. This can be run as root -directly on an openQA server (preferably the staging instance), or -from any system with an appropriate configuration in -`~/.config/openqa/client.conf`, as documented https://pagure.io/fedora-qa/fedora_openqa[here]. -The link to the results will appear as the first entry under Fedora -Updates on the front page of the web UI (at time of creation, any -updates that appear subsequently will go top of the list) with a name -like "BuildTAG_f39-rebuild". On the results page you see when clicking -this, green dots indicate passed tests, red dots indicate failed tests, -yellow dots indicate 'softfailed' tests (these are probably fine), and -blue dots indicate scheduled/running tests. Grey dots indicate a test -that was cancelled (probably because a parent or sibling test failed). -If you see all green (or green and yellow), things are good. All green/ -yellow and blue, wait for the blues to change color. Any red or grey, -call for help from a QA person to interpret the results (and don't -merge anything yet). - -Contact an openQA admin (look for the tall, dashingly good-looking ones -in https://matrix.to/#/#quality:fedoraproject.org[Fedora Quality chat]) -for help scheduling the tests and interpreting the results if you are -unsure. - -==== Post Mass Rebuild Tasks - -Once the mass rebuild script completes, and all the pending builds have -finished, the builds will need to be tagged. The `mass-tag.py` script -will accomplish this task. The script will: - -* Discover completed builds -* Trim out builds that are older than the latest build for a given -package -* Tag remaining builds into their final destination (without generating -email) - -. Clone or checkout the latest copy of the -https://pagure.io/releng[releng git repository] -. Run the `mass-tag.py` script (requires koji kerberos authentication) -+ -____ -.... -$ cd path/to/releng_repo/scripts -$ ./mass-tag.py --source f36-rebuild --target f36 -.... -____ -. Send the final notification to the -_devel-announce@lists.fedoraproject.org_ list -+ -____ -The contents should look something like this -https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/QAMEEWUG7ND5E7LQYXQSQLRUDQPSBINA/[example -email]. -____ diff --git a/modules/release_guide/pages/sop_pdc.adoc b/modules/release_guide/pages/sop_pdc.adoc deleted file mode 100644 index eb233c2..0000000 --- a/modules/release_guide/pages/sop_pdc.adoc +++ /dev/null @@ -1,7 +0,0 @@ -== Product Definition Center - -This is a stub SOP for PDC. The doc really lives in the infra SOPs, -here: https://infrastructure.fedoraproject.org/infra/docs/pdc.rst - -Please update and maintain that SOP. Leave this document here as a -breadcrumb to make it easier to find that doc. diff --git a/modules/release_guide/pages/sop_promoting_container_content.adoc b/modules/release_guide/pages/sop_promoting_container_content.adoc deleted file mode 100644 index 146bffc..0000000 --- a/modules/release_guide/pages/sop_promoting_container_content.adoc +++ /dev/null @@ -1,50 +0,0 @@ -== Promoting Container Content - -=== Description - -Even though the promotion of content is aimed to be fully automated, -sometimes there is the need or desire to promote content from the -candidate registry to the stable registry. Below is how to accomplish -this using https://github.com/projectatomic/skopeo[skopeo]. - -=== Action - -This action should be performed on -`compose-x86-01.phx2.fedoraproject.org` by an user who is a member of -the `sysadmin-releng` https://admin.fedoraproject.org/accounts/[FAS] -Group. - -The container image should be provided by the requester, they will have -the information from their container build in the -https://docs.pagure.org/releng/layered_image_build_service.html[Fedora -Layered Image Build System]. It will have a name resembling -`candidate-registry.fedoraproject.org/f26/foo:0.1-1.f26container`. - -Sync content between the candidate registry and production registry, -effectively "promoting it". Substitute the `$IMAGE_NAME` in the -following example with whatever the actual image name is that is -provided. (This would be everything after -`candidate-registry.fedoraproject.org`, so using the above example then -`$IMAGE_NAME` would be `f26/foo:0.1-1.f26container`.) - -.... -$ sudo skopeo copy \ - --src-cert-dir /etc/docker/certs.d/candidate-registry.fedoraproject.org/ \ - --dest-cert-dir /etc/docker/certs.d/registry.fedoraproject.org/ \ - docker://candidate-registry.fedoraproject.org/$IMAGE_NAME \ - docker://registry.fedoraproject.org/$IMAGE_NAME -.... - -=== Verification - -In order to verify, we need to inspect the stable registry, again with -https://github.com/projectatomic/skopeo[skopeo] to ensure the image -metadata exists. - -.... -$ skopeo inspect docker://registry.fedoraproject.org/$IMAGE_NAME -.... - -In this JSON output you will see a list element titled `RepoTags` and in -there should be the `$VERSION-$RELEASE` listed there, following our -example above this entry would be `0.1-1.f26container`. diff --git a/modules/release_guide/pages/sop_rawhide_bodhi.adoc b/modules/release_guide/pages/sop_rawhide_bodhi.adoc deleted file mode 100644 index 8dec689..0000000 --- a/modules/release_guide/pages/sop_rawhide_bodhi.adoc +++ /dev/null @@ -1,105 +0,0 @@ -include::_partials/attributes.adoc[] - -== Enabling Rawhide in Bodhi - -=== Description - -This SOP covers the steps needed to enable Rawhide in Bodhi. - -==== Create the release in Bodhi - -In oder to start creating updates in Bodhi for rawhide, the release -needs to be created in Bodhi. Rawhide in Bodhi is represented by the -Fedora version (ie Fedora 31), but it is set in the prerelease state. - -===== Add the koji tags - -[source,subs="attributes+"] -.... -$ koji add-tag --parent f{rawhide} f{rawhide}-updates-candidate -$ koji add-tag --parent f{rawhide} f{rawhide}-updates-testing -$ koji add-tag --parent f{rawhide}-updates-testing f{rawhide}-updates-testing-pending -$ koji edit-tag --perm autosign f{rawhide}-updates-testing-pending -$ koji add-tag --parent f{rawhide} f{rawhide}-updates-pending -$ koji add-tag --parent f{rawhide} f{rawhide}-override -.... - -===== Change the koji targets - -[source,subs="attributes+"] -.... -$ koji edit-target f{rawhide} --dest-tag f{rawhide}-updates-candidate -$ koji edit-target f{rawhide}-candidate --dest-tag f{rawhide}-updates-candidate -$ koji edit-target rawhide --dest-tag f{rawhide}-updates-candidate -.... - -===== Create the release in bodhi - -[source,subs="attributes+"] -.... -$ bodhi releases create --name "F{rawhide}" --long-name "Fedora {rawhide}" --id-prefix FEDORA --version {rawhide} --branch f{rawhide} \ - --dist-tag f{rawhide} --stable-tag f{rawhide} --testing-tag f{rawhide}-updates-testing --candidate-tag f{rawhide}-updates-candidate \ - --pending-stable-tag f{rawhide}-updates-pending --pending-testing-tag f{rawhide}-updates-testing-pending \ - --state pending --override-tag f{rawhide}-override --create-automatic-updates --not-composed-by-bodhi -.... - -The important flags are [.title-ref]#--not-composed-by-bodhi# which -tells bodhi not to include the rawhide updates in the nightly pushes and -[.title-ref]#--create-automatic-updates# which tells bodhi to -automatically create an update listen to koji tag (build tagged with the -pending-testing-tag) messages. - -===== Bodhi configuration - -Bodhi is configured to required zero mandatory days in testing for the -rawhide release. This is done in ansible -roles/bodhi2/base/templates/production.ini.j2 with the following. - -[source,subs="attributes+"] -.... -f{rawhide}.pre_beta.mandatory_days_in_testing = 0 -.... - -===== Robosignatory configuration - -Robosignatory needs to be configured to signed the rawhide builds before -these builds are tested by the CI pipeline. - -[source,subs="attributes+"] -.... -{ - "from": "f{rawhide}-updates-candidate", - "to": "f{rawhide}-updates-testing-pending", - "key": "fedora-{rawhide}", - "keyid": "3c3359c4" -}, -.... - -==== Branching Rawhide - -When it is time to branch rawhide, a new release should be created (eg. -F32) following the steps above. The old rawhide release (eg. F31) should -stay configured as rawhide until we active Bodhi for it (2 weeks later). -To activate Bodhi on the old rawhide (eg. F31) the existing release in -bodhi should be modified has follow. - -[source,subs="attributes+"] -.... - $ bodhi releases edit --name "F{rawhide}" --stable-tag f{rawhide}-updates - --no-create-automatic-updates --composed-by-bodhi -.... - -===== Robosignatory configuration - -At Bodhi activation time the Robosignatory configuration needs to be -update to match the normal configuration of bodhi releases. - -[source,subs="attributes+"] -.... -{ - "from": "f{rawhide}-signing-pending", - "to": "f{rawhide}-updates-testing-pending", - "key": "fedora-{rawhide}", - "keyid": "3c3359c4" -}, -.... diff --git a/modules/release_guide/pages/sop_release_container_base_image.adoc b/modules/release_guide/pages/sop_release_container_base_image.adoc deleted file mode 100644 index f2366ea..0000000 --- a/modules/release_guide/pages/sop_release_container_base_image.adoc +++ /dev/null @@ -1,79 +0,0 @@ -include::_partials/attributes.adoc[] - -== Release the Fedora Container Base Image - -=== Description - -This SOP covers the steps involved in changing and releasing the Fedora -Container Base image. - -Fedora releases 2 container base images, [.title-ref]#fedora# and -[.title-ref]#fedora-minimal#. These images are available on 3 registries -[.title-ref]#registry.fedoraproject.org#, [.title-ref]#quay.io# and -[.title-ref]#DockerHub# (fedora-minimal is not available in DockerHub). - -==== Modify a base image (Kickstart) - -Base images are built in koji using the [.title-ref]#image-factory# -application to build the container image root filesystem (rootfs). -Kickstart files are used to configure how the image is built and what is -available in the image The solution consist of 3 Kickstarts. - -https://pagure.io/fedora-kickstarts/blob/main/f/fedora-container-common.ks[fedora-container-common] - -https://pagure.io/fedora-kickstarts/blob/main/f/fedora-container-base.ks[fedora-container-base] - -https://pagure.io/fedora-kickstarts/blob/main/f/fedora-container-base-minimal.ks[fedora-container-base-minimal] - -Changes made on the rawhide branch will results in the rawhide image, -other branches (f{current}, f{old_release}) should be used to modify other releases. - -==== Compose Configuration (Pungi) - -The configuration used to compose the container images is available in -the pungi-fedora repository. - -For rawhide the configuration is in - -https://pagure.io/pungi-fedora/blob/main/f/fedora.conf - -While for other releases the configuration is in a dedicated file - -https://pagure.io/pungi-fedora/blob/f31/f/fedora-container.conf - -==== Release on registry.fedoraproject.org and quay.io - -If you want to release the base image on registry.fp.o and quay.io you -can use the following script. - -https://pagure.io/releng/blob/main/f/scripts/sync-latest-container-base-image.sh[sync-latest-container-base-image.sh] - -You will need to run that script from on of the releng composer machines -in the infrastructure in order to have the credentials. - -If you do not have access to that machines, you can request the release -by opening a ticket on the https://pagure.io/releng/issues[releng -tracker]. - -The script can then be executed as follow - -[source,subs="attributes+"] -.... -$./sync-latest-container-base-image.sh {current} -$./sync-latest-container-base-image.sh {branched} -.... - -This will take care of pushing the [.title-ref]#fedora# and -[.title-ref]#fedora-minimal# images to both registries. - -==== Release on DockerHub - -Releasing on DockerHub is a little different since Fedora is an -"offical" image there. In order to release new images there we have to -update a Dockerfile and rootfs tarball on the following repo. - -https://github.com/fedora-cloud/docker-brew-fedora[docker-brew-fedora]. - -For the details on how to run the script please see - -https://github.com/fedora-cloud/docker-brew-fedora/blob/main/README.md[README]. diff --git a/modules/release_guide/pages/sop_release_package_signing.adoc b/modules/release_guide/pages/sop_release_package_signing.adoc deleted file mode 100644 index 95cc706..0000000 --- a/modules/release_guide/pages/sop_release_package_signing.adoc +++ /dev/null @@ -1,68 +0,0 @@ -== Release Package Signing - -=== Description - -For each of Fedora's public releases (Alpha, Beta, and Final) it is -Release Engineering's responsibility to sign all packages with Fedora's -GPG key. This provides confidence to Fedora's users about the -authenticity of packages provided by Fedora. - -The `/sop_create_release_signing_key` document explains the process for -creating the GPG key used. - -=== Consider Before Running - -This script takes a very long time to run, as much as 4 or 5 days, so it -needs to be started well in advance of when you need the packages all -signed. - -Signing all the packages will cause a lot of churn on the mirrors, so -expect longer than usual compose and rsync times, as well as potential -issues with proxies as file contents change but the name remains the -same. - -=== Action - -. Log into a system with `sigul` and start a `screen` or `tmux` session. -The signing process takes a long time--screen allows the process to -continue if you session gets disconnected. -+ -.... -$ screen -S sign -.... -+ -or -+ -.... -$ tmux new -s sign -.... -. Check out the Release Engineering `git` repo -+ -:::: - $ git clone https://pagure.io/releng.git -. Change directories to the `scripts` directory to execute -`sigulsign_unsigned.py`. -+ -For example, to sign everything for Fedora 13 Alpha we would issue: -+ -:::: - $ ./sigulsign_unsigned.py -vv --tag dist-f13 fedora-13 -+ -This signs the packages with verbose output so you can track progress -incrementally. - -=== Verification - -Once the signing is done, use `rpmdev-checksig` to verify that a package -has been signed. You can use the output of a recent rawhide compose to -test. In this example we use a released Fedora 12 package: - -.... -$ rpmdev-checksig /pub/fedora/linux/releases/12/Everything/i386/os/Packages/pungi-2.0.20-1.fc12.noarch.rpm -/pub/fedora/linux/releases/12/Everything/i386/os/Packages/pungi-2.0.20-1.fc12.noarch.rpm: MISSING KEY - 57bbccba -.... - -This output shows that the apckage was signed with key `57bbccba`, and -that this key does not exist in your local rpm database. If the key did -exist in the local rpm database it's likely there would be no output so -it's best to run this on a system that does not have gpg keys imported. diff --git a/modules/release_guide/pages/sop_stage_final_release_for_mirrors.adoc b/modules/release_guide/pages/sop_stage_final_release_for_mirrors.adoc deleted file mode 100644 index d51676b..0000000 --- a/modules/release_guide/pages/sop_stage_final_release_for_mirrors.adoc +++ /dev/null @@ -1,41 +0,0 @@ -include::_partials/attributes.adoc[] - -== Stage Final Release for Mirrors - -=== Description - -When the release has been fully tested and approved at the "Go/No-Go" -meeting it is ready for release to the Fedora mirrors. - -=== Action - -. Gather the needed info for running the staging script: Release -Version: the numerical version number of the release `{branched}` ComposeID: The -ID of the Compose Label: Compsoe label for the location in stage -`{branched}_RC-1.2` for example Key: the name of teh release key `fedora-{branched}` or -`fedora-{branched}-secondary` as examples Prerelease: 0 or 1 sets if the release -goes in test/ or not Arch: For secondary arches, changes some -internal locations -+ -[source,subs="attributes+"] -.... -$ scripts/stage-release.sh {branched} Fedora-{branched}-20160614.0 {branched}_RC-1.2 fedora-24 0 -.... -. Sync the release to the Red Hat internal archive following internally -documented - -=== Check and set EOL on previous releases to reflect reality - -. check PDC for active releases and respective EOL date -. if needed run the adjust-eol-all.py script from releng repository to -correct any mistakes - -=== Verification - -Verification is somewhat difficult as one cannot look at the content via -the web server due to permissions. Typically we ask somebody from the -Infrastructure team to give the tree a second set of eyes. - -=== Consider Before Running - -Hope the release is good! diff --git a/modules/release_guide/pages/sop_updating_comps.adoc b/modules/release_guide/pages/sop_updating_comps.adoc deleted file mode 100644 index 4c015ba..0000000 --- a/modules/release_guide/pages/sop_updating_comps.adoc +++ /dev/null @@ -1,59 +0,0 @@ -include::_partials/attributes.adoc[] - -== Updating Comps - -=== Description - -When we start a new Fedora development cycle (when we branch rawhide) we -have to create a new comps file for the new release. This SOP covers -that action. - -=== Action - -. clone the comps repo -+ -.... -$ git clone ssh://git@pagure.io/fedora-comps.git -.... -. Create the new comps file for next release: -+ -[source,subs="attributes+"] -.... -$ cp comps-f{branched}.xml.in comps-f{rawhide}.xml.in -.... -. Edit Makefile to update comps-rawhide target -+ -[source,subs="attributes+"] -.... -- -comps-rawhide: comps-f{branched}.xml -- - @mv comps-f{branched}.xml comps-rawhide.xml -+comps-rawhide: comps-f{rawhide}.xml -+ @mv comps-f{rawhide}.xml comps-rawhide.xml -.... -. Add the new comps file to source control: -+ -[source,subs="attributes+"] -.... -$ git add comps-f{rawhide}.xml.in -.... -. Edit the list of translated comps files in po/POTFILES.in to reflect -currently supported releases. -+ -[source,subs="attributes+"] -.... --comps-f{old_release}.xml -+comps-f{rawhide}.xml -.... -. Send it up: -+ -:::: - $ git push - -=== Verification - -One can review the logs for rawhide compose after this change to make -sure the right comps file was used. - -=== Consider Before Running - -Nothing yet.