If the next Rawhide IoT compose goes through and behaves as expected we can merge this, I think.

LGTM

This turned out not to work so great due to bugs in Pungi.

I've sent a pull request for Pungi, but working on it and looking at metadata has also made me think harder about what these names/properties really should be.

The concept that the "subvariant" is supposed to express is...what's the payload of this thing? It's important to remember we also have the "type" and "format" properties. I'm not...super happy with how we've wound up using those in most cases, though; a lot of the time they are identical, or very similar. Classically, 'type' was something like 'live' or 'dvd' (it disambiguates different ISOs), but with newer images we've tended to make it something very literally close to the 'format', e.g. 'raw-xz' or 'vhd-compressed'.

For bootc images...I'm trying to think what a sensible plan is. Imagine we've got the generic bootc base image, and dozens of other bootc images for other purposes, maybe spread across different composes (Fedora, Fedora-IoT, Fedora-Container...). How do we want to identify them?

Right now, they get 'ociarchive' as both their 'type' and their 'format', as do all other ostree container images built by pungi (e.g. the Atomic desktop images). This just tells us, well, they're OCI container images, nothing else.

Do we want to use the subvariant field for all bootc image identification, as this PR proposed? Or would we maybe want to look at changing the 'type' to 'bootc' for bootc images (we'd need another patch for pungi to do this), and then using the subvariant only to indicate which bootc image this is? In that case, perhaps the 'generic' bootc base image could have subvariant 'Everything' or 'Generic' or something, then the IoT one could be just 'iot', a Server one could be just 'server', etc...

thoughts?

A huge part of the value of OCI containers is standardizing a format for the delivery and encapsulation of a runnable application. And on that topic

Do we want to use the subvariant field for all bootc image identification,

bootc containers already have such an identifier in the containers.bootc=1 label that is included in that standard format.

I am not so sure about the value of debating the exact meaning of the schema for composes (a thing that only Fedora derivatives use) versus just expecting callers to inspect the target artifacts on their own and parse the standardized metadata.

Or alternatively if it was convenient, including a full copy of the container manifest/config JSON in the pungi metadata.

you have to find the artifacts to inspect them. one of the functions of the compose metadata is precisely to provide information on what artifacts are in the compose, so you don't have to hack up some kind of tree parser with magic knowledge of file extensions or whatever to do it for you. that's the metadata we're concerned with here. you only need to know one magic thing (the location of images.json) to get a nice bunch of data on what 'images' the compose contains, where they all are, and various properties of each.

this isn't relevant when you're working with a container registry, of course, but that's not the context we're talking about here at all. this metadata doesn't exist in that context. it is relevant when you're writing the thing which takes the raw images from the compose and publishes them on the container registry. which I did. in that case, it's very helpful indeed to have some handy metadata which tells you what container images are in the compose. so that's why I care.

as long as we're still building stuff in pungi composes I am going to care about the compose metadata. when we stop doing pungi composes I'll stop caring about pungi compose metadata. if we build a shiny konflux pipeline that handles the whole shebang including building and publishing the container, that's great. until then, this is what we got.

Thanks for all your work! I am not trying to downplay or denigrate it.

For sure with raw disk images (e.g. ISOs, qcow2) there is absolutely no standards at all and things vary wildly per distribution. Also there isn't really "a" standard for "versioned snapshot of RPMs/debs" even either, so at the time it was made having pungi/compose tie those two together makes absolute sense, and continues to do so.

it's very helpful indeed to have some handy metadata which tells you what container images are in the compose. so that's why I care.

Absolutely. My point was specifically about extended metadata for those container images, which duplicates/overlaps with standard metadata that is available in those container images directly.

still, you need metadata to find the OCI container images at all. at which point it's rather more convenient to read the info on the image's contents from the same metadata rather than having to do some more metadata parsing to get it out of the image itself. If I'm already looking at the dict that tells me the image has type 'bootable-container' and format 'ociarchive', I may as well just go ahead and also read the subvariant field too.

I get your point that OCI images are better than other image types here, but our legacy compose process is always going to have to deal with all those image types - we're not gonna stop building ISOs any time soon - and it's usually going to be more work in that context to have separate codepaths than it is to just use the one that works for everything.

productmd images have to have a subvariant, so we may as well make it accurate.

embedding the OCI image metadata in the productmd metadata would require changing productmd, i think, it doesn't really have a mechanism for embedding arbitrary extended data AFAIK. it's a fairly prescriptive format.

anyway, I just realized we're in the old PR - we already merged https://pagure.io/fedora-iot/pungi-iot/pull-request/102 a couple of days ago. the intended state after that PR is merged is given in the commit message:

"So we will have these basename - subvariant - type - formats:

• Fedora-IoT-bootc - IoT - bootable-container - ociarchive
• Fedora-base-bootc - base - bootable-container - ociarchive"