@kalev -- we can probably issue an update to gnome-software to poll something like https://pdc.fedoraproject.org/rest_api/v1/releases/ instead, before F24 goes EOL, right? This won't help F23 and previous users, but OTOH those would be unsupported.

We could put a Magazine post together to address more ancient Fedora users, if that helps.

Isn't there https://apps.fedoraproject.org/mdapi/branches (looks to be broken ATM :/ [1] )?

Isn't there https://apps.fedoraproject.org/mdapi/branches (looks to be broken ATM :/ [1] )?

I thought that was for packages as opposed to releases, but I could be wrong.

Is GNOME Software just hitting https://admin.fedoraproject.org/pkgdb/api/collections/? Could we simply replace that with redirect to a static json page listing up through F26 and keep that around basically forever?

Is GNOME Software just hitting https://admin.fedoraproject.org/pkgdb/api/collections/? Could we simply replace that with redirect to a static json page listing up through F26 and keep that around basically forever?

Also a possibility. But wouldn't that expire down the road too? Also, that file would respond to all hits on that expired API, and that could encourage people to use outdated docs rather than PDC (or whatever).

I mean this as a stopgap until PDC or wherever else can take up the slack, and GNOME Software on current releases updated to use that.

Looking at the mirror stats, I'm basically concerned with keeping the upgrade functionality available for people in F23 for the next or so, until that drops down to at least the level of F20-F22. Or same shifted back six months if we have to include F24 too because of not having a solution in place in time before F24 goes EOL.

Is GNOME Software just hitting https://admin.fedoraproject.org/pkgdb/api/collections/? Could we simply replace that with redirect to a static json page listing up through F26 and keep that around basically forever?

Also a possibility. But wouldn't that expire down the road too? Also, that file would respond to all hits on that expired API, and that could encourage people to use outdated docs rather than PDC (or whatever).

I think this is exactly what we need to do to make sure older Fedora releases where we can no longer push updates remain upgradable, and then for newer Fedora releases we can then push a gnome-software update that switches it to using the PDC json API.

Offhand it feels to me like it'd be saner to add the metadata into the rpm-md. This is a pretty small amount of information, and that way it'd get mirrored and managed the same way the other update metadata does, and no rely on a centralized service.

I filed an upstream bug on this: https://bugzilla.gnome.org/show_bug.cgi?id=786491
with a link to the PDC issue on why PDC isn't yet usable for this purpose:
https://github.com/product-definition-center/product-definition-center/issues/294

Adding @kevin and @pingou to this ticket -- I don't think we've set a date yet for pkgdb to be turned off, but it seems like we should still likely handle this case for users who install an older or even EOL (but still upgradeable) release after we do, with a redirect. We can add the JSON output to the fp.o website alongside other static content, trivially.

See also: https://pagure.io/fedora-infrastructure/issue/6267

(lets track this in every issue tracker! :)

I'd be ok making a hand crafted artisinal json or something as a stop gap, but I am very strongly against keeping it around. We had this before with preupgrade and it was error prone and labor intensive and often out of sync with other things.

FWIW, we now have a stopgap in the PDC data so that nothing needs to change until F28. We're already discussing with PDC team to get this into one of their sprints before spring.

https://github.com/product-definition-center/product-definition-center/pull/482

The PR is open upstream for a while now to address this issue. I pinged to try and rouse a review.

@kalev, what's the status of this? I notice that distro upgrades are already working fine for F28, and even the artwork is ready on time for a change.

@catanzaro -- AIUI they're working because we're using the "hand crafted artisinal json" stopgap, still.

Yes, that's correct. We haven't looked at switching to the new thing yet.

Keep tracking https://github.com/product-definition-center/product-definition-center/issues/294 - the PR has been merged so the new endpoint exists in PDC, but AFAIK, no-one has actually populated it with any data yet, or set up a releng SOP to ensure that it will always be promptly updated.

Sorry, I lost track of this ticket among other work -- can anybody tell me whether or how the upstream PDC deprecation affects GNOME Software being able to detect/run upgrades? Or is this no net change in Fedora for now?

gnome-software is still using the pkgdb json data. As far as I know, the PDC status in Fedora is still the same, that the data hasn't been populated. I don't know how the upstream PDC deprecation affects the instance in Fedora infrastructure. @nirik, do you know?

Yeah, I think it changes the plan for the thing we didn't actually implement yet, leaving us at the current state of "need manual work".

So, it's a bit of a mess. I have been trying to sort a way forward for the last few weeks in fact.

I was going to propose switching pkgdb off and then serving a static json file at the endpoint until the pdc replacement was ready then switch to that. However, I just learned that taskotron is still using pkgdb for crtipath, so we need to address that first.

So, there's 3 balls in the air:

• pkgdb that we want to shut off, but gnome-software and taskotron are using it still.
• pdc that lots of things are using
• pdc-ng/replacement which doesn't exist yet.

@kalev would gnome-software follow a redirect to the url it's using now? Or just the hard coded url it has now? If it can follow a redirect that would be helpful, as we could then just redirect it to a static file.

The plan for critpath was to just move it to bodhi as it's the only thing we though used it. However, since taskotron does, we need to figure that out again. Should it be in the pdc-replacement? Should bodhi have it and have a api to expose it for taskotron? I guess we need to get bodhi folks and taskatron folks and pdc replacement folks together.

I believe the idea is to move critpath to bodhi and expose it in the API, so taskotron could query it as needed.

I was going to propose switching pkgdb off and then serving a static json file at the endpoint until the pdc replacement was ready then switch to that. However, I just learned that taskotron is still using pkgdb for crtipath, so we need to address that first.

Couldn't we do that for both? Two json files instead of one?

From IRC minutes ago:

  kparal | pingout: it pulls critpath list from it, but doesn't really use it atm  
  kparal | we could simply disable it                                              

I believe the idea is to move critpath to bodhi and expose it in the API, so taskotron could query it as needed.

I was not sure there was a plan to make an API for it. I thought that we were thinking bodhi was the only thing using it, so it could just keep an internal copy. Hopefully an api to expose that isn't too hard.

I was going to propose switching pkgdb off and then serving a static json file at the endpoint until the pdc replacement was ready then switch to that. However, I just learned that taskotron is still using pkgdb for crtipath, so we need to address that first.

Couldn't we do that for both? Two json files instead of one?

Well, it's more than 2, it's 1 per branch... oh wait, ha. It's supposed to be per branch, but it seems to now only return f26 and master (both from the same endpoint).

kparal | pingout: it pulls critpath list from it, but doesn't really use it atm
kparal | we could simply disable it

That would be nice for now, so we don't have to worry about it. Once thats disabled I can set up a static json for gnome-software either on the current url or at a better one if it can handle redirects.
Then we can turn off pkgdb entirely and see what else breaks.

gnome-software currently uses pkgdb json API to get metadata about new Fedora releases. With planned pkgdb shutdown, gnome-software will have no way to download that information. We'll need two things:
1) make sure the pkgdb API (in frozen mode) is around in a static read-only mode so that people can keep upgrading from F24 and older (a year at least)
2) add new API somewhere and fix gnome-software in F25+ to use the new API

Well every recent distro upgrade has gone OK. I assume this issue is obsolete? Does anything here still require WG tracking?

No it doesn't mean that, it means we still didn't actually get rid of pkgdb json yet :/

It is now hand updated by releng every time we branch, cut a release, or a release goes EOL, because we live in a ridiculous world.

IIRC, PDC does now have the necessary features to provide sufficient lifecycle information on releases, but there still isn't a mechanism for actually updating it when the information changes. But I can't find the ticket any more...

OK but this seems like a releng problem, not a problem for the Workstation WG, yes? Unless there is a specific request for the WG to make a decision or take some action?

at present, yeah, but in theory at least the idea is still that the collections json will go away at some point and at that point gnome-software is going to need to work with pdc or something else. I've no idea if anyone is actively working this any more or what.

Right, there's a manually edited each release json file we keep around for this. ;(

We would love to... not do that. We currently have pdc, but its dead upstream and we are trying to replace it.
@cverna is exploring a simple replacement for it which we hope to have some kind of proof of concept up soon.

After that we will need gnome-software to query this new thing instead of our manual janky json blob.

Interestingly, I was having a discussion about this just a day ago with @hellcp about implementing this in openSUSE.

A potential idea for a simple replacement (for @cverna to consider) that came out of that discussion:

• A service that returns AppStream data about new OS upgrades (wired into packages? pdc? something else?). It would return a query of API collections and metainfo files to be fetched for gnome-software to use.

This would also make it easier for supporting across other tools...

Keep in mind, appstream has operating-system component type already, managing it separately in a way which would allow to perform upgrades could be a way to go.

For openSUSE, we have several prerequisites to take care of, because we have upgrades from leap-x.y to leap-x.y++ and leap-x.y to tumbleweed to actually handle. For leap that's fairly easy, we finally started using releasever for leap repositories, for tumbleweed upgrades the current plan is to do that through installing a repo package and doing an upgrade based on that. Generating appstream metadata with dependencies like that seem like the most distro and package management independent.

I think this has long since become a releng problem, and tracking here is no longer valuable.

If releng needs us to implement something in GNOME Software, please open a new issue once that's more well-defined.

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/workstation/tickets/issues/23

Please continue any further discussion there.