Issue #12923: Packit receives Anubis HTML page - fedora-infrastructure

fedora-infrastructure

#12923 Packit receives Anubis HTML page

Closed: Fixed Dec 3, 2025 by kevin. Opened Nov 20, 2025 by mmassari.

Not always but sometimes (it seems when we are trying to run the Packit Fedora CI after a rebase in a Pagure repo) we get the HTML page instead of the packit.yaml config.

In all of ours log the traceback looks like the following but for different packages.

Message - This event
error
10:40:48.607 AM
  parser says
  in "<unicode string>", line 4, column 19:
                height: 100%;
                      ^
  mapping values are not allowed here
packit.config.package_config
error
10:40:48.605 AM
Cannot load package config None.
Httplib
info
12:50:43.433 AM
https://src.fedoraproject.org/fork/packit/rpms/awscli2/raw/33837e4c461f6a62bc305f5e1ff38737d8da6424/f/.packit.yaml
{
http.fragment: ,
http.method: GET,
http.query: ,
http.response.status_code: 200,
reason: OK,
thread.id: 140001251296832
}
Httplib
info
12:50:43.417 AM
https://src.fedoraproject.org/api/0/fork/packit/rpms/awscli2/tree/33837e4c461f6a62bc305f5e1ff38737d8da6424/
{
http.fragment: ,
http.method: GET,
http.query: ,
http.response.status_code: 200,
reason: OK,
thread.id: 140001251296832
}
Httplib
info
12:50:43.209 AM
https://src.fedoraproject.org/api/0/fork/packit/rpms/awscli2/tree/33837e4c461f6a62bc305f5e1ff38737d8da6424
{
http.fragment: ,
http.method: GET,
http.query: ,
http.response.status_code: 301,
reason: Moved Permanently,
thread.id: 140001251296832
}
Base project: PagureProject(namespace="rpms", repo="awscli2", username="packit", is_fork=True) owned by packit
{
asctime: 2025-11-20 00:50:42,979,
hostname: packit-worker-short-running-3,
isotime: 2025-11-20T00:50:42.979565Z,
msgid: packit--prod,
procid: 1,
sd: -,
task_info: task.steve_jobs.process_message[a980255e-477f-462c-aeeb-8b40bf82d97f]
}

kevin commented Nov 20, 2025

Does packit use a specific user-agent or headers ? Does it always connect from the same network/ip(s)?

I'll look in logs and see if I can find the above...

Metadata Update from @james:
- Issue priority set to: Waiting on Reporter (was: Needs Review)
- Issue tagged with: anubis, medium-gain, medium-trouble

Nov 20, 2025

mmassari commented Nov 21, 2025

@nforro added it: https://github.com/packit/packit-service/blob/96d8af5389225cc674f9cf9514146e61190f312b/packit_service/utils.py#L267-L270

We weren't able to properly test it. The change is applied, at the moment, only on our staging instance.
if you can use it, we should see the exceptions disappear from the staging instance and then we will apply the same change also on our production instance.

Thank you.

kevin commented Nov 21, 2025

So, take a look at what content resolver is doing:

https://github.com/fedora-eln/content-resolver/pull/100

In particular if you set Accept and User-Agent it should leave you alone...

If that doesn't work, we can look at trying to allowlist based on something...

mmassari commented Nov 26, 2025

We are finally sure we are adding headers in our requests (at least for the staging instance), but we are still getting the Anubis page :-(
These are the headers we have in the request when we get the Anubis page:
{'Authorization': '<redacted>', 'Accept': 'application/yaml', 'User-Agent': 'ogr/0.57.0.post1.dev7+g7524d05ce (hello@packit.dev)'}.

Do you have a whitelist of allowed user-agents? Or should these headers have worked as they are?

If you need to whitelist the user-agent, then adding ogr isn't the best choice since it is a library. So we should decide how to proceed and which user-agent string to use, based on how much effort we can put into it, and we will let you know.

nikromen commented Nov 27, 2025

I see this commit blocked anything behind /fork/
https://pagure.io/fedora-infra/ansible/c/ceb9a4f94b87b806d0658fb977779087a7450b19
even getting the raw files - so you'll get challenge every time. And the problem seems to occured in similar time as the patch.

Not always but sometimes...

probably because when you get the correct file content, it's probably not a forked repo, sitting behind this rule

I think the easiest way how to get through this (from Maja's view) would be adding something like

  - name: allow raw files
    path_regex: /raw/
    action: ALLOW

to the config. But that will allow it for literally everyone.

Do you have a whitelist of allowed user-agents?

I don't see any in the current policies. Allowing specific pre-agreed user-agents related to the fedora tooling/automation seems also like a reasonable thing to do.

If you need to whitelist the user-agent, then adding ogr isn't the best choice since it is a library.

yep... if you e.g. agree on packit-service/<version> then

bots:
  - name: allow trusted bots
    user_agent_regex: ^packit-service/
    action: ALLOW
# rest of the file

should work... WDYT about this @kevin ? I don't think these rules should hurt the protection provided by the config.

EDIT: I tried to curl some raw file that is in forked project with various headers, including user agent and accept and I can confirm that I am not able to get pass the anubis challenge

Edited Nov 27, 2025 by nikromen

kevin commented Nov 30, 2025

Is this always .packit.yaml ? or ?

I'd be ok allowlisting that one file, but all of /raw/ seems like it would be open to abuse by the scrapers. ;(

I guess user-agent would be ok...

nforro commented Dec 1, 2025

Is this always .packit.yaml ? or ?

No, it can be also spec file and .fmf/version (but there we care only about file existence, so Anubis HTML is fine). But in the future we might need to read more files. Also Packit config filename can have multiple forms: \.?packit\.ya?ml.

nforro commented Dec 1, 2025