#3471 Change: ptrace scope bpf jit harden
Closed: Invalid by salimma. Opened by amoloney.

This proposal aims to improve the security of Fedora and increase its resilience against attacks, including mitigating vulnerabilities and attacks that could be enabled by user behavior: targeting average users as of today, it is also our responsibility to implement measures to preemptively protect users and predict that users, e.g., enable third party repos from unknown parties or manually install packages or drivers that no longer get updates or are not implemented by best practices, or run scripts that are not trustworthy: the protection we can offer here is limited, but the kernel ships with several means that are intended to offer mitigation. To avoid misunderstandings: these means are not just to mitigate user-caused vulnerabilities/attacks but also to add additional security layers that can make a difference if others fail (e.g., vulnerabilities in our packages).

Enable kernel.yama.ptrace_scope, kernel.kptr_restrict and net.core.bpf_jit_harden by default in the kernel to increase the security of Fedora and mitigate some vulnerabilities and attacks. kernel.yama.ptrace_scope is already enabled by default upstream, but on Fedora it is currently disabled by default because of a package that accidentally became a dependency for default installations.

Enable ptrace_scope at value 1 (kernel default 1; or consider 2 if FESCo concludes this cannot cause issues), kernel.kptr_restrict at value 1 (or consider 2 if FESCo concludes this cannot cause issues) and bpf_jit_harden at 1 (or 2 if FESCo concludes this cannot cause issues). However, with regards to bpf_jit_harden, risks at 2 might be less predictable, and FESCo might be more careful before using 2: if someone comes up with reproducible reasons why to not set 2 as default or if FESCo cannot exclude noteworthy amounts of bpf_jit activities through privileged users in some use cases (2 = bpf_jit_harden is imposed both on privileged and unprivileged while 1 is imposed only on unprivileged), then 1 might be better for bpf_jit_harden: bpf_jit_harden only risks a performance decrease (respectively power-consumption increase) and only as far as it is used, but use cases of any of our targeted audiences involving privileged bpf_jit activities could be harder to determine/predict (respectively more realistic to occur) than use cases with unprivileged bpf_jit activities.

Theoretically, kernel.yama.ptrace_scope is already at 1, but the package elfutils-default-yama-scope overwrites this because it accidentally became a default package: therefore, add this package to fedora-obsolete-packages to ensure it is removed from all installations on their next update and then remove this package as dependency and remove it from the repository at all because the risk of accidentally re-reinstall it by default on Fedora installations remains, as we experience it at the moment. Further, it is common in the Fedora community, and propagated among users and supporters, to assume that installed tools/applications "do not hurt/impact" except using some space. This packages breaks with this common assumption and thus introduces further risks: ptrace_scope has the capacity to prevent malicious processes (e.g., vulnerable software or self-installed malicious software) to take control of other processes or their data.

Many Linux distributions keep ptrace_scope already by default (e.g., Ubuntu, Arch Linux, openSuSE), as it is set by the upstream kernel, and their users seem to be satisfied to keep this default (value 1, but also consider 2 for Fedora).

With regards to bpf_jit_harden, the Kernel Docs mention that formally, the settings 1 and 2 rather than 0 are performance trade-offs, but I could not identify a performance impact on my installation with 2 (Arch Linux already sets 2 by default in its hardened kernel) with average activities (surfing on the Internet/Firefox, watching movies in browser and vlc, editing/writing documents/code with git/kate/kwrite/nano/mcedit, emails, copying large amounts of files, working on battery, Internet on wifi & usb-ethernet), although as mentioned above, use cases that are impacted by 2 might be more realistic than 1 (FESCo might be able to evaluate this more reliably). Also, although Fedora and RHEL differ in many respects, it might be indicative that for security-sensitive RHEL installations, 2 is already suggested for "bpf_jit_harden", tested, and partly required, by DISA (see Discussion topic mentioned below, post #12, which suggests that practical cases as far as known have not identified performance/power-consumption issues).

Further, set kernel.kptr_restrict to either 1 (2 if FESCo concludes that this cannot cause negative impacts). Given its circumstances, I could not identify a negative impact that could affect one of our targeted audiences, although 2 might need a wider review by FESCo. This means can avoid the use of kernel exploits in some circumstances, and although in our case it could be possible that attackers might be able to mitigate kptr_restrict, the level of experience will change: in some circumstances, using a kernel exploit can be as easy as a google search with copy/paste, and be taught early in average (online) pentesting courses. Yet, the means that become necessary in cases that are related to kptr_restrict, deep expert knowledge might become necessary to use an exploit, if possible at all.

Details about the settings and what they can mitigate are elaborated in the Kernel Docs and in the Arch Wiki (whose arguments in this respect can be transferred to Fedora): See documentation section below.

Owners, do not implement this work until the FESCo vote has explicitly ended.
The Fedora Program Manager will create a tracking bug in Bugzilla for this Change, which is your indication to proceed. See the FESCo ticket policy and the Changes policy for more information.

REMINDER: This ticket is for FESCo members to vote on the proposal. Further discussion should happen in the Discourse discussion linked above. Additional discussion may happen on the Fedora Devel mailing list.


To avoid confusion and allow this ticket to be linked with its earlier discussions in search queries/SEO: this is the proposal that was discussed under the name
F44 Change Proposal: Mitigate vulnerabilities/attacks by enabling kernel.kptr_restrict and net.core.bpf_jit_harden by default, and by obsoleting a package that risks to accidentally disable kernel.yama.ptrace_scope by default [SystemWide]

I have the feeling that "if the title is so long that it spans across three lines of text" is a good indicator that the proposal is doing too much at once :grin:

I was not sure if the package removal takes place as separate "security fix/update" or not (and if so, how/to what extent), therefore I added it because it is a dependency of the actual proposal :) Ironic is that most discussions seem to have been about the package removal of a package, which was never intended to remain that way, and thus was only included as part of the proposal to ensure the actual change to be not broken by it :D However, it is also brought together a little to raise some awareness of this type of issue :) But I understand your point of course.

if FESCo agrees to also set ptrace_scope to 2 rather than the upstream kernel default

ptrace_scope=2 is a very bad idea overall. There is no security benefit from preventing ptracing of a process that is your child, since the caller generally has full control over what code and in what environment the child process is running.

30-yama-ptrace.conf would not be installed by default

This is not useful. Either install the file as 30-yama-ptrace.conf.example (see rpm -ql systemd-networkd|rg example for inspiration), or since it's a one line config, just include the instructions how to override the setting in 20-yama-scope.conf. Actually, the file already has those instructions, so just update/reverse them to match the new default value.

Summary

It's not a summary. It's an essay of its own. Please editorialize the document to have a short Summary and a more concise Detailed Description.

Scope

This section is supposed to contained an itemized list of things to do, so that people can see who does what and what remains to be done. It cannot fulfil this role when it contains multiple paragraphs of texts and a bunch of hypothetical scenarios. Please remove the hypotheticals and prose from this section, and also the itsy bitsy details of implementation.

I still think it'd be very nice if gdb, strace, and possibly other tools were modified to print an explanation if this setting prevents them from working. I think it'll save us a lot of surprised queries from developers who use Fedora but don't necessarily follow fedora-devel. And the patches should be fairly trivial, just a few self-contained lines at the point where the error is reported. If you don't want or cannot work on this yourself, you could at least reach out to the maintainers to see what they think.

ptrace_scope=2 is a very bad idea overall. There is no security benefit from preventing ptracing of a process that is your child, since the caller generally has full control over what code and in what environment the child process is running.

As mentioned, the most important means is to have at least 1. 2 is not only child processes though, but I agree this would be its major impact. So leave it 1 is ok and the major request.

This is not useful. Either install the file as 30-yama-ptrace.conf.example (see rpm -ql systemd-networkd|rg example for inspiration), or since it's a one line config, just include the instructions how to override the setting in 20-yama-scope.conf. Actually, the file already has those instructions, so just update/reverse them to match the new default value.

There is a misunderstanding here (I think). What you write ends up in the situation what we already have: ptrace_yama would be set again to 0. Therefore, we could leave it with the current file created by the package (which is a different one, I think a 10...conf or 15...conf file), which is doing already what you suggest here. Keep in mind that the file with 30-yama-ptrace.conf would be necessary only if we want to use the setting 2 rather than 1 :

The existing file 20... needs to be used to change the kernel default from 1 to 2, rather than from 1 to 0. This means, for the users who need a file to set 0 if they want to use gdb and strace with all possibilities, a new file is necessary that would then overrule the file 20..., which already contains 2. So we need a file 30... which then sets 0 again for those who want it. The latter would therefore not be installed by default.

If FESCo would reset the setting to its kernel-approved default 1 (so not to 2) and only remove the unapproved change to the setting 0, the file 20... would be unchanged as it is, but then be uninstalled by default while available to those who want to manually install it. So in that case, no 30... file would be necessary at all. Hope that resolves the misunderstanding.

If that is not a misunderstanding and you mean it the way I read it in the given context, so to formalize that it is by default 0 and users have to enable it manually to 1 or 2 the way you elaborate with the help of the config's comments: we have evidence that the developers were not able to identify this file and its implications when they were hit by its impacts, which is why they bypassed it with the current means, leaving an unapproved temporary means as a persistent one system-wide. Why to assume that average users, who do not experience explicitly that they have less security in areas that can be exploited, will know about the file, read its comments and act correspondingly when already developers were not able to do so? Sorry for the radical formulation, but that is how I would understand your comment in the given context (?).

I still think it'd be very nice if gdb, strace, and possibly other tools were modified to print an explanation if this setting prevents them from working. I think it'll save us a lot of surprised queries from developers who use Fedora but don't necessarily follow fedora-devel

Agreed. But we have only one person who volunteers to do it at some time, and they made clear they do not know when they can spare the time to modify these tools. I cannot do that. Disabling relevant security features to avoid queries of a group that is not the majority of users should be no option though. Or at least, if FESCo really agrees to do so (and thus keep the unapproved change), it should be formalized, and officially approved in the aftermath that this is officially what Fedora is doing.

It's not a summary. It's an essay of its own. Please editorialize the document to have a short Summary and a more concise Detailed Description.

I understand your point. I can put it on my To Do list, but my way of elaborating technical things, when I have only one-directional text-only available, will not change, which surely is a weakness of mine. Splitting this into the requested three proposals and elaborate it in the requested way will either take more time than I can invest atm, or it would need a co-author who is more used to formulate this type of request. I already calculated the time I can invest for the current proposal in a "radical" way to get it done.

If FESCo agrees to undo the unapproved system-wide change of ptrace_scope, which means ptrace_scope becomes the kernel-default 1 again without packages available that can undermine this, I am happy to create the elaborated documentation as this does not take much time for me as long as anyone/developers provide the requested information (as written in the proposal), even if the other parts of the proposal are rejected. This was the major point of the proposal anyway. But I do not know when I can "output" the three proposals as requested.

The issue around the package links of course also to the question if it needs FESCo-approval if gdb/strace would keep imposing a system-wide change, once installed, without informing the user in any way (or if there would be a way to tell the user on installation?). Also, the question if there is a risk that this issue (accidental installation by default, even if no gdb/strace is installed) could occur again if the package remains available (and if/how to mitigate that, if decided so). These questions would become relevant if FESCo agrees to remove the package from being installed by default, but leaving it as dependency for gdb.

Should we again discuss this proposal in this ticket? The wall of text makes that a bit hard imho. There are three separate policy proposals in this change request, which imho makes it really confusing. I cannot usefully comment on the kptr_restrict or the bpf policies since the impact isn't really clear. And the yama scope proposal seems to just demand that Fesco tell the maintainers of the user space observability tools to drop their current policy/package and then report what breaks so that the proposer can document that breakage. Which I feel is not serious.

Should we again discuss this proposal in this ticket?

No, of course not. I saw a high likelihood for a misunderstanding around the ptrace config files and thus clarified the point.

The wall of text makes that a bit hard imho.

Sorry for that. But complex technical topics in this realm in an English one-direction text-only communication is not my strength, especially if i cannot spare much time. With less text I learned misunderstandings are more likely in that situation. I wanted to raise some awareness of an issue and opportunity, and didn't expect anything to happen otherwise, and maybe raise some awareness that this community works with very different assumptions on its two ends, but I couldn't invest much time and was also hoping some others show interest to team up to work on it and hopefully get a feedback of expectations and if it is realistic to get support and acceptance at all. Some was achieved. If we get ptrace back to its original default, I would see this as great success.

As you indicate, at this time we talk in here mostly if ptrace 0 or 1, and how to implement it. The offer for the documentation stands, and maybe the volunteer who wants to update the tools' to contain expressive error messages can find some time too. No new discussion :)

“I didn't have time to write a short letter, so I wrote a long one instead.”
― Mark Twain (disputed)

:smile:

I appreciate the work that went into this Change proposal, but I do agree that this should have been three proposals, not one. Making it one large proposal resulted in there being confusion about the scope and which change would actually have what effects.

Personally I can only speak to the change for the yama.ptrace_scope defaults since I looked into this more in depth, and I would like us to move to the kernel default of 1. I have no idea what effect the other two changes would have, which kind of makes it hard for me to vote +1 here.

Coincidentally, this ticket has now been open for voting for four weeks but has not received a single vote yet - which I think has never happened before (?) - so it looks like other FESCo members might share my hesitancy here ...

What should we do about this proposal then? Not voting on it will not make it "go away" :)

If I would have to vote on this, I would put -1 because I think this proposal needs to be broken up and relevant stakeholders need to be engaged to be part of each Change.

What should we do about this proposal then? Not voting on it will not make it "go away" :)

"If there are no votes at all, the proposal is considered rejected and the status quo will be maintained."

So, not voting on it actually does make it go away... Technically this is already rejected.

That said, I agree with @ngompa here. This is not a single proposal and needs to be split up. We can consider the individual parts separately and with more care than trying to stuff all of this into a single folder in my brain.

“I didn't have time to write a short letter, so I wrote a long one instead.”

xD Actually, sometimes this can be easier and quicker ^^

Personally I can only speak to the change for the yama.ptrace_scope defaults since I looked into this more in depth, and I would like us to move to the kernel default of 1.

I would have no problem with limiting the proposal to this (the proposal was intended to allow that way anyway^^), and still providing the documentation part on my side (given the requested information from any developer(s)). This is what most discussion has been dedicated about, and a lot of opinions about ptrace_scope have been formulated from all parts of the community. At the same time, as already formulated by some in the original topic (before the proposal), it can be questioned if this issue (setting ptrace_scope back to 1 by any means) is even related to any proposal, as the change was a mix of an improvisation when something broke, and a resulting accident. It can be argued that the proposal here just allowed many people to add their opinions about if and how. So if you reject the proposal but implement ptrace_scope=1 by some means, my offer still stands about it.

What should we do about this proposal then? Not voting on it will not make it "go away" :)

Sorry for causing this issue :)

We still have time for F44. I hope that this can move forward, but it seems that we're currently stuck. @py0xc3 Can you withdraw this one and resubmit as three separate ones?

Also if you are still interesting in improving the policy around the yama scope, could you first discuss with the current default-yama-scope maintainers and the package maintainers/users relying on it?

Your current wording feels very adversarial. It reads (to me) as if you are just demanding their package gets obsoleted and you take over. I am sure that if you work with the current maintainers, before writing a Change Proposal, we can refine it in ways that benefits more people.

We still have time for F44. I hope that this can move forward, but it seems that we're currently stuck. @py0xc3 Can you withdraw this one and resubmit as three separate ones?

I can do, but I am not sure if I can do an expressive split until the next release (currently involved in a company formation). I agree that this is not a time critical update.

I can put the bpf and kptr on the To Do list as these are indeed no critical element, even if not done after F44. But it might be useful if someone could create something "sufficiently expressive" for the ptrace_yama case, as this can have more realistic impacts on average users especially given the increased need for involving external/untrusted software many of our users deploy without fully understanding it (at least to get it off the default system-wide condition as gdb is not a condition on average user's installations anyway; it doesn't really justify atm the "security by default for average users" we market Fedora with, especially compared to other distributions). Not sure if something with one or two liners and links at each section would suffice to have something to "formally process" about ptrace_yama=1? (That might be possible in the subsequent weeks)

Can you withdraw this one

Sure. Is this done just by changing the category in the wiki again?

Your current wording feels very adversarial

To be clear: it wasn't intended that way, although I admit, I read their original bug ticket that way. I still see it critical that a security-critical change was introduced system-wide without following the process while undoing it needs a proposal. But I also understand and respect that they had to solve something in an improvised way when it was broke, and this leads people in general to emphasize on something without always considering every implication/risk on the left and right, and then it caused an unintended accident, and with that in mind I wrote the proposal. I differentiate between the accident (system-wide deployment) and the vulnerable practice to solve a time-critical gdb issue leading to the accident. But thanks for the incentive that it reads that way, that is appreciated and I will consider it in future when formulating such things!

and the package maintainers/users relying on it?

Not sure if that is applicable to the accident, which is the more critical part of the proposal. Average users do not rely on it, but get it installed by default. The gdb solution is something I think should be replaced, but it is not the critical part, and I'm fine to postpone this, but the systemwide deployment of the package by default for everyone, without the actual need, is what I would consider to try to get done more quickly than waiting for f45+ or sophisticated new proposals, as it is no "real" default dependency but an unintended accident with side effects, while undoing the system-wide default deployment would not impact any user group negatively. While it is not realistic to talk to all users of gdb/strace, some had been involved early, already before the proposal was created. It was users of it that actually elaborated the situation in the topic prior to the proposal (mentioned in docs of the proposal).

Can you withdraw this one

Sure. Is this done just by changing the category in the wiki again?

Done (hope that suffices that the system auto-processes the rest?)

To be clear: it wasn't intended that way, although I admit, I read their original bug ticket that way. I still see it critical that a security-critical change was introduced system-wide without following the process while undoing it needs a proposal. But I also understand and respect that they had to solve something in an improvised way when it was broke, and this leads people in general to emphasize on something without always considering every implication/risk on the left and right, and then it caused an unintended accident, and with that in mind I wrote the proposal. I differentiate between the accident (system-wide deployment) and the vulnerable practice to solve a time-critical gdb issue leading to the accident. But thanks for the incentive that it reads that way, that is appreciated and I will consider it in future when formulating such things!

Try to avoid calling the current policy "improvised", "unintended" or an "accident". That really rubs people (me) the wrong way. It feels like you are not interested in the reasons for the current policy setup, the design, why certain choices were made or actually helping out with improving the current setup.

The current outcome is that the package (and thus ptrace scope = 0) is imposed system-wide to all Fedora users by default, including all users not installing or using gdb or anything comparable. That was not intended, was it? I don't mean this question as accusation or so but to clarify: if that was intended, then it was already the data in the discussion of the original topic, so the one prior to the change proposal, that was misinterpreted.

The current outcome is that the package (and thus ptrace scope = 0) is imposed system-wide to all Fedora users by default, including all users not installing or using gdb or anything comparable.

imho elfutils-libs (or at least programs based on it) are comparable to gdb, it provides libraries for process introspection. Having elfutils-libs depend on the default-yama-scope was simpler than trying to find all leaf packages that were broken otherwise.

That was not intended, was it? I don't mean this question as accusation or so but to clarify: if that was intended, then it was already the data in the discussion of the original topic, so the one prior to the change proposal, that was misinterpreted.

The intend is to not break user space programs by default. Compare it with making sure there is a selinux policy in place for applications that need to be able to access resources.

Why was this change not introduced by a system-wide change proposal? I understand your point that the system-wide impact was intended (sorry, don't know a better word here than "intended" but it's meant neutral :). You argue I should involve more stakeholders, but I read the tickets that involving other stakeholders or a change proposal is what was avoided when this was introduced.

The misinterpretations of the original topic are (I assume here this is also the reason for others who've fallen prone to the misunderstanding) based on this having caused a system-wide change that was not proposed as such, while bug tickets contained comments that discussed to avoid a change proposal at FESCo but not discussed impacts or interests of other stakeholders, while some people experienced an issue that needed attention (broken gdb etc.). The outcome was the assumption of an improvised solution intended for gdb/strace and the absence of intention for imposing this system-wide.

Compare it with making sure there is a selinux policy in place for applications that need to be able to access resources.

With your example in mind, this did not introduce a policy but disable SELinux at all.

Y'all, this really isn't the place to carry on discussions. If you need to hash out details or get "how we got here" nailed down, there's the discussion forum for that.

Metadata Update from @salimma:
- Issue tagged with: meeting

Let's bring this up quickly in the meeting - I think the consensus is to just close this based on the reluctance of everyone to vote and that this should be refiled as separate Change Proposals, but let's make sure everyone is on the same page.

Can you withdraw this one


Sure. Is this done just by changing the category in the wiki again?


Done (hope that suffices that the system auto-processes the rest?)

I tried to withdraw it when zbyszek asked for it, but not sure if that is possible at this time or if more is necessary than just changing the categories in the wiki as I did it (?), but at this time, just agreeing to close it on your side in the meeting might be indeed easier for you / the least time intensive variant :) Thanks for taking care.

Metadata Update from @salimma:
- Issue close_status updated to: Invalid
- Issue status updated to: Closed (was: Open)

Metadata