freeipa

#7685 [pyasn1] not supported upstream's version
Closed: fixed Opened by slev.

Closed: fixed
Reopen Issue

FreeIPA use pyasn1 python modules, which have a differ versions from an upstream.

rpm -q python3-pyasn1
python3-pyasn1-0.3.7-2.fc28.noarch
rpm -q python3-pyasn1-modules
python3-pyasn1-modules-0.3.7-2.fc28.noarch

Upstream's ones:

pyasn1 0.4.4
pyasn1-modules 0.2.2

Using the latest version there is an one error at least:

ipa -d cert-show 0xA
...
ipa: DEBUG: Destroyed connection context.rpcclient_140097019718064
ipa: ERROR: an internal error has occurred

This can be reproduced on WebUI (click on cert 10).
Apache error_log:

[Thu Sep 06 07:35:26.325622 2018] [:warn] [pid 27879:tid 139945519863552] [client 10.88.129.33:40350] failed to set perms (3140) on file (/var/run/ipa/ccaches/admin@IPASL.IPATEST)!, referer: https://dcsl.ipasl.ipatest/ipa/ui/
[Thu Sep 06 07:35:26.497961 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350] ipa: ERROR: non-public: PyAsn1Error: <TagSet object at 0x7f4780535978 tags 0:32:16> not in asn1Spec: <OctetString schema object at 0x7f47803e6cf8 tagSet <TagSet object at 0x7f479bb28d68 tags 0:0:4> encoding iso-8859-1>
[Thu Sep 06 07:35:26.497994 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350] Traceback (most recent call last):
[Thu Sep 06 07:35:26.498001 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]   File "/usr/lib/python3/site-packages/ipaserver/rpcserver.py", line 370, in wsgi_execute
[Thu Sep 06 07:35:26.498007 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]     result = command(*args, **options)
[Thu Sep 06 07:35:26.498012 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]   File "/usr/lib/python3/site-packages/ipalib/frontend.py", line 450, in __call__
[Thu Sep 06 07:35:26.498017 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]     return self.__do_call(*args, **options)
[Thu Sep 06 07:35:26.498036 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]   File "/usr/lib/python3/site-packages/ipalib/frontend.py", line 478, in __do_call
[Thu Sep 06 07:35:26.498042 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]     ret = self.run(*args, **options)
[Thu Sep 06 07:35:26.498047 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]   File "/usr/lib/python3/site-packages/ipalib/frontend.py", line 800, in run
[Thu Sep 06 07:35:26.498053 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]     return self.execute(*args, **options)
[Thu Sep 06 07:35:26.498058 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]   File "/usr/lib/python3/site-packages/ipaserver/plugins/cert.py", line 1220, in execute
[Thu Sep 06 07:35:26.498063 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]     self.obj._parse(result, all)
[Thu Sep 06 07:35:26.498068 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]   File "/usr/lib/python3/site-packages/ipaserver/plugins/cert.py", line 493, in _parse
[Thu Sep 06 07:35:26.498074 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]     cert.san_general_names)
[Thu Sep 06 07:35:26.498079 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]   File "/usr/lib/python3/site-packages/ipalib/x509.py", line 319, in san_general_names
[Thu Sep 06 07:35:26.498095 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]     gns = self.__pyasn1_get_san_general_names()
[Thu Sep 06 07:35:26.498101 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]   File "/usr/lib/python3/site-packages/ipalib/x509.py", line 351, in __pyasn1_get_san_general_names
[Thu Sep 06 07:35:26.498106 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]     ext['extnValue'], asn1Spec=univ.OctetString())[0]
[Thu Sep 06 07:35:26.498111 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]   File "/usr/lib/python3/site-packages/pyasn1/codec/ber/decoder.py", line 1318, in __call__
[Thu Sep 06 07:35:26.498117 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350]     '%s not in asn1Spec: %r' % (tagSet, asn1Spec)
[Thu Sep 06 07:35:26.498124 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350] pyasn1.error.PyAsn1Error: <TagSet object at 0x7f4780535978 tags 0:32:16> not in asn1Spec: <OctetString schema object at 0x7f47803e6cf8 tagSet <TagSet object at 0x7f479bb28d68 tags 0:0:4> encoding iso-8859-1>
[Thu Sep 06 07:35:26.498134 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350] 
[Thu Sep 06 07:35:26.498366 2018] [wsgi:error] [pid 27875:tid 139945084823296] [remote 10.88.129.33:40350] ipa: INFO: [jsonserver_session] admin@IPASL.IPATEST: cert_show('10', cacn='ipa', all=True, version='2.229'): InternalError

It is need to more deep test.
Additional question: Why does IPA/Fedora use not upstream's versions?

Screenshot_20180906_104531.png

Fedora doesn't have a newer version of pyasn1, because we run exactly into this problem. It looks like a breaking change in pyasn1. :(

It is sad :(
Is there an issue tracker?

I opened a bug on the pyasn1 bug tracker, https://github.com/etingof/pyasn1/issues/143

Hey, sorry for the regression!

I think it has been introduced by the overhaul in pyasn1 related to ASN.1 ANY type support...

I suspect that if you basically remove the extra decoding at line #349:

            der = ext['extnValue']
            gns = decoder.decode(der, asn1Spec=rfc2459.SubjectAltName())[0]

That might help because pyasn1>0.3.7 unwraps the ANY container, while pyasn1 <=0.3.7 does not.

In the grand scheme of things, unwrapping ANY container actually makes more sense because it can be tagged and all these containerization details logically belong to the "parent" data structure, rather than to whatever is serialized in ANY...

Having said that, one solution could be to update FreeIPA code in that regard. If that is not an option, I can probably introduce a configuration setting to pyasn1>0.4.4 to preserve old behaviour.

But there must be a better way! With the introduction of the OpenType feature in 0.4.1, pyasn1 can fully decode the certificate (e.g. including all known extensions) on its own. You just pass decodeOpenTypes=True to the decode() function. So that could be another way to update and probably simplify FreeIPA code.

Let me know if you want me to help you out with any of these changes.

FreeIPA has to stay compatible with 0.3.7 and eventually support 0.4.* additionally to 0.3.7+. What's your suggestion here?

Assuming the problem has been diagnosed correctly, I can think of these options:

  1. Introduce conditional ANY container decoding in FreeIPA based on pyasn1 version being used -- the ANY container will be unwrapped only if pyasn1<=0.3.7
  2. Introduce conditional ANY type decoding in FreeIPA based on pyasn1 version being used -- the whole extnValue decoding block will run in FreeIPA only if pyasn1<=0.3.7
  3. Introduce a new option to pyasn1 decode() function -- if set to a non-default value, the new behaviour of unwrapping ANY type would be enabled

It seems that option 3 is least invasive to FreeIPA while option 2 is most beneficial to FreeIPA as it defers certificate decoding fully to pyasn1.

WDYT?
Edited by etingof

Hey, sorry for the regression!
I think it has been introduced by the overhaul in pyasn1 related to ASN.1 ANY type support...
I suspect that if you basically remove the extra decoding at line #349:
der = ext['extnValue']
gns = decoder.decode(der, asn1Spec=rfc2459.SubjectAltName())[0]

That might help because pyasn1>0.3.7 unwraps the ANY container, while pyasn1 <=0.3.7 does not.
In the grand scheme of things, unwrapping ANY container actually makes more sense because it can be tagged and all these containerization details logically belong to the "parent" data structure, rather than to whatever is serialized in ANY...
Having said that, one solution could be to update FreeIPA code in that regard. If that is not an option, I can probably introduce a configuration setting to pyasn1>0.4.4 to preserve old behaviour.
But there must be a better way! With the introduction of the OpenType feature in 0.4.1, pyasn1 can fully decode the certificate (e.g. including all known extensions) on its own. You just pass decodeOpenTypes=True to the decode() function. So that could be another way to update and probably simplify FreeIPA code.
Let me know if you want me to help you out with any of these changes.

This helps:

der = ext['extnValue']
gns = decoder.decode(der, asn1Spec=rfc2459.SubjectAltName())[0]

At least certificate is shown without error.

Metadata Update from @cheimes:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/2381

Metadata Update from @cheimes:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1631253

master:

  • e1a30d3ce32aef2492689de8eff09360f5480a99 Workaround for pyasn1 0.4

ipa-4-7:

  • 85a54ef83a9461d89080d2cadf04fda2cc2ca1da Workaround for pyasn1 0.4

Metadata Update from @cheimes:
- Issue assigned to cheimes
- Issue priority set to: normal
- Issue set to the milestone: FreeIPA 4.7.1

The issue has been fixed in the latest release.

Metadata Update from @cheimes:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)
Metadata
None
None
None
Normal
https://github.com/freeipa/freeipa/pull/2381
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.