#15 Prepare jss for jdk 11 where APIs have been deprecated and some packages are no longer public
Closed: fixed Opened by emaldonado.

Filing this ticket in advance so it doesn't fall though the cracks and cause problems later. I'll update the ticket as we have more info.

While investigating issues on https://pagure.io/jss/issue/14 we discovered deprecation usage that may prevent building JSS once Fedora upgrades to java-11-openjdk and that becomes the standard jdk.
Please refer to https://fedoraproject.org/wiki/Changes/java-11-openjdk-TechPreview.


This is also needed for PKI: https://pagure.io/dogtagpki/issue/2982

Metadata Update from @edewata:
- Custom field component adjusted to None
- Custom field feature adjusted to None
- Custom field origin adjusted to None
- Custom field proposedmilestone adjusted to None
- Custom field proposedpriority adjusted to None
- Custom field reviewer adjusted to None
- Custom field type adjusted to None
- Custom field version adjusted to None

See: https://bugzilla.redhat.com/show_bug.cgi?id=1592196#c0

Additional info from @emaldonado:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=875589

Warning: this is a rather long post to dump everything I have learned so far.

The JSS build fails with 4 errors and 100 deprecation warnings:

$ grep error: typescript.build
./org/mozilla/jss/netscape/security/x509/RevocationReasonAdapter.java:20: error: package java.xml.bind.annotation.adapters does not exist
./org/mozilla/jss/netscape/security/x509/RevocationReasonAdapter.java:29: error: cannot find symbol
./org/mozilla/jss/provider/java/security/JSSKeyStoreSpi.java:27: error: package javax.xml.bind is not visible
./org/mozilla/jss/tests/SymKeyDeriving.java:43: error: package sun.security.pkcs11.wrapper is not visible

Basically two errors:
1) error: javax.xml.bind is not visible and
2) error: package sun.security.pkcs11.wrapper is not visible

This what I have learned about the second one.

The debian folks had the same problems as mentioned in previous post.
Jdk 9 and forward jdk uses modules, packages belong to modules and
can be made not public if they module owner choose to.
More info on modules can be found at
https://www.oracle.com/corporate/features/understanding-java-9-modules.html

This site https://fedoraproject.org/wiki/Changes/Java9TechPreview,
though now obsolete as it was meant for f27, has some good info on
the "How To Test section" that can be adapted to 10 which is available for
Rawhide:

    Install java-10-openjdk
    Run (as root) 'alternatives --config java' and select the 10 version
    Run 'java -version' to ensure that it is correct

I did that on my Rawhide system.

They have added a nice tool to check for dependencies called jdeps.
When I had opendk 8 as the one:

$ jdeps -jdkinternals /usr/lib64/jss/jss4.jar
jss4.jar -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.172-14.b11.fc29.x86_64/jre/lib/ext/sunpkcs11.jar
   org.mozilla.jss.tests.SymKeyDeriving (jss4.jar)
      -> sun.security.pkcs11.wrapper.PKCS11Constants        JDK internal API (sunpkcs11.jar)
Warning: JDK internal APIs are unsupported and private to JDK implementation that are subject to be removed or changed incompatibly and could break your application.
Please modify your code to eliminate dependency on any JDK internal APIs.
For the most recent update on JDK internal API replacements, please check:
https://wiki.openjdk.java.net/display/JDK8/Java+Dependency+Analysis+Tool

Following the instructions one can madke the TechPreviow the default:

$ sudo alternatives --config java
[sudo] password for emaldonado:
There are 2 programs which provide 'java'.
  Selection    Command
-----------------------------------------------
   1           java-openjdk.x86_64 (/usr/lib/jvm/java-10-openjdk-10.0.1.10-12.fc29.x86_64/bin/java)
*+ 2           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.172-14.b11.fc29.x86_64/jre/bin/java)
Enter to keep the current selection[+], or type selection number: 1
[emaldonado@localhost jss]$ sudo alternatives --config javac
There are 2 programs which provide 'javac'.
  Selection    Command
-----------------------------------------------
*+ 1           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.172-14.b11.fc29.x86_64/bin/javac)
   2           java-openjdk.x86_64 (/usr/lib/jvm/java-10-openjdk-10.0.1.10-12.fc29.x86_64/bin/javac)
Enter to keep the current selection[+], or type selection number: 2

After switching to the openjdk 10 reran the jdeps and got the same message.

Since the modules where introduced the sun.security.pkcs11.wrapper package is wrapped by a module. The PKCS11Constants interface apparently was intended for the use on the SunPKCS11 JCE provider.

Looking at the code there is a comment that it was made based on pkcs11t.h.

* This is mainly the content of the 'pkcs11t.h' header file.

In that case nothing prevents the team from creating its own
org.mozilla.jss.pkcs11.PKCS11Contants.java to export those constants to the rest
of JSS- and possibly client code as well.

Couldn't find a tool to create a java interface from a header file so it would
have to be done manually. Fortunately, the pkcs11 headers don't change
frequently and we do have our pkcs11 working group representative that can alert the team when changes are approved and shipped.

A bit or progress report or I should say lacl or progress and an exprimental patch so I don't loose the stuff. I tried to address java.xml.bind related errors.

./org/mozilla/jss/netscape/security/x509/RevocationReasonAdapter.java:20: 
error: package java.xml.bind.annotation.adapters does not exist
./org/mozilla/jss/netscape/security/x509/RevocationReasonAdapter.java:29: 
error: cannot find symbol
./org/mozilla/jss/provider/java/security/JSSKeyStoreSpi.java:27: 
error: package javax.xml.bind is not visible

Those are EE packages removed from Java SE . From what I have read Oracle recommends using JAVAX - https://docs.oracle.com/javase/tutorial/jaxb/intro/examples.html
Fedora ships javax support as part of glassfish-api package. Debian has something similar and both are from Apache.
CAVEAT: I'm still learning the stuff so this is merely a naive attempt to move forward with the build.

Also needed to address the fact that the javah tool is no longer available by adding the '-h $jni_header_dir' flag to the javac invocation. Also added our own PKCS11Constants as mentioned earlier. Here is a the not-yet-working patch for your information.

experimental.patch

@emaldonado suggested the possibility to create PKCS #11 constants from pkcs11t.h from NSS instead. See also:

  • https://github.com/nss-dev/nss/blob/master/lib/util/pkcs11t.h
  • https://www.oasis-open.org/committees/document.php?document_id=55657&wg_abbrev=pkcs11
  • https://www.oasis-open.org/policies-guidelines/ipr

The failure lines are these:

make[4]: Entering directory '/home/emaldonado/dogtagpkijss/jss/org/mozilla/jss/util'
gcc -o Linux4.18_x86_64_glibc_PTH_64_DBG.OBJ/jssutil.o -c -g -D_POSIX_SOURCE -D_DEFAULT_SOURCE -D_XOPEN_SOURCE -fPIC -DLINUX2_1  -Wall -Werror-implicit-function-declaration -Wno-switch -pipe -DLINUX -Dlinux -DHAVE_STRERROR -DXP_UNIX -DDEBUG -UNDEBUG -DDEBUG_emaldonado -D_REENTRANT -DUSE_UTIL_DIRECTLY -I../../../../../dist/Linux4.18_x86_64_glibc_PTH_64_DBG.OBJ/include -I../../../../../dist/public/jss -I../../../../../dist/private/jss -I/etc/alternatives/jre_11_openjdk/include -I/etc/alternatives/jre_11_openjdk/include/linux -I../../../../../dist/public/nspr20 -I../../../../../dist/public/nss  -I/etc/alternatives/jre_11_openjdk/include jssutil.c
jssutil.c:15:10: fatal error: _jni/org_mozilla_jss_util_Password.h: No such file or directory
 #include "_jni/org_mozilla_jss_util_Password.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Looking at that failed line you'll see that the following is missing:
    -I../../../../../dist/Linux4.18_x86_64_glibc_PTH_64_DBG.OBJ/private/jss/_jni

I see one like this on the successful build log for f28 .

We need a way to enable the gcc command to find the jni generated header files.

How do we achieve this? Are there tweaks needed for some of files in the conf or coreconf directories?

This is the latest version of the Experimental patch.
You need a Rawhide system with glassfish-jaxb and glassfish-jaxb-api installed as well as java-11-openjdk-11.

and set the following:

export JAVA_HOME=/etc/alternatives/jre_11_openjdk
export USE_64=1

experimentalV2.patch

Another thing:
sudo alternatives --config java
and
sudo alternatives --config javac
will let you switch among the installed jdks.

experimentalV2.patch

In this version I got rid of unwanted lines relating to binaries. It's still to big of a patch so addressing somewhat unrelated issues. It's best that I split it in two to make it easier to review.

The first phase patch. The low-hanging fruit as they say.

part1.patch

The second phase to be appied after the previous one. This is the hard and not yet working part.

part2.patch

3 cleaner patches coming next

Was able to build on Rawhide with JDK 11 active and the tests passed. Many warnings remain but that's for another day. @edewata suggested creating a branch and pushing the changes there. Will do that next.

Pushed: https://github.com/emaldona/jss/commits/PREPARE_FOR_OPEN_JDK_11_BRANCH

Pushed: https://github.com/emaldona/jss/commits/PREPARE_FOR_OPEN_JDK_11_BRANCH

The commit to that branch that so far has given me the best results is
https://github.com/emaldona/jss/commit/d62752ec8635a43f35a9c2c2f93dd4e412f7351c

notesOnVariousPlatforms.txt

Notes I made on the work so we don't forget when the time comes to resume this effort a few months from now.

Updated https://github.com/emaldona/jss/commits/PREPARE_FOR_OPEN_JDK_11_BRANCH
That was one to partially sync. up with recent changes result of to other issues that were resolved. Still a work in progress and it will be a while before we can actively resume working on this issue.

Opened jss PR#32 to fix the first issue.

Opened jss PR#32 to fix the first issue.

Thanks, I picked that change and applied in my jss fork and JDK 11 related branches.
While doing it I noticed that RevocationReasonAdapter had the same dependency.
Extract from my commit log to my branch:
- Fix from upstram PR #32, expanded to apply to RevocationReasonAdapter
- Remove from RevocationReasonAdapter.java this line:
- "import javax.xml.bind.annotation.adapters.XmlAdapter;"
- RevocationReasonAdapter.java no longer extends XmlAdapter

As JSS now compiles with with JDK11 and is passing on master, I'm going ahead and closing this issue. Some relevant PRs can be found with the jdk9+ label.

JDK 11 tests passes on master for both Fedora 29 and Debian testing.

Metadata Update from @cipherboy:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata