MainTask’ not found – Traveler: Class ‘com/lotus/sync/admin/MainTask’ not found

TL:DR When you start traveler on Domino 14 on Centos and get “Exception in thread “main”…” replace Notes.jar with an older version or upgrade to FP2 version and you are good to go.

So you installed Domino 14 on Centos Stream 9 Kernel 5.14.0-477 and then installed Traveler 14 (Traveler_14.0_Linux_ML_EA2.tar.gz) and the task does not start with the following errors reported:

Exception in thread "main" java.lang.NoClassDefFoundError: org.omg.CORBA.UserException
        at java.base/java.lang.ClassLoader.defineClassInternal(ClassLoader.java:472)
        at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:433)
        at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
        at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:1174)
        at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:961)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:867)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:825)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1095)
Caused by: java.lang.ClassNotFoundException: org.omg.CORBA.UserException
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:827)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1095)
        at java.base/java.lang.ClassLoader.defineClassImpl(Native Method)
...
Traveler version: 14.0.0.0 Build 202311091445_30 using DB jdbc:derby:ntsdb;create=true
	at com.lotus.sync.admin.MainTask.main(MainTask.java:2765)
	at lotus.domino.NotesThread.run(Unknown Source)
	at com.lotus.sync.admin.MainTask.runNotes(MainTask.java:2517)
	at com.lotus.sync.admin.MainTask.loadMobilityServer(MainTask.java:518)
	at com.lotus.sync.caf.admin.CAFAdmin.startCAF(CAFAdmin.java:134)
	at com.lotus.sync.dca.BackEndManager.startup(BackEndManager.java:226)
com.lotus.sync.util.ComponentNotStartedException: Could not load Converter Info, see previous errors
SEVERE  main-7f13287fb740 *system CAFAdmin.startCAF#145 The component CAF could not be started.    Internal error: com.lotus.sync.util.ComponentNotStartedException: Could not load Converter Info, see previous errors Exception Thrown: com.lotus.sync.util.ComponentNotStartedException: Could not load Converter Info, see previous errors
...

And the JVM

show jvm
> WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by lotus.notes.AgentSecurityManager (file:/opt/hcl/domino/notes/14000000/linux/ndext/Notes.jar)
WARNING: Please consider reporting this to the maintainers of lotus.notes.AgentSecurityManager
WARNING: System::setSecurityManager will be removed in a future release
08/14/2024 01:30:00 PM  JVM: Java Virtual Machine initialized.
08/14/2024 01:30:00 PM  Java runtime version: 17.0.8.1+1
08/14/2024 01:30:00 PM  JVM version: JRE 17 Linux amd64-64-Bit Compressed References 20230824_549 (JIT enabled, AOT enabled) OpenJ9   - d12d10c9e OMR      - e80bff83b JCL      - 8ecf238a124 based on jdk-17.0.8.1+1
08/14/2024 01:30:02 PM  Router: Updating values from Configuration document

There is no mention of this on any support site and even after raising a support call with HCL we did not make progress. When HCL support asks for the dump files and there are none run this command “tell traveler systemdump” and the full and very usefull dumps will appear in IBM…Traveler logs. Pity HCL supported omitted that. Hey ho.

After a few days of getting nowhere we made a copy of the prod server on Proxmox and started pulling the server apart. In the process finding the uninstall as well. It is not in the installation directory but rather, as the documentations points out in “Domino program directory/_HCL_Traveler_installation/traveler” When something is right in front of your eyes but you still don’t see it.

We took a Domino 12 server, installed Traveler 12 and got it working. We then upgraded the server to Domino 14 and upgraded Traveler to version 14.

By replacing Notes.jar on the Notes 14 server with the Notes.jar file on the upgraded server Traveler started working.

-rwxr-xr-x 1 root root  1223321 Aug 22 12:34 Notes.jar (copied from the upgraded server)
-rwxr-xr-x 1 root root  2090724 Nov  9  2023 Notes.jar.notworking

scp -r /opt/hcl/domino/notes/14000000/linux/ndext/Notes.jar  root@172.22.10.xx:/opt/hcl/domino/notes/14000000/linux/ndext

We the started Traveler and it worked.

Traveler: WARNING *system Status change from Green to Red: The APNS certificate for Verse Production has expired.
Traveler: WARNING *system Status change from Green to Red: The APNS certificate for Verse for Citrix Production has expired.
Traveler: WARNING *system Status change from Green to Red: The APNS certificate for To Do Production has expired.
Traveler: Restarting HTTP server for configuration changes.
XSP Command Manager terminated
HTTP Server: Shutdown
HTTP Server: Using Internet Site Configuration View
Traveler: Server started.

As the APN expired we downloaded and installed Domino 14 FP2, Traveler FP1 and the updated APN’s. The good news is the Notes.jar file has been upgraded and now works properly. Here are some links –> https://brandlrainer.info/2024/08/22/hcl-notes-domino-14-0fp2-and-hcl-notes-traveler-14-0-fp1-is-available/ and https://alichtenberg.cz/hcl-notes-domino-14-0-fp2-is-available/ use https://planetlotus.org/

In /opt/hcl/domino/notes/14000000/linux/ndext/

Before FP2

-rwxr-xr-x 1 root root  1223321 Aug 22 12:34 Notes.jar
-rwxr-xr-x 1 root root  2090724 Nov  9  2023 Notes.jar.notworking

After FP2 

-rwxr-xr-x 1 root root  2090651 Aug  9 16:08 Notes.jar
-rwxr-xr-x 1 root root  2090724 Nov  9  2023 Notes.jar.notworking

In /opt/hcl/domino/notes/14000000/linux/ndext/140FP2

-rwxr-xr-x 1 root root 2090651 Aug  9 16:08 Notes.jar

In /opt/hcl/domino/notes/14000000/linux/ndext/14000000/ (Backed up by the FP2 installer)
-rwxr-xr-x 1 root root 1223321 Aug 22 12:34 Notes.jar

The new JVM in use

sho jvm
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by lotus.notes.AgentSecurityManager (file:/opt/hcl/domino/notes/14000000/linux/ndext/Notes.jar)
WARNING: Please consider reporting this to the maintainers of lotus.notes.AgentSecurityManager
WARNING: System::setSecurityManager will be removed in a future release
08/23/2024 09:15:38 AM  JVM: Java Virtual Machine initialized.
08/23/2024 09:15:38 AM  Java runtime version: 17.0.10+7
08/23/2024 09:15:38 AM  JVM version: JRE 17 Linux amd64-64-Bit Compressed References 20240116_670 (JIT enabled, AOT enabled) OpenJ9   - 2c3d78b48 OMR      - ea8124dbc JCL      - 2aad089841f based on jdk-17.0.10+7

--------
So the current JVM is
JRE 17 OpenJ9 based on jdk-17.0.10+7
and was previously
JRE 17 OpenJ9 based on jdk-17.0.8.1+1

The version of Centos Stream 9 used for Domino 14.

# hostnamectl
   Static hostname: (unset)
Transient hostname: 
         Icon name: computer-vm
           Chassis: vm 🖴
        Machine ID: 15feca34bc9a4dfdbf9defcac98cb4d1
           Boot ID: 8eb668b3a38e41a099c730315a6d7c0a
    Virtualization: kvm
  Operating System: CentOS Stream 9
       CPE OS Name: cpe:/o:centos:centos:9
            Kernel: Linux 5.14.0-447.el9.x86_64
      Architecture: x86-64
   Hardware Vendor: QEMU
    Hardware Model: Standard PC _Q35 + ICH9, 2009_
  Firmware Version: rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org

My question is this. Are we the only customer to run Domino 14 with Traveler on Centos Stream 9?