Compile Lucee Source Offline

mahogany · November 25, 2020, 5:50pm

Good Evening,

as the topic title says I’m trying to compile the lucee source without an internet connection. Of course this won’t work because maven has to fetch all the needed dependencies. I’m talking about successive builds with prefetched libraries etc…

Running maven with an active internet connection to compile lucee works without any issues. (e.g. mvn package). But removing the connection and rerunning maven (with additional --offline option) doesn’t work and the build fails with the following errors:

__build_archives:
--- omitted uncessary lines ---
     [echo] ---------- 2020-11-25 17:23:24 - building different archives that get bundled (admin,doc) with help of the temporary generated lucee.jar ----------
     [java] Buildfile: /build/source/ant/build-create-archive.xml
     [java]
     [java] all:
     [java]      [copy] Copying 56 files to /build/source/temp/archive/context
     [java]    [script] ERROR: Failed to download the bundle  [com.sun.jna:5.5.0] from [https://update.lucee.org/rest/update/provider/download/com.sun.jna/5.5.0/?allowRedirect=true&jv=14.0.2-internal] and copy to [/build/source/temp/archive/base/lucee-server/bundles/com-sun-jna-5-5-0.jar]
     [java]    [script] javax.servlet.ServletException: java.io.IOException: Failed to download the bundle  [com.sun.jna:5.5.0] from [https://update.lucee.org/rest/update/provider/download/com.sun.jna/5.5.0/?allowRedirect=true&jv=14.0.2-internal] and copy to [/build/source/temp/archive/base/lucee-server/bundles/com-sun-jna-5-5-0.jar]
     [java]    [script]         at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:394)
     [java]    [script]         at lucee.loader.engine.CFMLEngineFactory.initEngineIfNecessary(CFMLEngineFactory.java:260)
     [java]    [script]         at lucee.loader.engine.CFMLEngineFactory.getInstance(CFMLEngineFactory.java:166)
     [java]    [script]         at lucee.runtime.script.BaseScriptEngineFactory.<init>(BaseScriptEngineFactory.java:59)
     [java]    [script]         at lucee.runtime.script.LuceeScriptEngineFactory.<init>(LuceeScriptEngineFactory.java:27)

__build_testbox:
Warning: Nashorn engine is planned to be removed from a future JDK release
     [echo] ---------- 2020-11-25 17:23:25 - execute testcases false ----------
     [java] Buildfile: /build/source/ant/run-testcases.xml
     [java]
     [java] run:
     [java]      [echo]
     [java]      [echo]   _____         _   ____
     [java]      [echo]  |_   _|__  ___| |_| __ )  _____  __
     [java]      [echo]    | |/ _ \/ __| __|  _ \ / _ \ \/ /
     [java]      [echo]    | |  __/\__ \ |_| |_) | (_) >  <
     [java]      [echo]    |_|\___||___/\__|____/ \___/_/\_\
     [java]      [echo]
     [java]      [echo] ---------- 2020-11-25 17:23:26 - start TestBox testcases ----------
     [java]    [script] ERROR: Failed to download the bundle  [com.sun.jna:5.5.0] from [https://update.lucee.org/rest/update/provider/download/com.sun.jna/5.5.0/?allowRedirect=true&jv=14.0.2-internal] and copy to [/build/source/temp/archive/base/lucee-server/bundles/com-sun-jna-5-5-0.jar]
     [java]    [script] javax.servlet.ServletException: java.io.IOException: Failed to download the bundle  [com.sun.jna:5.5.0] from [https://update.lucee.org/rest/update/provider/download/com.sun.jna/5.5.0/?allowRedirect=true&jv=14.0.2-internal] and copy to [/build/source/temp/archive/base/lucee-server/bundles/com-sun-jna-5-5-0.jar]
     [java]    [script]         at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:394)
     [java]    [script]         at lucee.loader.engine.CFMLEngineFactory.initEngineIfNecessary(CFMLEngineFactory.java:260)
     [java]    [script]         at lucee.loader.engine.CFMLEngineFactory.getInstance(CFMLEngineFactory.java:166)
     [java]    [script]         at lucee.runtime.script.BaseScriptEngineFactory.<init>(BaseScriptEngineFactory.java:59)
     [java]    [script]         at lucee.runtime.script.LuceeScriptEngineFactory.<init>(LuceeScriptEngineFactory.java:27)

The error message is pretty obvious. Without a working internet these bundles cannot be downloaded. But is it necessary to download these files everytime a rerun the compilation? As it seems the bundles get stored in ${rootDir}/temp while downloading them. Every new build clears this directory via <delete dir="${temp}" />.

Is it possible to cache these files like it’s done with the extensions?

Let me know if you need any more informations. Maybe my description is to vague I’m by far no expert when using maven/ant or java at all.

The version I tried was the latest 5.3 release on github. But I don’t think that the exact version number is very important.

Thank you in advance for your attention to this matter.

John

OS: NixOS 21.03 (Okapi) x86_64
Java Version: OpenJDK 64-Bit Server VM (build 25.272-b10, mixed mode)
Maven Version: Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Lucee Version: Lucee 5.3.8.112

Zackster · November 25, 2020, 8:05pm

I’m not java expert either, but I guess that com.sun.jna is missing from the pom.xml might be the reason? If that’s what maven uses to cache locally?

The argon2 seems to work, which is the line before

github.com

lucee/Lucee/blob/5.3/core/src/main/java/META-INF/MANIFEST.MF#L365


      
          sun.jndi.providerutil;bundle-version=1.2.4,
          javax.mail.activation;bundle-version=1.6.2.0000L,
          sun.security.jaas;bundle-version=1.2.4,
          tagsoup;bundle-version=1.2.1.0002L,
          w3c.dom;bundle-version=1.1.0,
          org.lucee.commons.email;bundle-version=1.2.0,
          org.lucee.jsch;bundle-version=0.1.55,
          org.lucee.jzlib;bundle-version=1.1.3,
          xmpcore;bundle-version=5.1.2.0002L,
          org.lucee.argon2;bundle-version=2.7.0,
          com.sun.jna;bundle-version=5.10.0
          Require-Extension: 7E673D15-D87C-41A6-8B5F1956528C605F;name=MySQL;label=MySQL;version=8.0.30,
          99A4EF8D-F2FD-40C8-8FB8C2E67A4EEEB6;name=MSSQL;label=MS SQL Server;version=7.2.2.jre8,
          671B01B8-B3B3-42B9-AC055A356BED5281;name=PostgreSQL;label=PostgreSQL;version=42.2.20,
          2BCD080F-4E1E-48F5-BEFE794232A21AF6;name=JDTsSQL;label=jTDS (MSSQL);version=1.3.1,
          CED6227E-0F49-6367-A68D21AACA6B07E8;name=Admin;label=Lucee Administrator;version=1.0.0.3,
          D46D49C3-EB85-8D97-30BEC2F38561E985;name=Doc;label=Lucee Documentation;version=1.0.0.2,
          17AB52DE-B300-A94B-E058BD978511E39E;name=S3;label=S3;version=0.9.4.156,
          87FE44E5-179C-43A3-A87B3D38BEF4652E;name=EHCache;label=EHCache;version=2.10.0.31,
          D46B46A9-A0E3-44E1-D972A04AC3A8DC10;name=Chart;label=CFChart;version=1.0.19.24,
          FAD1E8CB-4F45-4184-86359145767C29DE;name=Hibernate;label=Hibernate;version=3.5.5.87,

github.com

lucee/Lucee/blob/5.3/loader/pom.xml#L662


      
              <scope>provided</scope>
            </dependency>
          <!-- <dependency>
            <groupId>xdb</groupId>
            <artifactId>xdb</artifactId>
            <version>1.0.0</version>
            <scope>provided</scope>
          </dependency>
          <dependency>
            <groupId>xmlparserv2</groupId>
            <artifactId>xmlparserv2</artifactId>
            <version>1.2.2</version>
            <scope>provided</scope>
          </dependency> -->
          <dependency>
            <groupId>xmpcore</groupId>
            <artifactId>xmpcore</artifactId>
            <version>5.1.2.0002L</version>
            <scope>provided</scope>
          </dependency>
          <dependency>

You could try adding it as a dependency?

mahogany · November 25, 2020, 10:26pm

Extending the pom.xml with

  <dependency>
      <groupId>org.sun</groupId>
      <artifactId>jna</artifactId>
      <version>5.5.0</version>
  </dependency>

doesn’t work. At least it isn’t as simple as I imagined.
Maven fails with the following error:

--- previous output omitted...
Downloading from res: https://oss.sonatype.org/content/repositories/releases/org/lucee/argon2/2.7.0/argon2-2.7.0.jar
Downloaded from res: https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm-all/4.2/asm-all-4.2.jar (216 kB at 6.8 kB/s)
Downloading from res: https://oss.sonatype.org/content/repositories/releases/org/sun/jna/5.5.0/jna-5.5.0.jar
Downloaded from res: https://oss.sonatype.org/content/repositories/releases/javax/websocket/javax.websocket-api/1.1/javax.websocket-api-1.1.jar (37 kB at 1.1 kB/s)
Downloaded from res: https://oss.sonatype.org/content/repositories/releases/org/slf4j/slf4j-nop/1.7.12/slf4j-nop-1.7.12.jar (4.1 kB at 123 B/s)
Downloaded from res: https://oss.sonatype.org/content/repositories/releases/org/lucee/argon2/2.7.0/argon2-2.7.0.jar (415 kB at 10 kB/s)
Downloaded from res: https://oss.sonatype.org/content/repositories/releases/org/lucee/httpcomponents-httpclient/4.5.10/httpcomponents-httpclient-4.5.10.jar (771 kB at 19 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/sun/jna/5.5.0/jna-5.5.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  05:02 min
[INFO] Finished at: 2020-11-25T22:07:16Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project lucee: Could not resolve dependencies for project org.lucee:lucee:jar:5.3.8.112-SNAPSHOT: Could not find artifact org.sun:jna:jar:5.5.0 in repo (https://raw.githubusercontent.com/lucee/mvn/master/releases) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project lucee: Could not resolve dependencies for project org.lucee:lucee:jar:5.3.8.112-SNAPSHOT: Could not find artifact org.sun:jna:jar:5.5.0 in repo (https://raw.githubusercontent.com/lucee/mvn/master/releases)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:269)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:147)
    at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:248)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:202)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

A quick look at the lucee/mvn-Repository confirms the absence of the com.sun.jna artifact. What surprises me is the following maven output line:

Downloading from central: https://repo.maven.apache.org/maven2/org/sun/jna/5.5.0/jna-5.5.0.jar

Zackster · November 26, 2020, 3:23pm

maybe it’s because it’s been moved and redirects aren’t cached or something?

https://mvnrepository.com/artifact/com.sun.jna/jna

Zackster · November 26, 2020, 3:32pm

actually, I can’t see any reference in the java source code for jna

as cfx c++ tags where removed a while back in 5.2 [LDEV-1654] - Lucee

maybe it’s unused, you could try just removing it from the manifest?

mahogany · November 26, 2020, 5:26pm

Removing the artifact from the manifest didn’t work.

[echo] ---------- 2020-11-26 17:22:04 - copy extensions in place ----------
     [copy] Copying 18 files to /build/lucee-5.3.8.112-dependencies/temp/archive/base/lucee-server/context/extensions/available
     [echo] ---------- 2020-11-26 17:22:04 - getting and unzip testbox ----------
    [unzip] Expanding: /build/lucee-5.3.8.112-dependencies/temp/testbox-2.2.0.zip into /build/lucee-5.3.8.112-dependencies/temp/testbox
     [echo] ---------- 2020-11-26 17:22:05 - building different archives that get bundled (admin,doc) with help of the temporary generated lucee.jar ----------
     [java] Unable to locate tools.jar. Expected to find it in /nix/store/fl8aghgsn2lv07vf3nvjmbn7r90xdx94-openjdk-8u272-b10-jre/lib/openjdk/lib/tools.jar
     [java] Buildfile: /build/lucee-5.3.8.112-dependencies/ant/build-create-archive.xml
     [java]
     [java] all:
     [java]      [copy] Copying 56 files to /build/lucee-5.3.8.112-dependencies/temp/archive/context
     [java]    [script] javax.servlet.ServletException: org.osgi.framework.BundleException: Unable to resolve org.lucee.argon2 [39](R 39.0): missing requirement [org.lucee.argon2 [39](R 39.0)] osgi.wiring.package; (osgi.wiring.package=com.sun.jna) Unresolved requirements: [[org.lucee.argon2 [39](R 39.0)] osgi.wiring.package; (osgi.wiring.package=com.sun.jna)]
     [java]    [script]         at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:394)
     [java]    [script]         at lucee.loader.engine.CFMLEngineFactory.initEngineIfNecessary(CFMLEngineFactory.java:260)
     [java]    [script]         at lucee.loader.engine.CFMLEngineFactory.getInstance(CFMLEngineFactory.java:166)
     [java]    [script]         at lucee.runtime.script.BaseScriptEngineFactory.<init>(BaseScriptEngineFactory.java:59)
     [java]    [script]         at lucee.runtime.script.LuceeScriptEngineFactory.<init>(LuceeScriptEngineFactory.java:27)

It seems to be a dependency of the argon library.

Zackster · November 26, 2020, 7:05pm

time to file a bug

mahogany · November 26, 2020, 8:59pm

I’ll look into it. For my purposes its not very important to compile lucee from source. I’m still able to take the war/jar files from the cdn directly.

Background: My goal is to package lucee as nixpkg and nixos service. For example to jenkins package uses a precompiled war file and the service references this war directly. This allows anyone using nixos to setup a lucee system via a single configuration file.

Zackster · November 28, 2020, 1:06am

bug filed [LDEV-3164] - Lucee