Datadog dd-java-agent Integration

dev support
1

Hi,

Has anyone done anything with Lucee and Datadog at all?

We’ve installed the dd-java-agent.jar file in the lucee/lib directory.
Updated the setenv.sh file so that it’s loaded as a javaagent “-javaagent:lib/dd-java-agent.jar”.
On loading Lucee the console output shows it loading and running

[main] INFO datadog.trace.agent.ot.DDTraceOTInfo - dd-trace - version: 0.12.0~8bed6011
[main] INFO datadog.trace.agent.ot.DDTracer - New instance: DDTracer-13d9b21f{ serviceName=jedi-dd-test, writer=DDAgentWriter { api=DDApi { tracesEndpoint=http://localhost:8126/v0.3/traces } }, sampler=AllSampler { sample=true }, defaultSpanTags={}}
[main] INFO datadog.trace.agent.tooling.VersionLogger - dd-trace-ot - version: 0.12.0~8bed6011
[main] INFO datadog.trace.agent.tooling.VersionLogger - dd-trace-api - version: 0.12.0~8bed6011
[main] INFO datadog.trace.agent.tooling.VersionLogger - dd-java-agent - version: 0.12.0~8bed6011

However whenever we attempt to run any MS SQL calls (using the Microsoft Vendor driver) we always get

lucee.runtime.exp.NativeException: io/opentracing/Scope at com.microsoft.sqlserver.jdbc.SQLServerConnection.createStatement(SQLServerConnection.java:2912) at com.microsoft.sqlserver.jdbc.SQLServerConnection.createStatement(SQLServerConnection.java:2603) at lucee.runtime.type.QueryImpl.execute(QueryImpl.java:269) at lucee.runtime.type.QueryImpl.<init>(QueryImpl.java:226) at lucee.runtime.tag.Query.executeDatasoure(Query.java:1135) at lucee.runtime.tag.Query._doEndTag(Query.java:695) at lucee.runtime.tag.Query.doEndTag(Query.java:585) at lucee.runtime.functions.query.QueryExecute.call(QueryExecute.java:86) at lucee.runtime.functions.query.QueryExecute.call(QueryExecute.java:42) at jedi.index_cfm$cf.call(/jedi/index.cfm:14) at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:938) at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:828) at lucee.runtime.listener.ModernAppListener._onRequest(ModernAppListener.java:218) at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:43) at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2478) at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2468) at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2441) at lucee.runtime.engine.Request.exe(Request.java:44) at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1089) at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1037) at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102) at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:528) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1099) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:670) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1520) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1476) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoClassDefFoundError: io/opentracing/Scope ... 44 more Caused by: java.lang.ClassNotFoundException: io.opentracing.Scope not found by com.microsoft.sqlserver.mssql-jdbc [50] at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1639) at org.apache.felix.framework.BundleWiringImpl.access$200(BundleWiringImpl.java:80) at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:2053) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 44 more

Compaining that “io.opentracing.Scope” ClassNotFoundException.

I can manually createObject(“java”, “io.opentracing.Scope”) and this returns an object, but the SQL calls still fail.

Has anyone got any ideas?

1 Like
2

I’d be interested to hear about your progress with this. We use Datadog for server / container metrics but haven’t yet tried the Java agent.

This may not be particularly helpful, but out of curiosity have you tried the jTDS MSSQL driver?

The MS SQL driver is in an extension (OSGi ) so I guess it’s possible that it can’t see the opentracing packages. Can you also try copying the JAR into /usr/local/tomcat/lib/ to see if that helps? Otherwise I’m not sure, might need some input from @micstriit :slight_smile:

3

I’ve basically put it on a back-burner at the moment. I believe it’s OSGi related.

I can install it and run it on a lucee4 express, but not a lucee5 express.

I’ve tried the MS Vendor, jTDS and the Postgres drivers, each report the same issue.

On a Datadog channel it’s been suggested that the issue is:

The problem is likely that felix is not delegating to the bootstrap classpath correctly. Maybe there’s a way to configure what packages should be on the bootstrap classpath?
The agent adds those classes to the bootstrap to ensure they’re accessible on any classpath that delegates to the bootstrap.

However I’m unsure how I can play with that.

I attempted to convert the dd-agent.jar to an OSGi bundle and include it in the bundles directory of lucee, but still had the same issues.

4

@micstriit Do you have any advice on how to resolve this?

5

I understand that Lucee is supposed to have a datadog integration built in now. @Zackster has it been pushed to the current release and, if so, do you have any more information on how to set this up?

6

I’m chasing up the info.

There’s a custom logging layout for datadog (no admin ui yet), available in 5.3.9 and the later 5.3.8 SNAPSHOTs like 5.3.8.223

<logger appender="resource" appender-arguments="path:{lucee-config}/logs/mapping.log" layout="datadog" level="error" name="mapping"/>

The agent gets installed via JVM args and has the following options

-javaagent:${DATADOG_AGENT}
-Ddd.profiling.enabled=${DD_PROFILING_ENABLED} 
-Ddd.version=${DD_VERSION}
-Ddd.logs.injection=${DD_LOGS_INJECTION:true}
-Ddd.trace.enabled=${DD_TRACE_ENABLED:true}
-Ddd.trace.sample.rate=${DD_TRACE_SAMPLE_RATE}
-Ddd.service=${DD_SERVICE}
-Ddd.env=${DD_ENV}

Tracking deets here
https://luceeserver.atlassian.net/browse/LDEV-4091

7

I’m not sure what to do with this info. Hopefully, it will be worked into the admin UI soon. I could really use this right now…

8

The only aspect which will be added (soon) to the admin UI is the assigning of the datadog layout

As the DD agent has to be configured at the JRE level using the above parameters, Lucee doesn’t provide an admin interface for such parameters.

The JIRA task has further links explaining that configuration

9

Thank you. I’m denser than most - a good real-world example is worth 1000 pictures for folks like me if you can spare a few extra minutes.

10

I have literally provided you with a real world example of the config required!

If you can’t spare a few minutes to try it out yourself…

11

From where do these values come? To what do they pertain? How are they populated?

image
image1377×277 25.2 KB

12

sorry, see the task in jira for updated info