OSGI: How to handle the (wrong) bundles startup order

Context

In some situations it is needed that some of the OSGI bundles are started in a specific order.

A concrete case is when the Apache Camel is used in the context of the OSGI. If one of the OSGI bundles are an user defined Apache Camel component and another bundle uses this user defined Camel component into a (Camel) route then the bundle containing the Camel component should be started before the bundle that is using the Camel component.

Solution – modify the bundle/s start level

The first solution would be to modify the start level of the bundle that you want to start later.

Apache Felix offers the “bundlelevel” command:

bundlelevel - set bundle start level or initial bundle start level
scope.
flags:
-i, --setinitial set the initial bundle start level
-s, --setlevel set the bundle's start level

So something like:

 bundlelevel -s newStartLevel bundleId

will do the trick.

The advantage  of this solution is that you do not need any programming skills to do it and you can apply it on any bundle (even on the bundles that you are not controlling the content).

The drawback of this solution is that is totally manual (at least in case of Apache Felix server).

The OSGI specification also  defines the OSGI Start Level API which provides the following functions:

  • Controls the beginning start level of the OSGi Framework.
  • Is used to modify the active start level of the Framework.
  • Can be used to assign a specific start level to a bundle.
  • Can set the initial start level for newly installed bundles.

Using the OSGI Start Level API it is possible to programmatically set the start level:

Bundle bundle = framework.getBundleContext().installBundle(location);
BundleStartLevel bundleStartLevel = bundle.adapt(BundleStartLevel.class);
bundleStartLevel.setStartLevel(xxx);

Solution – use a BundleListener

The basic idea is that the bundle B (than needs the bundle A to be active) will wait until the the bundle A is marked as started. This can be achieved by implementing a BundleListener to the level of bundle B.

The implementation of the “bundleChanged” method of the listener will look like this:

public void bundleChanged(BundleEvent bundleEvent) {
    String symbolicName = bundleEvent.getBundle().getSymbolicName();
    int eventType = bundleEvent.getType();

    if ("The Bundle A Symbolic Name".equals(symbolicName)
        && BundleEvent.STARTED == eventType) {
        //here we know that bundle A is started
        //so can do something that will need
        //bundle A
     }
}

The advantage of this approach is that the bundle developer is in control of the behavior. On the the other side this approach will not work if you do not own the code of the bundle that you want to start later.

How to programmatically set-up a (HTTP) proxy for a Selenium test

Context

In the context of a (Java) Selenium test it was needed to set-up a http proxy at the level of the browser. What I wanted to achieve it was exactly what is shown in the next picture but programmatically. In this specific case the proxy was BurpPro proxy but the same workflow can be applied for any kind of (http) proxy.

Solution

I know this is not really rocket science but I didn’t found elsewhere any clear explanation about how to do it. In my code the proxy url is injected via a (Java) system property called “proxy.url“.

And the  code looks like this:

String proxyUrl = System.getProperty("proxy.url");
if (proxyUrl != null) {
    Proxy proxy = new Proxy();
    proxy.setHttpProxy(proxyUrl);

    FirefoxOptions options = new FirefoxOptions();
    options.setProxy(proxy);
    
    driver = new FirefoxDriver(options);
} else {
    driver = new FirefoxDriver();
}

How to intercept and modify Java stacktraces

This ticket was triggered by a “simple” requirement: “Change all the package names in the logs of a Java application (especially the stacktraces) from ‘abc.efg’ (put here whatever you want as name) to ‘hij.klm’ (put here also whatever you want as name) “. The first idea that popped in my mind was to change the packages names at the code level, but this was not feasible because of (rather) big codebase, the use of the (Java) reflexion and the tight timeline.

In the following lines, I will discuss possible solutions to implement this (weird) requirement.

 

Extend the log4j ThrowableRenderer

If the project is using log4j1x as log library, then a solution would be to create your own throwable renderer by extending the org.apache.log4j.spi.ThrowableRenderer. The (log4j) renderers are used to render instances of java.lang.Throwable (exceptions and errors) into a string representation.

The custom renderer that replaces the packages starting with “org.github.cituadrian” by “xxx.yyy” will look like this:

package org.github.cituadrian.stacktraceinterceptor.log4j;

import org.apache.log4j.DefaultThrowableRenderer;
import org.apache.log4j.spi.ThrowableRenderer;

public class CustomThrowableRenderer implements ThrowableRenderer {
    private final DefaultThrowableRenderer defaultRenderer =  
                   new DefaultThrowableRenderer(); 
    
    @Override 
    public String[] doRender(Throwable t) {
      String[] initialResult = defaultRenderer.doRender(t); 
      for (int i = 0; i < initialResult.length; i++) { 
        String line = initialResult[i]; 
        if (line.contains("org.github.cituadrian")) { 
           initialResult[i] = line.replaceAll("org.github.cituadrian", "xxx.yyy"); 
        } 
       } 
      return initialResult; 
    }
}

Basically, the custom renderer is delegating the task of creating a String from a Throwable to a DefaultThrowableRenderer and then it checks and replace the desired package names.

In order to be used, the renderer should be defined in the log4j.xml file:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration debug="true"
    xmlns:log4j='http://jakarta.apache.org/log4j/'>
 
  <throwableRenderer class= 
     "org.github.cituadrian.stacktraceinterceptor.log4j.CustomThrowableRenderer"/>
...

Use a log4j2 pattern layout

If your project is using log4j2 as logging library, then you can use a (log4j2) layout pattern.  The layout pattern will look like:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
 <Appenders>
 <Console name="STDOUT" target="SYSTEM_OUT">
 <PatternLayout pattern=
  "%replace{%class %log %msg %ex}{org\.github\.cituadrian}{xxx\.yyy}"/>
 </Console>
...

 

Modify (a.k.a. Weaving) the java.lang.StackTraceElement class with AOP

Before even explaining what it really means, I have to warn you that weaving JDK classes is rarely necessary (and usually a bad idea) even if it’s possible using an AOP framework like AspectJ.

For this case I used the AspectJ as AOP framwork because the weaver (aop compiler) is able to do binary weaving, meaning the weaver takes classes and aspects in .class form and weaves them together to produce binary-compatible .class files that run in any Java VM. The command line to obtain a weaved jar is the following one:

ajc -inpath rt.jar Aspect.java -outjar weavedrt.jar

In the case of weaving JDK classes one extra step is necessary in order to make the application work; we must create a new version of the rt.jar file  or create just a small JAR file with the JDK woven classes which then must be appended to the boot-classpath of the JDK/JRE when firing up the target application. The command line to execute the target application is the following one:

java -Xbootclasspath/<path to weavedrt.jar>;<path to aspectjrt.jar> TargetApplication

If you don’t want to worry about all the technical details of weaving and executing the application and you are using Maven then you can use the (marvelous) SO_AJ_MavenWeaveJDK project from gitHub (that handles everything using Maven)

The aspect that will modify the stacktrace packages looks like:

package org.github.cituadrian.stacktraceinterceptor.app;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
@Aspectpublic class StackTraceInterceptorAspect {
    @Pointcut("execution(String java.lang.StackTraceElement.getClassName()) "
            + "&& !within(StackTraceInterceptorAspect)")     
    public void executeStackTraceElementGetClassNamePointcut() {}        
    
    @Around("executeStackTraceElementGetClassNamePointcut()")    
    public Object executeStackTraceElementGetClassNameAdvice(    
                   final ProceedingJoinPoint pjp) throws Throwable {        
        Object initialResponse =  pjp.proceed();         
        if (initialResponse instanceof String 
               && ((String)initialResponse).startsWith("org.github.cituadrian")) {     
                 return ((String)initialResponse).replaceFirst("org.github.cituadrian", "xxx.zzz"); 
        }        
        return initialResponse;    
    }
 }

In a nutshell, the StackTraceInterceptorAspect will intercept all the calls to the java.lang.StackTraceElement#getClassName method and it will change the returned result of the method if the class name contains the string “org.github.cituadrian”.

If you are interested to learn more about AspectJ I really recommend you to buy a copy of the AspectJ in action (second edition) book.

 

Modify and shadow the java.lang.StackTraceElement class

 Using AOP just to intercept and modify a single method of a single class is a little bit over-killing. In this case there is another solution; the solution would be create a custom version of the java.lang.StackTraceElement class and add this custom class in the boot-classpath of the JDK/JRE when firing up the target application, so the initial version will be shadowed by the custom version.

An implementation of StacktraceElement class can be found here. So you can modify by hand the java.lang.StackTraceElement#getClassName method or the java.lang.StackTraceElement#toString method.

 To execute the target application, you must create a jar with the modified class and add it into the boot-classpath (something similar to the AspectJ solution):

java -Xbootclasspath/<path to custom class.jar> TargetApplication

 

 

How to remotely connect to an in-memory HSQLDB database

hsqldbVery often in-memory instances of HSQLDB are used in the context of unit tests; the unit test starts a database instance (eventually on a random port), provision the database with some data, run the test against the database end then stop it.

Now, from time to time you need to debug the unit tests and sometimes you also need to run manually some queries on the in-memory database (using HSQLDB manager or any other software).

So here are the steps in order to be able to connect to an in-memory HSQLDB instance:

  • Start the DB in the “remote open db” mode.This is driven by the “server.remote_open” property that is false by default (you can look to the code of org.hsqldb.server.ServerProperties class to see other properties that might be interesting). The code that starts a dataabse instacne in remote open mode will look something like:
HsqlProperties props = new HsqlProperties();
props.setProperty("server.remote_open", true);
....add more properties here
Server server = new Server();
server.setProperties(props);
server.start();
  • Connect to the data base instance. Now that the database has started the next step is to connect to data base instance. Because we started an in-memory instance the connection url is a memory database url that looks like jdbc:hsqldb:mem:instanceName You will not be able to connect using this url because neither the host and the port are available, instead you should use a server database url that looks like jdbc:hsqldb:hsql://host:port/instanceName

 

How to find (buggy) calls of java.lang.Object.equals() with incompatible types

Introductionjavabug

All this started from a change (and a mistake) done by one of my co-workers that did a small change in the code with rather dramatic consequences;

So, imagine we have a (java) class T having a private field f of type String:

public class T {
    private Boolean f;

    public void setF(final Boolean f) {
        this.f = f;
    }
    public Boolean getF() {
        return f;
    }
    .... 
}

Now, imagine (again) that the type of f field will change from String to something else, let’s say Boolean; the compiler will help you to find the places where the setF method is called with the wrong parameter and also where the getF method do not comply with the new signature.

But there is at least one case where the compiler cannot help you; it is the case of the call of java.lang.Object.equals method. So something like “aString”.equals(t.getF()) in the case when the f field is a boolean will always return false because we try to check the equality for incompatible types.

The goal of this ticket is to find solutions to catch this kind of problem at runtime or (ideally) at compile time. Anyway, if you want to catch this kind of problem in your code and you do not want to reinvent the wheel (as I will do in this ticket), just use the FindBugs. FindBugs is capable to catch this kind of error (EC_UNRELATED_TYPES) and many, many more (see FindBugs Bug Descriptions).

How to find the buggy calls at runtime

The only way that I see to catch the buggy equals calls in a generic way is to use AOP. The basic idea is to write an aspect that will intercept all the calls to the equals methods and check that the caller and the callee are of the same type.

For the implementation I choose to use AspectJ framework (for a very good introduction to AspectJ I strongly recommend the AspectJ in action book) and the code it looks like this:

import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.aspectj.lang.annotation.Pointcut;
import org.github.adriancitu.equals.WrongUseOfEqualsException;

@Aspect
public class EqualsCheckerAspect {
    @Pointcut(
            "execution(public boolean Object+.equals(Object)) "
            + "&& target(target) "
            + "&& args(methodArgument)"
            + "&& !within(EqualsCheckerAspect)")
    public void executeEqualsPointcut(
            final Object target, 
            final Object methodArgument) {
    }
    
    @Before("executeEqualsPointcut(target,methodArgument)")
    public void executeEqualsAdvice(
            final Object target, 
            final Object methodArgument) throws Exception {
       
        if (target != null && !target.getClass().equals(methodArgument.getClass())) {
            throw new WrongUseOfEqualsException("Tried to call the equals methods on different classes types");
        }
    }
}

I will try briefly to explain what the previous code is doing; this is not a tutorial about AspectJ or AOP so, some of the definitions will be a little bit approximative, so AOP purists please forgive me.

The aspect (which is a Java class annotated with the @Aspect annotation) contains 2 parts; the pointcut (the method annotated with the @Pointcut annotation) and the advice (the method annotated with the @Before annotation).

The pointcut represents the parts of program flow that we want to intercept; in our case we want to intercept all the calls to the method having the following signature boolean equals(Object) on any instance of java.lang.Object class and on any subclasses (that’s the meaning of the + from the ..boolean Object+.equals). The  target and args are just AspectJ structures that helps to capture the caller and the callee of the equals method.

The advice represents the code that will be executed when a poincut is intercepted. In our  case we would like to do the verification that the caller and the callee are of the same type before the execution of the equals method (this is the meaning of the @Before annotation). The advice code represented by the executeEqualsAdvice method is quite simple, it just retrieve the caller and the callee objects (already captured by the target and args) and check that are instances of the same class.

This solution will fix the problem but it have some drawbacks:

  • the check is done at runtime
  • you have a dependency on AspectJ

How to find the buggy calls at compile time

It would be really nice to be able to catch this problem directly at compile time and eventually (not mandatory) having no other technical dependency.

Our salvation is coming from the javac plug-in mechanism which is new in Java8. The javac plug-in mechanism allows to a user to specify one or more plug-ins on the javac command line, to be started soon after the compilation has begun. There is no official tutorial about how to craft and use a javac plug-in, the only information that I’ve found is this Javadoc link and this blog ticket.

The javac plug-in mechanism gives access to the abstract syntax tree of the compiled program by implementing the visitor pattern so we will use this mechanism to catch and check the executions of the equals methods. The entire code of the plug-in can be found on this GitHub repository.

The steps of crafting a (javac) plug-in are the following ones:

  1. The entry point of the plug-in should implement the com.sun.source.util.Plugin interface. The code of the class can be found here : RuntimeEqualsCheckPlugin.java
  2. Create a class that implements the com.sun.source.util.TaskListener interface in order to perform additional behavior after the type-checking phase. The code of the class can be found here: EqualsMethodCallTaskListener.java
  3. Create an abstract syntax tree visitor by extending the com.sun.source.util.TreePathScanner<R,P>
    end extend the behavior for the method invocation. The code for this class can be found here: CodePatternTreeVisitor2.java

The most important part of CodePatternTreeVisitor2.java class is the overridden visitMethodInvocation method. I will show you the code but the most important lesson that I learned is that in the code you have to work with trees, everything is a subtype of the Tree interface.

public Void visitMethodInvocation(MethodInvocationTree methodInvocationTree, Void aVoid) {
  final List<? extends ExpressionTree> arguments = methodInvocationTree.getArguments();
  final ExpressionTree methodSelect = methodInvocationTree.getMethodSelect();
  switch (methodInvocationTree.getKind()) {
      case METHOD_INVOCATION:
        Tree.Kind methodSelectKind = methodSelect.getKind();
        switch (methodSelect.getKind()) {
            case MEMBER_SELECT:
             //t1.equals
             //or
             //field.equals
             MemberSelectTree memberSelectTree = (MemberSelectTree) methodSelect;
             //it's a equals method invocation
             if (isEqualsCall(
                  new TreePath(getCurrentPath(), methodSelect),
                  methodInvocationTree.getArguments() != null ?
                       methodInvocationTree.getArguments().size() : 0)) {
                //t1
                //or
                //field
                ExpressionTree expression = memberSelectTree.getExpression();
                TypeMirror callerType =
                  trees.getTypeMirror(
                            new TreePath(getCurrentPath(), expression));
                Optional argumentType =
                            omputeArgumentType(methodInvocationTree.getArguments());
                if (argumentType.isPresent() 
                      && !callerType.equals(argumentType.get())) {
                      System.err.println("Try to call equals on different parameters at line "
                          + getLineNumber(methodInvocationTree)
                          + " of file " +
                          currCompUnit.getSourceFile().getName()
                          + "; this is a bug!"
                                );
                 }
              }
          }
        }
        return super.visitMethodInvocation(methodInvocationTree, aVoid);
    }

The steps to execute the (javac) plug-in are the following ones:

  1. Set up a file called com.sun.source.util.Plugin located in META-INF/services/ of your plug-in code because the plug-ins is located via java.util.ServiceLoader. The content of the file should contain the name of the plug-in (com.github.adriancitu.equals.com.github.adriancitu.equals.runtime.RuntimeEqualsCheckPlugin in this case).
  2. Create a jar with all the plug-in files and the com.sun.source.util.Plugin file.
  3. Execute your plug-in against the javac using the  -processorpath flag to indicates the path where the plug-in JAR file is located and -Xplugin flag to indicate the name of the plug-in to run. In our case the command will be something like :
    javac -processorpath plugIn.jar
    -Xplugin:RuntimeEqualsCheckPlugin ./NoDependenciesTest.java

So finally was possible to have a compile time solution, the only drawback that I see for the plug-in solution is that the API is not very user friendly.