ElasticSearch:How to enable scripting on the embedded server

The goalelastic

In this ticket I will present all the required steps in order to enable the scripting on an embedded elasticsearch server. I will not explain how to create and use an embedded server but if you need more infos you can read this ticket Embedded Elasticsearch Server for Tests.

If you execute a script within the embedded server the following exception will be thrown;

Caused by: java.lang.IllegalArgumentException: script_lang not supported [xxxxx]
    at  org.elasticsearch.script.ScriptService.getScriptEngineServiceForLang(ScriptService.java:211)

 Code changes

 The easiest way to create an embedded server is the following one:

 Node node = NodeBuilder.nodeBuilder()
              .local (true)
              .settings(
               //add settings here
               )
              .node ();

The problem with this approach is that the NodeBuilder.node() method it will create a node class with an empty list of plug-ins; see the NodeBuilder.node code:

   /*
    * Builds the node without starting it.
    */
    public Node build() {
        return new Node(settings.build());
    }
    //Node(Settings) constructor
     /**
     * Constructs a node with the given settings.
     *
     * @param preparedSettings Base settings to configure the node with
     */
     public Node(Settings preparedSettings) {
        this(InternalSettingsPreparer
                .prepareEnvironment(preparedSettings, null),
             Version.CURRENT,
             Collections.<Class<? extends Plugin>>emptyList());
     }

In order to create a node having a list of desired plug-ins the second constructor of the Node class should be used:

protected Node(
                Environment tmpEnv, 
                Version version, 
                Collection<Class<? extends Plugin>> classpathPlugins)

The problem with this (second) constructor is that it is protected so we will need to extend the Node class to be able to use the protected constructor and we will need to extend also the NodeBuilder class in order to use the new Node class.

And here is the code:

  //PluginNode class 
  private static class PluginNode extends Node {
      public PluginNode(Settings preparedSettings, List<Class<? extends Plugin>> plugins) {
         super(InternalSettingsPreparer.prepareEnvironment(preparedSettings, null), Version.CURRENT, plugins);
      }
  }

  //PluginNodeBuilder class
  public class PluginNodeBuilder extends NodeBuilder {
        private List<Class<? extends Plugin>> plugins = new ArrayList<>();
        
        public PluginNodeBuilder() {
            super();
        }
        //new method to add plug-ins       
        public PluginNodeBuilder addPlugin(Class<? extends Plugin> plugin) {
            plugins.add(plugin);
            return this;
        }

        @Override
        //use the new PluginNode to build a node 
        public Node build() {
            return new PluginNode(settings().build(), plugins);
        }
    };

And how to use the new node builder:

 Node node = new PluginNodeBuilder()
                .addPlugin(ExpressionPlugin.class)
                .addPlugin(GroovyPlugin.class)
                .local (true)
                .settings(
                 //add settings here
                 )
                .node ();

Enable scripting feature

Using the previous Java code it will not be sufficient to execute the scripts; you must also use the right settings at the server creation and add the (Maven) dependencies to your classpath.

Server settings

 Node node = new PluginNodeBuilder()
                .addPlugin(ExpressionPlugin.class)
                .addPlugin(GroovyPlugin.class)
                .local (true)
                .settings (Settings.settingsBuilder ()
                     ...
                     //this is common for all languages
                     .put ("script.inline", true)
                     //this are the settings for the groovy language
                     .put ("script.engine.groovy.inline.mapping", true)
                     .put ("script.engine.groovy.inline.search", true)
                     .put ("script.engine.groovy.inline.update", true)
                     .put ("script.engine.groovy.inline.plugin", true)
                     .put ("script.engine.groovy.inline.aggs", true)
                      
                     //this are the settings for the expression language
                     .put ("script.engine.expression.inline.mapping", true)
                     .put ("script.engine.expression.inline.search", true)
                     .put ("script.engine.expression.inline.update", true)
                     .put ("script.engine.expression.inline.plugin", true)
                     .put ("script.engine.expression.inline.aggs", true)

If you more details about all the possible settings options you can look to the ElasticSearch Scripting documentation.

Maven dependencies

For the expression language:

       <dependency>
            <groupId>org.apache.lucene</groupId>
            <artifactId>lucene-expressions</artifactId>
            <version>5.5.2</version>
            <scope>compile</scope>
        </dependency>

       <dependency>
            <groupId>org.elasticsearch.module</groupId>
            <artifactId>lang-expression</artifactId>
            <version>2.2.0</version>
            <scope>compile</scope>
        </dependency>

For the groovy language:

<dependency>
    <groupId>org.elasticsearch.module</groupId>
    <artifactId>lang-groovy</artifactId>
    <version>2.3.2</version>
</dependency>

ElasticSearch:How to programmatically update settings of an existing index

The goal of this ticket is to show how to update programmatically the settings of anelastic ElasticSeach index. I will take as example the ElaticSeachsynonyms. Imagine that for a specific index, you have the following synonyms settings:

 "analysis": {
      "filter": {
        "synonym_filter": {
          "type": "synonym",
          "ignore_case": true,
          "synonyms": [
            "Romania, RO",
            "Belgium, BE"
          ]
        }
      },
....

Now, imagine (also) that you want to add a new entry into the synonyms lists (“France, FR”). You could do this by using the ElasticSearch  REST interface (please go here if you want to know more Elasticsearch: updating the mappings and settings of an existing index) or you can use the Java API offered by ElasticSearch to do the same task programmatically:

//close the index before the update
client.admin().indices().close(new CloseIndexRequest(indexName));

//update the synonyms
client.admin().indices().prepareUpdateSettings(indexName).setSettings(
             Settings.builder()
                .put(
                        //setting prefix
                        "index.analysis.filter.synonym_filter",
                        //group name
                        "synonyms",
                        //settings
                        new String[] {"0", "1", "2"},
                        //values
                        new String []{
                                 "Romania,RO", 
                                 "Belgium,BE", 
                                 "France,FR"
                                      }
                       )
            ).get();

//open the index
client.admin().indices().open(new OpenIndexRequest(indexName));

 

Useful links:

ElasticSearch Doc – Update Indices Settings

(Unofficial) ElasticSearch Java API for the IndicesAdminClieant interface

(Unofficial) ElasticSearch Java API for the ImmutableSettings.Builder.put method

 

 

How to fix ElasticSearch client exception “A binding to org.elasticsearch.shield.transport was already configured at _unknown_. at _unknown_”

This ticket explains a possible solution  for the “A binding to org.elasticsearch.shield.transport was already configured at _unknown_. at _unknown_” exception when a Java ElasticSearch client tries to connect to a (ElasticSearch) cluster using Shield.

Environment

ElasticSearch version: 1.7.3

Shield version: 1.3.3

Context

The way to connect a Java ElasticSearch client to a cluster using Shield is quite straightforward; you can see the ElasticSearch documentation. The most important part (at least in the context of this problem) is the creation of the Settings instance:

Settings settings = ImmutableSettings.settingsBuilder()
                .put("cluster.name", clusterName)
                .put("shield.ssl.keystore.path", jksPath)
                .put("shield.ssl.keystore.password", jksPassword)
                .put("shield.transport.ssl", "true")
                .put("plugin.types", "org.elasticsearch.shield.ShieldPlugin")
                .build();
......

When the client is executed, the following strange stacktrace is thrown:

Full stacktrace

1) A binding to org.elasticsearch.shield.transport.filter.IPFilter was already configured at _unknown_.
  at _unknown_
2) A binding to org.elasticsearch.shield.transport.ClientTransportFilter was already configured at _unknown_.
  at _unknown_
3) A binding to org.elasticsearch.shield.ssl.ClientSSLService was already configured at _unknown_.
  at _unknown_
4) A binding to org.elasticsearch.shield.ssl.ServerSSLService was already configured at _unknown_.
  at _unknown_
4 errors
       at org.elasticsearch.common.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:344)
       at org.elasticsearch.common.inject.InjectorBuilder.initializeStatically(InjectorBuilder.java:151)
       at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:102)
       at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
       at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
       at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:59)
       at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:195)
       at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:125)

 

Root cause

The root cause of this problem is the line:

.put("plugin.types", "org.elasticsearch.shield.ShieldPlugin")

If this line is removed then the problem is solved. This property should be exclusively used with the 2.0 version of Shield and not with  1.3.3 version.

The moral of this story ? First of all you should use the right version of the ElaticSearch documentation (in my case I was running the 1.7.3  version but I used the documentation for the 2.o). The second point is  that ElasticSearch API is not very user friendly (I even dare to say that is badly designed). I would preferred that ImmutableSettings.Builder class to have a put method with a Java enum as first parameter not a Java String.

Shlomi Zeltsinger

Blockchain made simple