Setting up JMX for JConsole / VisualVM on EC2, plus Jetty Configuration

Inspecting applications via JMX can be a very valuable tool in order to keep track of Memory, CPU and any other kind of information that are provided by the MBeans of the remote Java application. However, to get it working correctly can be tricky, specially if you haven’t play with it before, as the documentation is a little sparse.

Before going into the details, here are some very valuable links:

In order to get VisualVM or JConsole working with remote applications you will need to set some JVM properties on the application you want to track. In the list below are the ones that worked for me, but keep in mind that you may have different results depending on the application and the Java version (yes, it happens). In a nutshell, you’ll need to pass these properties to the JVM:

-Dcom.sun.management.jmxremote
 -Dcom.sun.management.jmxremote.port=<port>
 -Dcom.sun.management.jmxremote.authenticate=false
 -Dcom.sun.management.jmxremote.ssl=false
 -Djava.security.policy=<policy file>
 -Dcom.sun.management.jmxremote.local.only=false
 -Djava.rmi.server.hostname=<your public hostname>
 -Dcom.sun.management.jmxremote.password.file=<password_file>

“com.sun.management.jmxremote.port requires the port number to listen for remote RMI connections. Make sure you specify an unused one.

java.rmi.server.hostname” is the public hostname (like example.com) of your server, without HTTP or anything else. You cannot use an internal address, otherwise you won’t be able to remotely access the service.

com.sun.management.jmxremote.authenticate” will require an username and password if set to true, otherwise anyone that knows (or discovers) the addres will be able to connect to your JMX registry.

com.sun.management.jmxremote.password.file” is only necessary if you set authenticate=true, and you should specify the full path of a text file with the username and passwords (in plain text) that are allowed to connect to your instance. More information about the sintaxt at http://docs.oracle.com/javase/1.5.0/docs/guide/management/agent.html#auth.

java.security.policy” is the path of a Java Policy File with the rules you need. For the sake of simplicity, you can create one with the following contents:

grant {
 permission java.security.AllPermission;
 };

You may or may not need to set the policy file (I didn’t).

You don’t need all of these properties in order to the remote JMX working. As I said before, it will depend of your luck. In my very own case, I used only these:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
-Djava.rmi.server.hostname=example.com

You should pass these values when starting your java application, like

java -Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.port=9001 [.....] -jar myapp.jar

Setting up EC2
The most important thing to set when running JMX on Amazon’s EC2 is to open the TCP ports to the IPs you will connect from, otherwise the client application won’t be able to reach the Agent. For the sake of simplicity, open all TCP ports from 0 to 65535, get it working and then keep open only the necessary ports. This approach is easier because the RMI registry that JMX uses (may?) create random ports of its communication, so you’ll have to inspect it (I am no expert on the subject, in other words).

On EC2 you configure the ports in the “Security Groups” section of the AWS Management Console.

Setting up Jetty
As a bonus, here’s how to get JMX working with Jetty. The process is fairly simple, it’s just a matter of adding the configuration keys to the right files.

First of all, you need to edit the file “start.ini” (located in Jetty’s root directory) and uncomment the line that has “etc/jetty-jmx.xml“, which is probably located at the end of the file, as shown in below:

Then, add each of the JVM properties in a different line in the same file (it doesn’t matter where). It should look like this:

Please note that “–exec” is a property specific to Jetty, and necessary to make it understand the other VM properties. “-Xshare:off” disables class data sharing, which is necessary to make VisualVM work correctly. The other settings are the ones described earlier.

The last step is to configure the file “etc/jetty-jmx.xml” to instruct it to automatically create a RMI registry and set the RMI connection properties, which are essential to remotely connect to it. My configuration file (based on Jetty 8) is below:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">
<Configure id="Server" class="org.eclipse.jetty.server.Server">
  <Call id="MBeanServer" class="java.lang.management.ManagementFactory" name="getPlatformMBeanServer"/>
  <New id="MBeanContainer" class="org.eclipse.jetty.jmx.MBeanContainer">
    <Arg>
      <Ref id="MBeanServer"/>
    </Arg>
  </New>
  <Get id="Container" name="container">
    <Call name="addEventListener">
      <Arg>
        <Ref id="MBeanContainer"/>
      </Arg>
    </Call>
  </Get>
  <Call name="addBean">
    <Arg>
      <Ref id="MBeanContainer"/>
    </Arg>
  </Call>
  <Get id="Logger" class="org.eclipse.jetty.util.log.Log" name="log"/>
  <Ref id="MBeanContainer">
    <Call name="addBean">
      <Arg>
        <Ref id="Logger"/>
      </Arg>
    </Call>
  </Ref>
  <Call name="createRegistry" class="java.rmi.registry.LocateRegistry">
    <Arg type="java.lang.Integer">1099</Arg>
    <Call name="sleep" class="java.lang.Thread">
      <Arg type="java.lang.Integer">1000</Arg>
    </Call>
  </Call>
  <New id="ConnectorServer" class="org.eclipse.jetty.jmx.ConnectorServer">
    <Arg>
      <New class="javax.management.remote.JMXServiceURL">
        <Arg type="java.lang.String">rmi</Arg>
        <Arg type="java.lang.String">ec2-177-71-143-137.sa-east-1.compute.amazonaws.com</Arg>
        <Arg type="java.lang.Integer">0</Arg>
        <Arg type="java.lang.String">/jndi/rmi://ec2-177-71-143-137.sa-east-1.compute.amazonaws.com:1099/jettyjmx</Arg>
      </New>
    </Arg>
    <Arg>org.eclipse.jetty:name=rmiconnectorserver</Arg>
    <Call name="start"/>
  </New>
</Configure>

That’s all. When you start Jetty, it will log (to the console or file, depending of how you configured it) the JMX address you will need to use in order to connect to it from your local machine. Please see the following image:

service:jmx:rmi://ec2-177-71-143-137.sa-east-1.compute.amazonaws.com/jndi/rmi://ec2-177-71-143-137.sa-east-1.compute.amazonaws.com:1099/jettyjmx

In my case, the address that I should use in VisualVM is the address of my EC2 instance (ec2-177-71-143-137.sa-east-1.compute.amazonaws.com):

If the connection is successful, then you can right click on the remote host name and insert the address of the RMI registry, like shown below:

If you were lucky enough, you should be able to instrument Jetty remotely:

Leave a Reply

Your email address will not be published. Required fields are marked *

*


* 6 = forty two

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>