Saturday, October 17, 2009

How to use jMeter in server mode over an SSH tunnel

INTRODUCTION

Apache JMeter is open source software, a 100% pure Java desktop application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions. (http://jakarta.apache.org/jmeter/)

jMeter can be run in "GUI", "non-GUI" and "server" modes. You can control and monitor multiple instances of  jMeter running in server mode with a single instance that is running in GUI mode. You may want to do this over a secure SSH connection when connecting to remote servers.

(This how-to covers tunneling of the jMeter instance-to-instance communication, not the jMeter-to-the-tested-application communication.)

THE SOLUTION

jMeter uses RMI with callbacks for inter-instance communication. We will configure jMeter and the SSH tunneling based on the great article "SSH Tunneling for Java RMI, Part-II" by Pankaj Kumar.

We will need to:
1. Force jMeter on server side (server mode) and client side (GUI mode) to bind to localhost
2. Set the RMI registry port and the method  invocation port of jMeter server to an arbitrary value
3. Force jMeter client to listen for RMI callbacks on an arbitraty port
4. Tunnel the RMI ports

To achieve points 1 and 3 we will need to patch jMeter source code and build it. The patch applies only to jMeter version 2.3.4.

I will suggest my jMeter code changes to be incorporated into the jMeter project. This may take some time and the changes may be even refused, so in between the patch is probably the only way how to tunnel over SSH. It is possible that if the code will be accepted by jMeter project, it will be somehow modified and the configuration for the next versions will be different.

WHAT THE PATCH DOES

- Adds jMeter parameter server.rmi.localhostname - jMeter by default 1) binds to the host's hostname and 2) refuses to bind to localhost. If this parameter is set, jMeter will bind to the specified ip/hostname and (only if this parameter is set) won't complain even if it is localhost.

- Adds jMeter parameter client.rmi.callbackport - jMeter will listed on this port for RMI callbacks.

PREREQUISITIES

JDK
Ant
SSH client, e.g. Putty (http://www.chiark.greenend.org.uk/~sgtatham/putty/)

HOW TO

Let's assume that:
- server RMI registry port is 55501
- server RMI method invocation port is 55511
- client RMI callback port is 55512

1. Download both the jMeter 2.3.4 binaries and source code (http://jakarta.apache.org/jmeter/) and extract them into the same directory of your choice

2. Download my patch and copy it over the original jMeter source code.

3. Build jMeter (http://wiki.apache.org/jakarta-jmeter/BuildingJMeter)

4. Copy it to both the server and client machine

5. Edit the server jmeter.properties:
server_port=55501
server.rmi.localhostname=127.0.0.1
server.rmi.localport=55511

6. Edit the client jmeter.properties:
remote_hosts=127.0.0.1:55501
client.rmi.callbackport=55512

You will also want to switch to batch mode to lower the network traffic:
mode=Batch
num_sample_threshold=250

7. Configure the tunnelling
Local port: 55501 -> 127.0.0.1:55501
Local port: 55511 -> 127.0.0.1:55511
Remote port: 55512 -> 127.0.0.1:55512

8. Start the jMeter server and client using (to associate remote stubs with localhost):
server (example for linux): ./jakarta-jmeter-2.3.4/bin/jmeter-server -Djava.rmi.server.hostname=127.0.0.1 &
client (example for win): jakarta-jmeter-2.3.4\bin\jmeter.bat -Djava.rmi.server.hostname=127.0.0.1

Done!

Next part: How to use jMeter in server mode over an SSH tunnel - Part 2 - Mutliple server instances

16 comments:

  1. thanks for this post. i have a question regarding step 7.do you mean that we need to create a tunnel between the client and server where all traffic going to port 55501 & 55511 on the client gets forwarded to the remote machine's 55501 & 55511 port?

    also, does a tunnel need to be setup from the remote to the client machine. this time using port 55512?

    im trying to run thru the configuration you setup and am getting the following exception:

    2009/12/16 18:00:03 INFO - jmeter.gui.action.RemoteStart: Initialising remote engine: 127.0.0.1:55501
    2009/12/16 18:01:03 ERROR - jmeter.gui.action.RemoteStart: Failed to initialise remote engine java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
    java.net.SocketTimeoutException: Read timed out

    ReplyDelete
  2. You are welcome.

    It means that all local traffic going to (localhost) ports 55501 and 55511 should be forwarded to the localhost interface at the remote machine. Also all traffic on remote machine going to (localhost) port 55512 should be forwarded to the port 55512 on the local localhost interface.

    (For more info about the remote and local ports terminology you can also have a look e.g. into the putty documentation.)

    ReplyDelete
  3. Hi,

    Congratulations for your post. I've followed the steps, and I can execute the test from my client host, and are executed in the server host.

    The problem is the listener in my clien. The results don't show in the listeners of my client. The logs of jmeter-server are:

    2010/02/10 17:05:52 INFO - jmeter.engine.StandardJMeterEngine: Running the test!
    2010/02/10 17:05:52 INFO - jmeter.samplers.RemoteListenerWrapper: Test Started on 127.0.0.1:55501
    2010/02/10 17:05:52 ERROR - jmeter.samplers.RemoteListenerWrapper: testStarted(host) java.rmi.ConnectException: Connection refused to host: 127.0.0.1; nested exception is:
    java.net.ConnectException: Connection refused
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:574)
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:185)
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:171)
    at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:94)
    at org.apache.jmeter.samplers.RemoteSampleListenerImpl_Stub.testStarted(Unknown Source)
    at org.apache.jmeter.samplers.RemoteListenerWrapper.testStarted(RemoteListenerWrapper.java:75)
    at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:277)
    at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:413)
    at java.lang.Thread.run(Thread.java:595)
    Caused by: java.net.ConnectException: Connection refused


    Any suggestions? Thank yoy very much!

    ReplyDelete
  4. Hi Fernando,
    hm hm, you could maybe verify that the tunnels are open using "netstat -tan | grep LISTEN" (linux syntax) on client (you should see 55501 & 55511) and server machines (55512) and that you have started both the jMeters using the correct parameters.. Please let me know if that helped.

    ReplyDelete
  5. Thank you. I've created the tunnels again, and started the server an client jmeter witch the correct parameter. Now, the listeners show me the results of the test.

    ReplyDelete
  6. Thank you for this and your instructions! This is going to be great for my amazon ec2 stress testing!

    ReplyDelete
  7. Hi,
    I have run all the steps described above yet I am getting an expception indicating that connection is refused in jmeter-server.log
    Would you please elaborate more about the tunnels that should be created, should all the tunnels created on the client machine?

    ReplyDelete
  8. Hi,
    yes, you open all the tunnels from the client side, you configure them in the SSH client software (e.g. putty). You can run the netstat command on the client and the server computers, to verify the tunnels are open, please see my previous comments above.

    Peter

    ReplyDelete
  9. Hi,
    I need to run jmeter-server on 5 slave machines, and controlling it from the jmeter client. I do have firewall blocking issue, so I did patch the jmeter code using your patch code. I created all ssh tunnels on the jmeter client machine, run the 5 jmeter-servers on the remote jmeter-server machines.
    Then I run the 'jmeter' client in the GUI mode. the GUI gave to the 'NotBoundException'. Look like it does not work in the GUI mode. I then run the -n (NON-GUI) mode, and looked at the remote jmeter-server.log, but see no transaction?
    My question is whether the patch code handles MULTIPLE jmeter-server RMI over SSH tunnel connections? If so, what could go wrong?
    Thanks.
    Henry

    ReplyDelete
  10. Hi Henry,
    I assume you need to configure the multiple jmeter server instances to use distinct ports e.g. 55501,55502,55501 for rmi registry port, 55511, 55510, 55509 for rmi invocation port and create tunnels for those ports. I assume it is enough to have one client callback port (55512). Client is supposed to work well in GUI mode. NotBoundException means that some of the ports the client jmeter is trying to use is already taken by some other application (or jmeter itself). If you configured the same port number for multiple server instances it can be the case (?).
    Cheers,
    Peter

    ReplyDelete
  11. Hi Peter,
    What a fantastic blog post! You are my person of the week.

    I tend to work over multiple projects a lot and this gem of a tip will really help me getting access to the hardware when I need it (now) and where I want it (located on the same subnet as the AUT, not on my desktop!)

    1 to 1 this works fine, I can connect, trigger the run, and see the results - all using port 22. Need an EC2 instance running as a slave to your desktop? No problem!

    But when I tried connecting multiple instances I ran into issues. The tests ran ok and the remote prompt gave the happy Starting the test..., Finished the test... messages but the client prompt hung and no results came back.

    My prompt looks like this:
    $ ./jmeter -n -r -t /tmp/test.jmx -l /tmp/itworks.jtl -Djava.rmi.server.hostname=127.0.0.1

    Created the tree successfully using /tmp/test.jmx
    Configuring remote engine for 127.0.0.1:55501
    Using remote object: UnicastRef [liveRef: [endpoint:[127.0.0.1:55511](remote),objID:[-38173f4:12f8ef3e582:-7fff, -7796582484032537710]]]
    Configuring remote engine for 127.0.0.1:55502
    Using remote object: UnicastRef [liveRef: [endpoint:[127.0.0.1:55512](remote),objID:[67754b3d:12f8ef3f896:-7fff, -2301734091042913621]]]
    Starting remote engines
    Starting the test @ Tue Apr 26 00:18:17 BST 2011 (1303773497705)
    Remote engines have been started

    and then nothing...

    I setup two sets of tunnels, basically like this:
    Local port: 55501 -> 127.0.0.1:55501
    Local port: 55511 -> 127.0.0.1:55511
    Local port: 55502 -> 127.0.0.1:55502
    Local port: 55513 -> 127.0.0.1:55513
    Remote port: 55512 -> 127.0.0.1:55512

    I also tried adding a second remote port:
    Remote port: 55519 -> 127.0.0.1:55519

    But this failed on the server when I tried to run RMI saying already in use.

    Any ideas?

    But still MASSIVE kudos for this solution, you have my thanks and respect.

    ReplyDelete
  12. Hi, I am glad that you find my article useful.

    Those, who have problems running multiple server instances might be interested in the next part: "How to use jMeter in server mode over an SSH tunnel - Part 2 - Mutliple server instances".

    Cheers,

    Peter

    ReplyDelete
  13. hi,
    i am new to J-meter. i had used J-meter perfectly to test the load on my application server locally. i mean application server and J-meter client both are running on same machine.
    now i have deployed the application on a remote server and J-meter client is running on my local...but this time it is not able to connect to the application using J-meter proxy.

    any help would be highly appreciated.

    thanks,

    ReplyDelete
  14. Hi,
    I have 2 machines in east coast and one machine from singapore location. when i add machine located at singapore, iam getting connection refused.

    All ports are open, nofirewall issues and machines are pingable.

    Can anyone assist as to how i can connect to the singapore machine which is in different subnet
    Thanks

    ReplyDelete
  15. 2013/03/19 14:20:17 INFO - jmeter.gui.action.RemoteStart: Initialising remote engine: :50000
    2013/03/19 14:20:38 ERROR - jmeter.gui.action.RemoteStart: Failed to initialise remote engine java.rmi.ConnectException: Connection refused to host: ; nested exception is:
    java.net.ConnectException: Connection timed out: connect

    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(Unknown Source)

    at sun.rmi.transport.tcp.TCPChannel.createConnection(Unknown Source)

    at sun.rmi.transport.tcp.TCPChannel.newConnection(Unknown Source)

    at sun.rmi.server.UnicastRef.newCall(Unknown Source)

    at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)

    at java.rmi.Naming.lookup(Unknown Source)

    at org.apache.jmeter.engine.ClientJMeterEngine.getEngine(ClientJMeterEngine.java:54)

    at org.apache.jmeter.engine.ClientJMeterEngine.(ClientJMeterEngine.java:67)

    at org.apache.jmeter.gui.action.RemoteStart.doRemoteInit(RemoteStart.java:176)

    at org.apache.jmeter.gui.action.RemoteStart.doAction(RemoteStart.java:79)

    at org.apache.jmeter.gui.action.ActionRouter.performAction(ActionRouter.java:81)

    at org.apache.jmeter.gui.action.ActionRouter.access$000(ActionRouter.java:40)

    at org.apache.jmeter.gui.action.ActionRouter$1.run(ActionRouter.java:63)

    at java.awt.event.InvocationEvent.dispatch(Unknown Source)

    at java.awt.EventQueue.dispatchEventImpl(Unknown Source)

    at java.awt.EventQueue.access$200(Unknown Source)

    at java.awt.EventQueue$3.run(Unknown Source)

    at java.awt.EventQueue$3.run(Unknown Source)

    at java.security.AccessController.doPrivileged(Native Method)

    at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)

    at java.awt.EventQueue.dispatchEvent(Unknown Source)

    at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)

    at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)

    at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)

    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)

    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)

    at java.awt.EventDispatchThread.run(Unknown Source)

    Caused by: java.net.ConnectException: Connection timed out: connect

    at java.net.DualStackPlainSocketImpl.connect0(Native Method)

    at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)

    at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)

    at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)

    at java.net.AbstractPlainSocketImpl.connect(Unknown Source)

    at java.net.PlainSocketImpl.connect(Unknown Source)

    at java.net.SocksSocketImpl.connect(Unknown Source)

    at java.net.Socket.connect(Unknown Source)

    at java.net.Socket.connect(Unknown Source)

    at java.net.Socket.(Unknown Source)

    at java.net.Socket.(Unknown Source)

    at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(Unknown Source)

    at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(Unknown Source)

    ... 27 more

    ReplyDelete
  16. This comment has been removed by a blog administrator.

    ReplyDelete