Running Apache Artemis with 32Mb of JVM heap memory

As part of the effort to run Maestro on the Minicloud, I decided to tweak the Maestro broker used by the test cluster. The broker component provides the infrastructure for the test nodes to exchange messages. It is used, for example, to synchronize the start and stop of the test execution and share success and failure notifications.

Given the resource restrictions I mentioned on the previous post, I decided to setup a dual purpose node, serving both as a Maestro client and as as Maestro broker. The virtual machine I setup for this purpose has only 1 vCPU and 512 Mb of RAM. At first sight is looks pretty small, but it should be sufficient for running both of the tasks allocated for it.

To accommodate for these resources, I decided to deploy Apache Artemis as the Maestro Broker.  One the cool things about Artemis is that it can run with ridiculously small amount of memory. The low scale of the Maestro Broker means that I can go pretty aggressive with the resource limitations and, as such, I decided go with as little as 32Mb of heap.

The first step to achieve this, is to adjust the artemis command-line for the download package. In its default setting, the heap is set to vary between 512Mb to 1Gb, which would exceed the capacity of the virtual machine I configured. This script runs the code that creates the broker instance, therefore, it’s not necessary to waste too much time tweaking it. Simply adjusting it for creating the broker instance should be sufficient.

This setting is adjusted on ${install.directory}/bin/artemis. The line to be changes is the one with JAVA_ARGS. I have adjusted the heap to a fixed size of 256Mb.

With that setting in place, the instance can be created. For example:

To start tweaking the instance,  the java heap memory for the instance can be adjusted. That is done on the  ${instance.directory}/etc/artemis.profile. Once again, the line to be adjusted is the one with JAVA_ARGS:

That alone should be sufficient for the very low scale of the Maestro Broker. However, given the very low memory settings, it is a good idea to disable a couple of extra things: the console and unused acceptors.

To disable the console we can modify the ${instance.directory}/etc/bootstrap.xml. The trick is to remove (or comment) the web element and all its contents. This is the content that should be removed:

Then, the last trick is to leave only the acceptors that are intended to be used in this broker instance. Since Maestro broker relies on MQTT, that’s the only one that can be left there. The acceptors configuration for Apache Artemis should look like this:

After that, the broker can be started. If you are unsure about the configuration, you can check the command lines using ps or jcmd.

In addition to these changes, the Artemis documentation mentions a lot of other tunnables that can be adjusted. For such a slow scale broker as this, I did not find the reason to adjust anything else other than what is mentioned here.

 

 

 

Minicloud: the Free OpenPower cloud

I recently got access to the Minicloud, a Free OpenPower Cloud hosted by Unicamp, one of the most prominent universities in Brazil. This cloud is a good way to ensure that your application provides good support for the Power architecture.

Having access to this cloud, I can help ensure that some of the projects I am interested about can be ran and supported on this architecture. Additionally, it offers me a Power environment to test the messaging-related packages I maintain on Fedora and CentOS – entirely on my free time.

To start with this cloud, I decided to go big and deploy Maestro there. Having only 4Gb, it means that I need to adjust very well the resources of my VMs so that I can maximize the usability of the cloud environment. I will write some posts about this.

The other project that I am interested in playing with in the Minicloud is Paho C, the MQTT client from Eclipse Foundation. As the Fedora maintainer for the package, having access to this environment means that I have a personal environment where I can ensure the quality of the packages.

Making it easy to play with Maestro

Maestro is a collection of tools that can be used for distributed performance testing. It is oriented towards testing of messaging oriented middleware (MOM), such as Apache ActiveMQ and its successor Apache Artemis , but can be extended to other performance test use cases.

Because Maestro forms a test cluster, it requires several components to run and deploying it can be tricky.

In order to simplify the development process as well as simplify it for users hoping to play with and run local tests, I created a set of containers (handled by Docker Compose) that can deploy a Maestro Test Cluster for local testing.

The process is extremely simple and you can go from 0 to the actual test execution in 3 commands (supposing docker-compose is already installed on your system):

1. Build and run the containers:

2.Run the client:

3. Execute a test (within the client):

After the test is completed, just point the browser to http://localhost:8000/ to read the report and view the details.

Additional details about this, such as how to tweak the tests or use a different messaging project, are available in this document within the Maestro Java source code.

 

MQTT Cheat Sheet

The MQTT (Message Queue Telemetry Transport) messaging protocol is a lightweight protocol well suited for the Internet of Things. It is a simple protocol and an ISO standard. It comes with an extensible reference documentation.

To help me learn about it and simplify my work flow, I created this cheat sheet (warning: PDF) which highlights the most relevant details about the protocol. You can also see it online here.

 

EOY 2016 links

Some links about interesting things I’ve read or watched by the end of 2016. Further comments are provided on each item of the list.

 

Some messaging stuff

Just sharing some messaging tools I have been working with recently.

The first one is this performance test tool: msg-perf-tool. There’s no secret here: you run the tool and it does its best to bring your messaging system to its knees (though this may not be the correct way how to test it … check the testing tips on the Github page). For now it supports only AMQP, but Stomp and MQTT support is on the way. You can find rpms for RHEL/Centos (6 and 7) and Fedora (22, 23 and 24) for i686 and x86_64 on my Copr profile here.

The second one is a web page that can display the performance data stored on an ElasticSearch database and present it in a beautiful way. I call it messaging performance center. Here’s how it looks like:

messaging performance tool
mpt-perf-ui

Some bits are still in progress, but it’s functional.

Lastly, there’s litestomp. A C implementation of the Simple (or Streaming) Text Oriented Messaging Protocol. It was built on top of what-seems-to-be the now defunct libstomp project. There’s a couple of bugfixes, a simpler and higher level API for ease of use. Still a work in progress, but you can download the current rpms for RHEL/Centos (6 and 7) and Fedora (22, 23 and 24) for i686 and x86_64 from my Copr project page on this link.