Automating shelving of Jenkins jobs

The current version of the shelve plugin does not support shelving multiple projects. This can be a pain if you have a large Jenkins deployment. Luckily, it is possible to shelve multiple projects using a simple bash script.

If you have the jenkis-cli jar on your system, this can be can in two steps:

  1. First generate a list of jobs to shelve:

  1. Iterate over the list to shelve the projects:

This script uses a small delay between every couple shelve requests. This is useful if your jobs contain a lot of data and Jenkins will take too long to shelve them.

 

 

Running Apache Artemis with 32Mb of JVM heap memory

As part of the effort to run Maestro on the Minicloud, I decided to tweak the Maestro broker used by the test cluster. The broker component provides the infrastructure for the test nodes to exchange messages. It is used, for example, to synchronize the start and stop of the test execution and share success and failure notifications.

Given the resource restrictions I mentioned on the previous post, I decided to setup a dual purpose node, serving both as a Maestro client and as as Maestro broker. The virtual machine I setup for this purpose has only 1 vCPU and 512 Mb of RAM. At first sight is looks pretty small, but it should be sufficient for running both of the tasks allocated for it.

To accommodate for these resources, I decided to deploy Apache Artemis as the Maestro Broker.  One the cool things about Artemis is that it can run with ridiculously small amount of memory. The low scale of the Maestro Broker means that I can go pretty aggressive with the resource limitations and, as such, I decided go with as little as 32Mb of heap.

The first step to achieve this, is to adjust the artemis command-line for the download package. In its default setting, the heap is set to vary between 512Mb to 1Gb, which would exceed the capacity of the virtual machine I configured. This script runs the code that creates the broker instance, therefore, it’s not necessary to waste too much time tweaking it. Simply adjusting it for creating the broker instance should be sufficient.

This setting is adjusted on ${install.directory}/bin/artemis. The line to be changes is the one with JAVA_ARGS. I have adjusted the heap to a fixed size of 256Mb.

With that setting in place, the instance can be created. For example:

To start tweaking the instance,  the java heap memory for the instance can be adjusted. That is done on the  ${instance.directory}/etc/artemis.profile. Once again, the line to be adjusted is the one with JAVA_ARGS:

That alone should be sufficient for the very low scale of the Maestro Broker. However, given the very low memory settings, it is a good idea to disable a couple of extra things: the console and unused acceptors.

To disable the console we can modify the ${instance.directory}/etc/bootstrap.xml. The trick is to remove (or comment) the web element and all its contents. This is the content that should be removed:

Then, the last trick is to leave only the acceptors that are intended to be used in this broker instance. Since Maestro broker relies on MQTT, that’s the only one that can be left there. The acceptors configuration for Apache Artemis should look like this:

After that, the broker can be started. If you are unsure about the configuration, you can check the command lines using ps or jcmd.

In addition to these changes, the Artemis documentation mentions a lot of other tunnables that can be adjusted. For such a slow scale broker as this, I did not find the reason to adjust anything else other than what is mentioned here.

 

 

 

Minicloud: the Free OpenPower cloud

I recently got access to the Minicloud, a Free OpenPower Cloud hosted by Unicamp, one of the most prominent universities in Brazil. This cloud is a good way to ensure that your application provides good support for the Power architecture.

Having access to this cloud, I can help ensure that some of the projects I am interested about can be ran and supported on this architecture. Additionally, it offers me a Power environment to test the messaging-related packages I maintain on Fedora and CentOS – entirely on my free time.

To start with this cloud, I decided to go big and deploy Maestro there. Having only 4Gb, it means that I need to adjust very well the resources of my VMs so that I can maximize the usability of the cloud environment. I will write some posts about this.

The other project that I am interested in playing with in the Minicloud is Paho C, the MQTT client from Eclipse Foundation. As the Fedora maintainer for the package, having access to this environment means that I have a personal environment where I can ensure the quality of the packages.

Making it easy to play with Maestro

Maestro is a collection of tools that can be used for distributed performance testing. It is oriented towards testing of messaging oriented middleware (MOM), such as Apache ActiveMQ and its successor Apache Artemis , but can be extended to other performance test use cases.

Because Maestro forms a test cluster, it requires several components to run and deploying it can be tricky.

In order to simplify the development process as well as simplify it for users hoping to play with and run local tests, I created a set of containers (handled by Docker Compose) that can deploy a Maestro Test Cluster for local testing.

The process is extremely simple and you can go from 0 to the actual test execution in 3 commands (supposing docker-compose is already installed on your system):

1. Build and run the containers:

2.Run the client:

3. Execute a test (within the client):

After the test is completed, just point the browser to http://localhost:8000/ to read the report and view the details.

Additional details about this, such as how to tweak the tests or use a different messaging project, are available in this document within the Maestro Java source code.

 

EOY 2016 links

Some links about interesting things I’ve read or watched by the end of 2016. Further comments are provided on each item of the list.

 

Google Sanitizers and Fedora 25

Here’s a tip if you are using Fedora and trying to use one of the Google Sanitizers with clang and is having problems doing so. One of the common problems is not being able to find libclang_rt.asan-x86_64.a:

This can be resolved by simply installing the compiler-rt package: