Apache RocketMQ benchmarks

This tutorial shows you how to run OpenMessaging benchmarks for Apache RocketMQ. You can currently deploy to the following platforms:

Initial setup

To being, you’ll need to clone the benchmark repo from the openmessaging organization on GitHub:

$ git clone https://github.com/openmessaging/openmessaging-benchmark
$ cd openmessaging-benchmark

You’ll also need to have Maven installed.

Create local artifacts

Once you have the repo cloned locally, you can create all the artifacts necessary to run the benchmarks with a single Maven command:

$ mvn install

Deploy a RocketMQ cluster on AlibabaCloud

You can deploy a RocketMQ cluster on AliCloud (for benchmarking purposes) using Terraform and Ansible. You’ll need to have both of those tools installed as well as the terraform-inventory plugin for Terraform.

In addition, you’ll need to:

Export the AccessKey and SecretKey

Once you’re all set up with AliCloud account, you’ll need to export the AccessKey and SecretKey.


Create resources using Terraform

With AccessKey and SecretKey in place, you can create the necessary AliCloud resources using just a few Terraform commands:

$ cd driver-rocketmq/deploy
$ terraform init
$ terraform apply

This will install the following ECS instances (plus some other resources, such as public IPs, a security group.):

Resource Description Count
RocketMQ Broker instances The VMs on which a RocketMQ broker will run 2
RocketMQ NameServer instances The VMs on which a RocketMQ NameServer node will run 1
Client instance The VM from which the benchmarking suite itself will be run 4

When you run terraform apply, you will be prompted to type yes. Type yes to continue with the installation or anything else to quit.

Once the installation is complete, you will see a confirmation message listing the resources that have been installed.


There’s a handful of configurable parameters related to the Terraform deployment that you can alter by modifying the defaults in the terraform.tfvars file.

Variable Description Default
region The AliCloud region in which the RocketMQ cluster will be deployed cn-hangzhou
availability_zone The availability zone of the specific region cn-hangzhou-b
private_key_file The name of file that can save key pairs data source after running terraform plan. rocketmq_alicloud.pem
image_id The AliCloud Machine Image to be used by the cluster’s machines centos_7_04_64_20G_alibase_201701015.vhd
instance_types The ECS instance types used by the various components ecs.mn4.xlarge (RocketMQ brokers), ecs.n4.xlarge (RocketMQ nameserver), ecs.mn4.xlarge (benchmarking client)

If you modify the private_key_file, make sure that you point to the appropriate SSH key path in ansible.cfg when running the Ansible playbook.

Running the Ansible playbook

With the appropriate infrastructure in place, you can install and start the RocketMQ cluster using Ansible with just one command:

$ ansible-playbook \
  --user root \
  --inventory `which terraform-inventory` \

SSHing into the client host

In the output produced by Terraform, there’s a client_ssh_host variable that provides the IP address for the client EC2 host from which benchmarks can be run. You can SSH into that host using this command:

$ ssh -i rocketmq_alicloud.pem root@$(terraform output client_ssh_host)

Running the benchmarks from the client hosts

The benchmark scripts can be run from the /opt/benchmark working directory.

Once you’ve successfully SSHed into the client host, you can run any of the existing benchmarking workloads by specifying the YAML file for that workload when running the benchmark executable. All workloads are in the workloads folder. Here’s an example:

$ sudo bin/benchmark \
  --drivers driver-rocketmq/rocketmq.yaml \

Although benchmarks are run from a specific client host, the benchmarks are run in distributed mode, across multiple client hosts.

There are multiple RocketMQ “modes” for which you can run benchmarks. Each mode has its own YAML configuration file in the driver-rocketmq folder.

Mode Description Config file
Standard Non-batch and at-least-once semantics rocketmq.yaml

The example used the “standard” mode as configured in driver-rocketmq/rocketmq.yaml.

Specify client hosts

By default, benchmarks will be run from the set of hosts created by Terraform. You can also specify a comma-separated list of client hosts using the --workers flag (or -w for short):

$ sudo bin/benchmark \
  --drivers driver-rocketmq/rocketmq.yaml \
  --workers, \ # or -w,

Downloading your benchmarking results

The OpenMessaging benchmarking suite stores results in JSON files in the /opt/benchmark folder on the client host from which the benchmarks are run. You can download those results files onto your local machine using scp. You can download all generated JSON results files using this command:

$ scp -i rocketmq_alicloud.pem root@$(terraform output client_ssh_host):/opt/benchmark/*.json .

Tearing down your benchmarking infrastructure

Once you’re finished running your benchmarks, you should tear down the AliCloud infrastructure you deployed for the sake of saving costs. You can do that with one command:

$ terraform destroy -force

Make sure to let the process run to completion (it could take several minutes). Once the tear down is complete, all AliCloud resources that you created for the RocketMQ benchmarking suite will have been removed.