日本語

Performance measurement of SINETStream

Published: December 13, 2019

In this site, we report the performance measurement results of SINETStream performed by the SINETStream development team at the Center for Cloud Research and Development, National Institute of Informatics. We plan to update the report regularly.

Measurement methods and environment

The following three types of programs were prepared, and the performance of writing/reading data via SINETStream was measured.

The measurement environment is as follows.

LAN environment

Writer, Broker, and Reader are all deployed on the server which is connected to the same LAN, and all communications are performed via LAN

Mobile environment

Writer and Reader are deployed on devices which are connected to the SINET mobile network, and communications between Writer and Broker and between Broker and Reader are performed via the mobile network.

Software

Communication performance of mobile network

ping (RTT)

source -> destination minimum average maximum standard deviation
raspi -> aws 28.049 ms 37.384 ms 47.254 ms 5.403 ms

iperf3 (throughput)

source -> destination average throughput
raspi -> aws 12.1 Mbits/s
aws -> raspi 9.90 Mbits/s

The performance of SINETStream

We compare the performance measurement results with four messaging system settings: (1) native Kafka, (2) native MQTT, (3) SINETStream with Kafka backend, and (4) SINETStream with MQTT backend.

The horizontal axis shows the sending message size (100B, 1KB, 64KB), and the vertical axis shows the throughput.

The evaluation results in a mobile environment are considered to be limited by the performance of the mobile network.

  1. Comparison of SINETStream and Native (LAN environment, no encryption)
    aws-aws-plain
  2. SINETStream performance (LAN environment, communication encryption by TLS)
    aws-aws-tls
  3. SINETStream performance (LAN environment, data encryption)
    aws-aws-crypto
  4. Comparison between SINETStream and Native (Reader side mobile environment, no encryption)
    aws-raspi-plain
  5. SINETStream performance (mobile environment on Reader side, communication encryption by TLS)
    aws-raspi-tls
  6. SINETStream performance (mobile environment on Reader side, data encryption)
    raspi-aws-tls
  7. Comparison of SINETStream and Native (Writer side mobile environment, no encryption)
    raspi-aws-plain
  8. SINETStream performance (Writer-side mobile environment, communication encryption by TLS)
    raspi-aws-tls
  9. SINETStream performance (mobile environment on Writer side, data encryption)
    raspi-aws-tls