Released June 11, 2024

Features and Highlights

New Grafana Interface

The perfSONAR Toolkit and MaDDash interfaces have been replaced with Grafana! Grafana is a popular visualization platform which is familiar to many of our users. It has a range of plugins that make it easier to display perfSONAR data in useful ways, allows for greater customization and opens the door for integration with other data sources.

When you install or upgrade a perfSONAR Toolkit, the default web interface for displaying results will automatically use Grafana. The installation process will automatically install and configure Grafana for you. It will also provide you with a default set of dashboards that display your results and give details about your perfSONAR host, so no requirement to build dashboards on your own.

We also now offer a way to create MaDDash-like displays in Grafana using a new pSConfig Grafana Agent. If you have an existing MaDDash installation then it will reamin untouched but we are encouraging everyone to move to the new system. A “cookbook” for setting-up a central OpenSearch Archive with Grafana can be found here.

Threaded iperf3 Support

iperf3 3.17.1 with multi-threading is now fully supported by pScheduler. In general, this allows measurement to run at higher throughput than previously possible with single-threaded iperf3. You can leverage this feature with the --parallel option of pscheduler task throughput.

Python pSConfig

pSConfig had been completely re-implemented in Python (formerly in Perl). This will make support easier going forward and is part of a larger effort over the next few releases to transition away from the last remnants of Perl in perfSONAR. There are no changes to the pSConfig template format, this is purely a source code change and should be mostly transparent.

Enhanced perfSONAR Instrumentation

perfSONAR installations now export a greater number of stats about the host system and perfSONAR applications such as pScheduler and pSConfig. This is done using Prometheus node_exporter and some custom perfSONAR endpoints. Information from these endpoints is collected periodically, stored in Opensearch and displayed in the default Grafana dashboards. The new pSConfig also has a host metrics agent than can centrally collect this information from multiple hosts and display them centrally (details in the Central Grafana Cookbook)

Better Troubleshooting of Archiving Issues

perfSONAR now includes a psarchive troubleshoot command to help pinpoint issues when there are problems storing results. It will check connectivity to components like OpenSearch and Logstash, try to fetch data and verify credentials. Along with enhancements to pscheduler troubleshoot and the previously mentioned instrumentation, this a part of an ongoing effort to make it easier to pinpoint problems with your perfSONAR installation.

Updated OS Support

The offically supported operating systems for 5.1.0 are:

  • Alma and Rocky Linux 9 (other EL variants may work, but at your own risk)
  • Debian 11
  • Debian 12
  • Ubuntu 20
  • Ubuntu 22

5.1.0 will NOT be released for CentOS 7/EL7, Debian 10 or Ubuntu 18 given all are at or close to their end-of-life at time of release.

Note: Support for Oracle Linux 8 will be offered in the near future since its support life is extended, but is not available in production at the time of this release.

Ubuntu 22 Docker Images

5.1.0 has replaced the CentOS 7 docker images with Ubuntu 22-based images. In most cases this should be a drop-in replacement but see the Docker documentation for more details.

Installation Helper Script

To assist with installation independent of operating system, perfSONAR is making available a helper script that detects operating systems, sets-up package repositories and installs the selected bundle. Examples that will install the latest production release are shown below:

# perfSONAR Toolkit
curl -s | sh -s - toolkit

# perfSONAR Testpoint
curl -s | sh -s - testpoint

# perfSONAR Archive
curl -s | sh -s - archive

For further help:

curl -s | sh -s - --help

Bug Fixes and Other Changes


  • Upgraded to OpenSearch 2.13.0. Note that for the Debian package you must now set the OPENSEARCH_INITIAL_ADMIN_PASSWORD environment variable on first install of the core, toolkit and archive bundles. Example: env OPENSEARCH_INITIAL_ADMIN_PASSWORD=perfSONAR123! apt install perfsonar-archive. This is a requirement of the official Opensearch Debian package and not something introduced by the perfSONAR team. The password set will be overwritten with a random password by the perfSONAR installation process, so you can copy/paste the command shown safely. This does not affect existing installations that are updating. Alternatively you can use the Installation Helper Script detailed earlier in these release notes.
  • Fixed issue with default permissions where SQL queries were not allowed
  • Multiple fixes to the initial Opensearch configuration process.
  • Added psarchive ism_reset command to reinstall default Index State Management policies that determine whne perfSONAR data gets deleted. This is a good script to run if you suspect your install may have incorrect ISM policies either due to a manual change or a bug in previous perfSONAR version.
  • Added psarchive repair command that should fix most opensearch and logstash issues in terms of authentication or generally being unable to start the process.


  • Removed references to for better IPv6-only support


  • This component is now end-of-life and replaced by Grafana


  • Updated logstash to 8.10.4
  • Add “prometheus-pipeline” for collecting host and application metrics
  • Fixed issue with null reverse hostname lookups

LS Cache Daemon

  • This component is now end-of-life

LS Registration Daemon

  • Updated systemd unit file to support automatic restarts and simplify process forking
  • Various SELinux updates


  • This component is now end-of-life and replaced by Grafana


  • This component is now end-of-life as its primary purpose was in support for MaDDash


  • Updated systemd unit file to enable auto-restarts
  • Various SELinux updates
  • Removed references to for better IPv6-only support
  • Fixed issue with TWAMP default socket location

Perl Shared Libraries

  • Removed references to for better IPv6-only support
  • Removed unused perl dependencies


  • Fixed issue where slip could reduce to negative
  • Fixed issue when some versions of ping would not accept a floating point deadline and improved error reporting
  • Fixed issue with RTT where deadline was not incorporated into expected duration
  • Add ip-version option to MTU test and tool
  • Added source-bind and dest-bind options to throughput test
  • Added –full switch to “task” and “watch” command for more diagnostics
  • Add mss support to iperf2
  • Add last scheduled/finished to “troubleshoot” output
  • Replaced references to JSON template with Jinja2
  • Better handling of broken pipes during archiving
  • Added protocol parameter to latency and latencybg
  • Added port option to simplestreamer
  • Multiple improvement around tool selection and error reporting
  • Multiple fixes for handling iperf2 output
  • Removed references to for better IPv6-only support
  • Better handle output when traceroute reports no hops
  • Removed pscheduler-archiver-esmond from default install of perfSONAR. It can still be installed by running a dnf or apt install of the pscheduler-archiver-esmond package.


  • This component is now end-of-life and replaced by Grafana


  • Removed ServiceWatcher since auto-restarts now handled by systemd and other cleanup handled by individual packages.
  • Numerous SELinux fixes to cleanup denied actions
  • Removed references to for better IPv6-only support
  • Added symlink to pscheduler logs from /var/log/perfsonar on Debian to match existing RPM behavior
  • Fixed issue where configure_nic_parameters could not find interface list on EL9

Raw changes