perfSONAR

Version 4.1b1

Release Date

June 29, 2018

Features/Changes

pSConfig

  • pSConfig is a new component that replaces the old MeshConfig component. It is a template framework for defining the tasks to be run by hosts. The templates can be used to not only configure the tasks on each host through pScheduler but also to display their results through MaDDash. See the introduction to pSConfig here for more information.
  • Some major benefits of pSConfig include:
    • Increased Extensibility - pSConfig can take full advantage of the extensibility provided by pScheduler. It supports any test, archiver or context plug-in installed on your pScheduler server. Best of all, as more plug-ins get added, no software update is required for pSConfig!
    • Consolidated Configuration Format - pSConfig defines all the tasks to be run in a JSON format it refers to as a template. There is no conversion between various file formats required as there was in MeshConfig. No longer do you need to edit an Apache Config-like format and convert it to JSON nor edit yet another format referred to a meshconfig-agent-tasks.conf for you local agent. Now there is only one format required no matter where the template lives.
    • Enhanced Control - pSConfig gives increased control over what each agent does. This includes not only familiar operations like defining archives locally, but also allows for agent-defined transforms of any aspect of the templates after they are downloaded. See the documentation here for more information
    • Better Diagnostics - pSConfig formats all logs of its agents in a parsable format. It also provides command-line utilities to help extract useful information from the logs. See the documentation for the pscheduler agent and maddash agent for examples.
  • The pSConfig Web Administrator (PWA) is a separate component that provides a web interface for creating pSConfig templates. It is a revision of the former MeshConfig Administrator component that entered beta with version 4.0. As such, it can also generate the old MeshConfig format for tests using options supported by both formats. For more information see the documentation here.
  • pSConfig uses a completely new JSON format to define templates. That being said, it is capable of reading files in the old MeshConfig format. This should give some flexibility in terms of when you migrate. It is recommended you migrate as soon as all your agents are updated to 4.1, since support for the old format will be dropped 6 months after the next major release. For more information on migration see the document here.

TWAMP

  • The Two-Way Active Management Protocol (TWAMP), described in RFC 5357, is an extension of the One-Way Active Management Protocol (OWAMP) that supplies two-way or round-trip measurements instead of unidirectional capabilities. TWAMP measurements are helpful because round-trip delays do not require host clock synchronization but can still provide additional accuracy compared to tools such as ping. It has the additional advantage that many routers are capable of running a vendor's implementation of a TWAMP server.
  • A pscheduler plug-in is available named pscheduler-tool-twping for running tests of type latency or rtt. Periodic measurements can be setup via the pScheduler CLI, pSConfig Web Administrator or through pSConfig templates setup by hand by specifying tool twping

Docker Support

  • Docker containers will be supported in 4.1 final release for tools and testpoint bundles. See the documentation here for more details.

pScheduler Updates

  • Added support for task re-writing to make on-the-fly changes to tasks at ingest to supplement the limit system.
  • Added support for "single-ended" throughput tests that allow for testing to a server not running pScheduler.
  • Improved the pscheduler troubleshoot command to check the maximum transmission unit (MTU) as well as enhanced options for checking on IPv4 vs IPv6.
  • Added an http test and associated tools for measuring various HTTP performance metrics

BWCTL Deprecation

  • BWCTL is no longer included in new installations of perfSONAR but will be left untouched on existing installations. For a complete discussion of BWCTL support including timelines and options see the upgrade notes.

CentOS 6 Deprecation

  • CentOS 6 is not supported for version 4.1. See the documentation here for a full discussion of CentOS 6 support and perfSONAR.

Debian 7 Deprecation

  • Debian 7 is not supported for version 4.1. See the documentation here for a full discussion of Debian 7 support and perfSONAR.

Important Upgrade Notes

  • See the documentation here for a full list of important notes about the upgrade process.

Detailed Bug Fixes and Changes

Bundles

  • Removed BWCTL dependency from pscheduler-tools bundle
  • Created package perfsonar-bwctl-compat as an option for installing BWCTL packages and associated pScheduler plug-ins.
  • Replaced dependencies on packages perfsonar-meshconfig-agent and perfsonar-meshconfig-guiagent with perfsonar-psconfig-pscheduler and perfsonar-psconfig-maddash respectively.
  • Added dependency on perfsonar-psconfig-publisher to perfsonar-centralmanagement bundle.
  • perfsonar-centralmanagement now sets-up the esmond cleaner cron job on install

BWCTL

  • This component has been removed from all new installations. See the upgrade notes.

Esmond

  • No changes this update

Graphs

  • Fixed duplication of paths in esmond URLs

I2Util

  • Stopped using deprecated sys_errlist

LS Cache Daemon

  • No changes this update

LS Registration Daemon

  • Removed support for registering deprecated or unused services: BWCTL, BWCTL MP, OWAMP MP, NDT, NPAD, Mesh, Phoebus and REDDnet
  • Added support for pscheduler-tests field on interface records. This allows advertisement of types of tests supported by an individual interface
  • Disabled test auto-discovery in esmond registration since these records generate substantial LS traffic but are unused.

MaDDash

  • No changes this update

MeshConfig

  • This component has been removed and replaced by pSConfig

Nagios

  • Renamed check_delay to check_rtt
  • Fix uninitialized string error caused by version not being set

OPPD

  • This component is no longer included with new installs. It's functionality is replaced by pScheduler.

OWAMP

  • Extended source code to support TWAMP
  • Fixed issues where execute permissions not set on startup script

Perl Shared Libraries

  • Added new package libperfsonar-psconfig-perl that provides libraries in support of pSConfig clients
  • Fixed incorrect logging statements in various libraries that were calling warning instead of warn
  • Removed unused references to MySQL in code
  • Remove BWCTL references from code
  • Consolidated toolkit_version and toolkit_rpm_version in Toolkit data service to just toolkit_version
  • Added support for singled-ended throughput tests in libraries that support toolkit GUI

pScheduler

  • Improved diagnostics when restoring configuration and data.
  • Attempt to put old data back in place if database load fails
  • Added MTU diagnostics to the troubleshooter.
  • Troubleshoot command tests archiving
  • Dropped required placeholder when importing tasks using the CLI and not changing parameters.
  • Made debugging of REST API server more robust.
  • Added initial support for command completion when using the Bourne Again SHell (BASH).
  • Added IPv6 support to the simplestream test and simplestreamer tool.
  • Added IPv6 support to the troubleshooter.
  • Improved handling of decisions about how to contact the local host.
  • Improved propagation of errors into CLI diagnostics.
  • Made clock sync information available in the CLI when a survey shows a pair to have large differences.
  • Added pscheduler-tool-twping
  • Added pscheduler-test-http
  • Added pscheduler-tool-psurl
  • Added support for singled-ended parameter to throughput tests
  • Moved BWCTL tools to pscheduler-bundle-extras package
  • Cleaned-up owner of files in /var/run
  • Added protocol field to pscheduler-test-rtt in support of TWAMP
  • Updated WSGI settings to use 10 processes and restart every 2.5 million requests
  • Improved diagnostics when no tools in common

pSConfig

  • Completely new component to replace MeshConfig. See documentation.

pSConfig Web Administrator

  • Renamed from MeshConfig Administrator to pSConfig Web Administrator
  • Added support for pSConfig templates
  • Added a way to specify central MA(s) for a Config
  • Added a way to archive to end hosts
  • Changed authentication model - registration disabled by default. It can be enabled or users can be created via CLI
  • Added support for twping tool (pSConfig-only)
  • Fixed switch breaking hostgroup by switching static to dynamic host groups and then back
  • Removed dependency of traceroute tests on a traceroute service in LS
  • Fixed issues where certain host details were not saved properly
  • Added ability to override the global MA in favor of one or more local MAs on a host
  • Added ability to add addresses to host that are not in LS
  • Added a way to override the MA URL for hosts
  • Fixed the mesconfig importer

pSTracerouteViewer

  • No changes this update

Toolkit

  • Updated pscheduler limits to use rewriter to enforce bandwidth limits
  • Added TWAMP rules to firewall
  • Removed references to BWCTL
  • Moved logrotate rules to include file to /etc/logrotate.d/perfsonar-toolkit
  • Updates toolkit web interface to save configuration in pSConfig format
  • Added psconfig-pscheduler-agent to toolikit services listing and removed meshconfig-agent
  • Removed meshconfig-agent from configdaemon
  • Removed no longer used psb_to_esmond script
  • Updated JSON service to to replace get_meshes action with get_templates and return pSConfig template URLs
  • Updated configure_owamp_testports to produce pSConfig transform file
  • Removed no longer relevant MySQL and web100 references
  • Removed OPPD firewall rules
  • Updated configure_firewall to check for existence of file it is editing
  • Cleaned-up enable_auto_updates script to prevent unnecessary stderr output during RPM install
  • Fixed issue where login message displayed incorrect version
  • Added script to find tests in archive that are running BWCTL
  • Added psconfig files to backup and restore scripts
  • Added support for single-ended throughput tests in test configuration web interface

Raw changes

Updated components: