Released November 2, 2020

Features

Python 3 Support

  • This release focuses on upgrading to Python 3. This change is necessary since all versions of Python 2 are considered end-of-life. This brings with it many changes to the perfSONAR code and package depedencies. In particular the pScheduler and esmond components are the most affected since they are the largest Python components. To learn more about the Python 2 end-of-life here.

PWA support for More Archiver Types

  • The pSConfig Web Administrator (PWA) has been updated to support more archivers beyond Esmond. It now includes suppport for setting-up a RabbitMQ archiver as well as an advanced option to define a an archiver using a raw JSON definition which should allow use of any pScheduler-supported archiver plug-in.

S3 Benchmark Test and Tool

  • Added a s3throughput test and a corresponding tool for measuring throughput of S3-compatible object storage services. NOTE: This tool is not currently available for Debian based distributions running arm64 or ppc64el architectures.

Ethr throughput tool plugin

  • Added throughput testing with Ethr. Note that because Ethr does not support bandwidth constraints, this plugin does not either and will decline to participate in any test with a bandwidth parameter.

pScheduler Batch Processor

  • Added a ‘batch’ command that allows multiple pScheduler jobs to be done serially or in parallel and returns the results as a single unit.

Bug Fixes and Changes

Bundles

  • Revised packages to use new PostgreSQL update scripts.

Esmond

  • Updated to Python 3
  • Updated to PostgreSQL 10
  • Update Django and Django Rest Framework versions

Graphs

  • Made selected time range consistent with dropdown

Perl Shared Libraries

  • Improved checks for running pScheduler proccesses

pScheduler

  • Updated all components to support Python3
  • Updated to PostgreSQL 10
  • Added Ethr throughput tool
  • Added S3 test and tool
  • Added batch processor support
  • Added support for bandwidth property to nuttcp tool
  • Added support for single-ended tests to nuttcp
  • Improved the archiver’s ability to handle plugin failures.
  • Improved the RabbitMQ archiver’s robustness.
  • Fixed buggy implementation of priorities and preemption to make the feature work as originally intended.
  • Added a pscheduler metrics command to export stats about pScheduler in JSON or Prometheus format
  • Fixed scheduling issue when slip pushed over scheduling horizon
  • Corrected a bug in the iperf2 plugin that was causing timeouts to be calculated incorrectly.
  • Added a require field to limit configuration’s classifiers that will classify based on none, one, any or all of the identifiers listed.
  • Improved ability to handle tasks with extremely-long durations.
  • Made IPv6 flow labeling available to trace tests.
  • Added preempted runs to the schedule plot.
  • Improved passage of debug state to forked processes for better logging.
  • Added a label parameter to archive specifications for improved diagnostic output.
  • Internal improvements that should make a marginal improvement in performance on busy or resource-constrained systems.

pSConfig

  • Updated MaDDash agent to support generating DNS and HTTP graphs
  • Added option to “psconfig pscheduler-stats” command to export stats in JSON or Prometheus format
  • Fixed issue where DNS lookup failure of local host would result in agent tryng to schedule tests for other hosts

pSConfig Web Administrator (PWA)

  • Added support for rabbitmq archivers
  • Added support for custom archivers using raw JSON
  • Fixed issue where bandwidth parameter was called tcp-bandwidth for throughput tests
  • Added a default slip for tests that run on regular interval
  • Fixed issue with specifying probe type for trace tests
  • Added ability to set schema version for test spec and archiver sections
  • Added support for importing JSON via copy/paste
  • Updated sca-auth to add a way to set roles in a given scope
  • Updated sca-auth to add ability to set scopes for single context via CLI
  • Updated pscheduler troubleshoot to check more information about host

Toolkit

  • Updated various setup scripts to Python 3
  • Updated packaging to support PostgreSQL 10 updates
  • Updated NTP server list

Raw changes

Updated components:

Tags: