perfSONAR

Security considerations

Managing Access

perfSONAR nodes are meant to be used, both by local users and the public at large, to perform a variety of network tests.  The open access policy is at odds with ways to mitigate the risk of exposing functionality to those that would cause harm.  The following is a possible approach for managing access to the host:

  • SSHD can be turned off completely if remote access to the machine via the terminal is not need (e.g. in cases where console access is available either directly, or indirectly)
  • If SSHD is turned on, consider using a jump host setup wherein access to the perfSONAR node can only be accomplished through a single (or set) of trusted hosts.  This type of restriction can be implemented in IPTables. 
  • SSH throttling can be installed via IPTables to limit the number of times (in short succession) that a host can attempt to log in.  This helps to thwart brute force attacks from a specific IP:
    • # Throttling of SSH
    • -A INPUT -p tcp --dport 22 --syn -m limit --limit 1/m --limit-burst 3 -j ACCEPT
    • -A INPUT -p tcp --dport 22 --syn -j DROP
  • HTTP access is a desirable feature to keep open, as it allows others to see the measurements of a host.  If protecting access to port 80 is required, consider using IPTables to restrict access to certain network subnets.
  • HTTPS access is normally only needed for administrators within a domain, consider using IPTables to restrict access to certain network subnets.
  • Measurement daemons like BWCTL and OWAMP work best when they are open to all IP ranges.  It is not recommended that IPTables be used to limit access to the tools.  Each daemon has a limits file that can be used to set test policy for users in different network subnets.  See http://docs.perfsonar.net/config_bwctl_limits.html for more information on ways to implement policy limits for BWCTL.  
  • Some measurement archives feature a RESTful API that allows users or automated tools to download data.  These are recommended to stay open and not be restricted by IPTables. 

Logging

Configure rsyslog to send system logs to an aggregator or analysis tool.  Consult syslog documentation for ways this can be done, and use the procedure that works best for your site. 

Host IDS

perfSONAR nodes come with the fail2ban host IDS.  This tool will analyze many of the logs on the host, and insert IPTables rules to prevent malicious users from repeatedly attacking system resources.  For example, a host that performs an SSH scan (e.g. attempting to log in with many user names over a short period of time) would be blocked from accessing the host. 

Users are encouraged to look into other host IDS systems (OSSEC, denyhosts, tripwire or others) as possible additions for a complete security solution.  Use of a site wide IDS, such as Bro, is also recommended to capture threats at the border. 

Passwords & Users

Normal precautions should be taken to protect the root, and web administrator, passwords as they can be used to make changes to the system. For example, safe password practices would recommend a password that contains a mixture of letters of different case, numbers, symbols, and a length greater than 8.  It is also not recommend to re-use passwords on multiple machines, in the event of a system breach.

It is not standard for users to have accounts on perfSONAR nodes, unless they anticipate doing a lot of debugging and need access to console tools.  When granting user accounts, be sure to evaluate if they will or will not need administrative control (e.g. use of the sudo tool). 

It is recommended that users not store private keys on the perfSONAR node, particularly if they use the same private key to log into multiple resources.  Suggested alternatives:

  • Only store public keys in authorized_keys files
  • Create specific public/private key pairs for accessing perfSONAR nodes (and do not store the private key on the host)
  • Implement two factor authentication

Infection Detection

Rkhunter and chkrootkit are tools designed to search a computer for various types of malware infections.  Each has has a set of heuristics and signatures that are used to detect possible trouble on your machine.  Chkrootkit can be run via cron, using a command like this:

0 3 * * * (/usr/bin/chkrootkit 2>&1 | mail -s "chkrootkit output" root@somesite.somedomain)

Rkhunter can be configured to run via the init system on a server. 

Regular Maintenance

Stay up to date on packages, see http://docs.perfsonar.net/manage_update.html for more information, and join the user or announcement list to receive notices. 

Using IPTables

The perfSONAR development effort has evaluated the use of IPTables (a Linux solution for restricting inbound and outbound communication) and has found that performance at 10Gbps levels is possible in some cases.  IPv4 traffic did not see a large delay for TCP or UDP port filtering.  IPv6 TCP traffic was also unaffected in any serious way by the use of IP6Tables; UDP traffic on the other hand can be dropped as the requested rates of transfer increase.  The development team recommends evaluating the use of IPTables with the software and hardware combination you intend to use before deployment to ensure there are no "self inflicted" wounds caused by the use of a firewall. More information can be found here: https://fasterdata.es.net/performance-testing/iptables-performance/

Using perfSONAR with Firewalls

Firewalls are used to impart security into a network by controlling incoming and outgoing traffic based on header information. E.g. Firewall rules can limit traffic to specific subnets, IPs, ports, and protocols. Firewalls inspect packets, and thus impart a delay in the delivery of traffic end to end.  This is true of both host based firewalls (e.g. IPTables) as well as appliances that sit within a network.  This delay can introduce jitter into a flow (e.g. disruption of the variation of arrival times of packets), or potentially drop packets completely if memory is exhausted due to the amount of processing for a given set of flows. For these reasons it is recommended that the perfSONAR device be allowed to test outside of the firewall environment.

If the perfSONAR device must be behind a firewall, a set of ports must be opened for incoming and outgoing traffic. Even with a set of ports identified for traffic, note that outbound filtering (a common technique used in modern firewalls) is not advised unless the peers you are testing with are using the exact same set of port ranges. If this is not the case, failures may result when the non-local end of a test chooses a port that is not in the local end's expected range. Note that the following port recommendations are only enforced on the local end, and the tools make no attempt to negotiate via protocol with the remote end of a test:

Service Protocol Local Ports Direction
Lookup Service (Server to Server) TCP 61617 Incoming/Outgoing
Lookup Service TCP 8090 Incoming/Outgoing
8096
BWCTL TCP 4823 Incoming/Outgoing
TCP/UDP Edit /etc/bwctld/bwctld.conf or /etc/bwctl-server/bwctl-server.conf, set peer_port to a range of values, open the TCP port for these values. It is recommended that 6001-6200 be used for this variable.
TCP/UDP Edit /etc/bwctld/bwctld.conf or /etc/bwctl-server/bwctl-server.conf and set 'test_port' to a specific range, and open the TCP/UDP ports for those ranges. It is recommended that 5001-5900 be used.
OWAMP TCP 861 Incoming/Outgoing
UDP Edit /etc/owampd/owampd.conf or /etc/owamp-server/owamp-server.conf, set testports to a range of values, open the tcp port for these values. It is recommended that 8760-9960 be used for this variable. To configure OWAMP clients, edit the /opt/perfsonar_ps/regular_testing/etc/regular_testing.conf or /etc/perfsonar/regulartesting.conf file to add the following stanza:
<default_parameters>
    type  powstream
    receive_port_range    8760-9960
</default_parameters>
NDT TCP 3001-3003 Incoming/Outgoing
7123 Incoming
NPAD TCP 8000 Incoming
8001-8020 Incoming/Outgoing
Apache/esmond TCP 80 Incoming
443
SSH (Optional) TCP 22 Incoming
NTP UDP 123 Outgoing
DNS UDP 53 Outgoing
Ping ICMP - Incoming/Outgoing
Traceroute/Ping UDP 33434-33634 Incoming
OPPD TCP 8090 Incoming/Outgoing

In versions of the perfSONAR Toolkit prior to 3.4, the following additional port ranges were needed:

Service Protocol Local Ports Direction
Echo TCP 7 Incoming
SNMP MA TCP 8065 Incoming
9990
Traceroute MA TCP (control), TCP/UDP/ICMP (measurement) 8086 Incoming/Outgoing
8087
PingER TCP (control), ICMP (measurement) 8075 incoming (control), outgoing (measurement)
perfSONAR-BUOY TCP 8085 Incoming
8569 Incoming/Outgoing
8570
Lookup Service TCP 8095 Incoming/Outgoing
9995

Even with this list of available ports, there is no guarantee that certain measurements (e.g. BWCTL and OWAMP) will succeed 100% of the time, namely because the choice of ports (or the decision to limit them to a certain range) is a site by site decision. Most tools pick a specific port range, and that is the only range that will work. These test tools wanted to provide the flexibility of allowing local sites to pick the local port ranges they use. To get this flexibility for what is accepted for local port selection, the trade-off is that you can not limit the destination port you send to. However, you always have control of the local-port selection. So, inbound traffic can be limited based on the destination port, and outbound traffic can be limited based on the source port.

In general, if a site is blocking outgoing traffic based on destination port, problems will result in using the BWCTL and OWAMP tools specifically since there is no way to limit the port choices of the remote sites. Likewise, if a site blocks traffic in either direction based on source port a similar situation will result.

Outbound filtering is therefore not recommended, and a policy to allow all outgoing traffic is a better choice for the measurement infrastructure. As for inbound filtering, sites may open up those destination ports that are actually used as service ports, but all source ports have to be allowed, as they are chosen by the other side.

Additional information can be found here: http://fasterdata.es.net/performance-testing/perfsonar/ps-howto/perfsonar-firewall-requirements/

NTAC Performance Working Group Statement

The NTAC Performance Working Group has published a document related to deploying perfSONAR while still justifying cybersecurity policy.  This document can be found here:

NTAC Performance Working Group Firewall Recomendations for perfSONAR (178.5 KB)