The first thought most people have when it comes to industrial, or OT, cybersecurity relates to the controls that should be implemented to secure a system. Examples include two-factor authentication (2FA), tightened firewall rules, zone-and-conduit segregation, limited user access, patching, anti-virus, or any other system change to improve cyber posture. Many cyber experts, however, are suggesting that this is not enough. To truly address risk, we must think instead in terms of resiliency.
In the OT space, cyber resilience refers to the ability to maintain production despite an adverse cyber event on any related system. Cyber controls, such as those mentioned above, are a key part of this. There are other equally important subjects to address as well, including external system reliance, back-up planning, and disaster recovery planning.
In a modern production environment, a system or network under consideration almost certainly relies on other systems and networks. These connections may be in place to communicate information vital for OT efficiency such as production orders, recipes, ingredient availability and equipment status. In some cases, the reliance is simply on availability – for example, it may be necessary that information, such as environmental records, are continually communicated to databases external to the control system. The key to OT resilience is determining other system dependencies and identifying steps to take in the event of a cyber event impacting those systems. In some cases, this may be as simple as disconnecting the OT networks and physically isolating them; in other cases, manual information transfers may be required.
While frequently ignored in cybersecurity projects, restoring a down system often requires access to a trustworthy backup. With the evolving nature of persistent threats, there is the added concern that backups have also been affected. The goal of maintaining an up-to-date backup library starts with making scheduled backups and storing a copy of any modified configuration into a library as part of the change control process. When doing this manually becomes onerous, consider incorporating automated backup software. Depending on your specific OT deployment there are several solutions in this space, many of which include the capability to detect unauthorized modifications and managing your controlled source.
Disaster recovery planning encompasses several different topics. First and foremost, it includes ensuring that you have access to backups of all software configurations if you need them (see above). A second aspect of disaster recovery addresses the hardware. Namely, it is vital for you to identify all of your OT hardware, identify criticality, obtain vendor’s contact information, and determine the appropriate spare parts inventory. In some cases, it may be determined that spare parts cannot be obtained to reduce risk exposure to an acceptable level. Upgrading the hardware may be the only option. Furthermore, the risk may still be too high, and the implementation of redundancy (such as hot-standby or other high-availability architecture) may be required. Finally, a software bill of material is required to match the configurations with the OT hardware. For PLCs and panel HMIs, a fairly simple approach of listing firmware and configuration name will suffice for many sites. For servers and other Windows-based machines, a detailed listing of all software versions and configuration is required. This includes more than just the SCADA itself, as wrong, missing, or outdated I/O servers and other software may still prevent the system from being restored to full functionality.
Finally, disaster recovery planning is incomplete without a documentation guide that can be used in an emergency. It is rare that system failures happen in the middle of a Tuesday morning with a fully-trained staff waiting for it. Instead, disaster tends to strike in the middle of the night, with restoration first attempted by sleep-deprived technicians and engineers. It is best to give them every possible chance to succeed.
Many sites find themselves paralyzed by where to start. If that is the case for you, reach out to ACE for help. We are available to talk about where you are, where you want to get to, and the options to make that journey. Whether we start with a roadmap exercise, creation of policies, or immediately implementing specific controls, we are here for you.