Performance Evaluation of Secure Industrial
Control System Design: A Railway Control System
Case Study
Xenofon Koutsoukos, Himanshu Neema,
Goncalo Martins, Sajal Bhatia, Janos Sztipanovits
Institute for Software Integrated Systems
Vanderbilt University Nashville, TN, USAKeith Stouffer, Chee Yee Tang, Richard Candell
National Institute of Standards and Technology
Gaithersburg, MD, USA
Abstract —Industrial control systems (ICS) are composed of
sensors, actuators, control processing units, and communication
devices all interconnected to provide monitoring and control
capabilities. Due to the integral role of the networking in-
frastructure, such systems are vulnerable to cyber attacks. In-
depth consideration of security and resilience and their effects
to system performance are very important. This paper focuses
on railway control systems (RCS), an important and potentially
vulnerable class of ICS, and presents a simulation integration
platform that enables (1) Modeling and simulation including
realistic models of cyber and physical components and their
interactions, as well as operational scenarios that can be used for
evaluations of cybersecurity risks and mitigation measures and
(2) Evaluation of performance impact and security assessment of
mitigation mechanisms focusing on authentication mechanisms
and firewalls. The approach is demonstrated using simulation
results from a realistic RCS case study.
I. I NTRODUCTION
The exponential growth of information and communica-
tion technologies over the last decade has given rise to
their expansion in real-world computing applications involving
physical processes. This expansion has led to the emergence
of closed-loop systems involving strong integration and coor-
dination of physical and cyber components, often referred to
as cyber-physical systems (CPS). These systems are rapidly
finding their way into various sectors of the economy, such
as industrial control systems, transportation, healthcare, and
critical infrastructure. Increasing dependence on CPS renders
them critical, and in-turn demands them to be secure, robust,
reliable, and trustworthy, but it also makes them very attractive
targets for cyber attacks.
Because of these disruptive changes, physical systems can
now be attacked through cyberspace and cyberspace can be at-
The work at Vanderbilt is supported by NIST (70NANB13H169). No
approval or endorsement of any commercial product by the National Institute
of Standards and Technology is intended or implied. Certain commercial
equipment, instruments, or materials are identified in this paper in order to
facilitate understanding. Such identification does not imply recommendation
or endorsement by the National Institute of Standards and Technology, nor
does it imply that the materials or equipment identified are necessarily the
best available for the purpose. This publication was prepared by United States
Government employees as part of their official duties and is, therefore, a work
of the U.S. Government and not subject to copyright.tacked through physical means. While CPS research addresses
the tight interaction between the physical and cyber parts
from the performance point of view, in-depth consideration
of security and resilience in an integrated manner is still in
early stages. The complex nature of CPS, mainly due to tight
coupling of cyber and physical phenomena, makes securing
such systems a challenging problem. A multi-vector attack
exploiting a combined set of vulnerabilities from individual
components, none of which might pose a serious threat to
the stand-alone component, can have damaging effects in the
overall system.
Industrial control systems (ICS) are a specific class of
CPS in the juncture of control systems and cyber systems.
ICS are composed of sensors, actuators, control processing
units, and communication devices all interconnected to provide
monitoring and control capabilities. In contrast to traditional
computing systems, ICS must perform their critical functions
without interruption. This paper focuses on railway control
systems (RCS), an important and potentially vulnerable class
of ICS and CPS. Cybersecurity is vital for ensuring that these
systems can provide their critical services without disruptions
that may result in catastrophic damages.
The objectives of this work are to analyze the cybersecurity
risks of RCS, propose mitigation mechanisms, and evaluate
their effectiveness as well as their performance impact on
system operations. We propose to achieve these goals by
developing a simulation integration platform that enables (1)
Modeling and simulation of RCS including realistic models of
cyber and physical components and their interactions, as well
as operational scenarios that can be used for evaluations of
cybersecurity risks and mitigation measures and (2) Evaluation
of performance impact and security assessment of mitigation
mechanisms. The main innovation of our approach is that
research processes and results are documented as executable
software models, simulations, and generated data that support
cybersecurity analysis and design in a quantifiable manner. It
should be noted that RCS are treated as any other network
critical infrastructure and hence the proposed approach can be
directly applied to other classes of ICS.
The paper presents a simulation-based integration platform for RCS in order to perform experiments and acquire mea-
surements to characterize performance and impact of secure
control system design. The developed simulation integration
platform uses a modular approach to integrate two open-
source simulators: OMNeT++ [1] and Train Director [2].
The integration is based on a software tool infrastructure
developed at the Institute for Software Integrated Systems
at Vanderbilt University called Command and Control Wind
Tunnel (C2WT) [3] which enables large scale heterogeneous
simulations.
The platform enables the evaluation of the performance
impact of implementing security solutions, complying with the
ICS cybersecurity standards. The communication model used
is based on the Advanced Train Control System (ATCS) [4]
and the implemented security solutions comply with ICS
cybersecurity guidelines [5]. In addition, the platform allows
the evaluation of the performance of these applied security
solutions against cyber-attacks. Specifically, this paper focuses
on the evaluation of authentication mechanisms and firewalls.
Authentication mechanisms in RCS incur both computational
and communication overhead. Although the computational
overhead is typically very small in modern microprocessor
architectures, the communication overhead can result in time
delays that need to be taken into consideration in the system
design. Firewalls can serve a central role in securing RCS
against a variety of external attacks and depending on the
implementation, they can incur negligible performance impact.
The rest of the paper is organized as follows. Section
2 presents the simulation integration platform, Section 3
describes RCS focusing on the ATCS standard, Section 4
describes the simulation of RCS, Section 5 presents the
evaluation results for the performance impact of authentication
mechanisms and firewalls, and Section 6 concludes the paper.
II. C OMMAND AND CONTROL WINDTUNNEL
A common problem with developing large-scale hetero-
geneous simulations is the complexity and effort required
to integrate domain-specific simulation tools. Development
challenges include how to integrate multiple simulation en-
gines with varying semantics and how to integrate simulation
models and manage the complex interactions between them.
The High Level Architecture (HLA) provides the structural
basis for simulation interoperability, distributed simulation,
and is the standard technical architecture for heterogeneous
simulations [6]. HLA provides application programming in-
terfaces (APIs) that have helped to reduce the complexity
of integrating multiple different simulation engines, but many
challenges remain in such environments. As an example, HLA
does not specify any tools to design or deploy a federation.
It primarily standardizes runtime support for various tasks,
such as coordinated time evolution, message passing, and
shared object management. As a result, the HLA framework
requires a significant amount of tedious and error-prone hand
development integration code [3].
C2WT was developed to address the challenges present in
the HLA framework [3]. C2WTis a graphical environment fordesigning and deploying heterogeneous simulation federations.
Its primary contribution is to facilitate the rapid development
of integration models, and to utilize these models throughout
the lifecycle of the simulated environment. An integration
model defines all the interactions between federated models
and captures other design intent, such as simulation engine-
specific parameters and deployment information. SIM uses the
Generic Modeling Environment [7] and a custom Domain-
Specific Modeling Language (DSML) for the definition of
integration models. This language facilitates the easy capture
of all of the design details for the simulation environment.
C2WT integration models follow the conceptual architecture
depicted in Figure 1. A simulation environment is composed
of multiple ‘federates’, each of which includes a simulation
model, the engine upon which it executes, and some amount of
specialized glue code to integrate the engine with the simula-
tion bus. Both the engine configuration and the integration (or
‘glue’) code needed for each federate is highly dependent upon
the role the federate plays in the environment, as well as the
type of simulation engine being utilized. The main differences
from HLA are the automatic generation of engine configura-
tions, glue code to integrate the engine with the simulation
bus, as well as scripts that allow the automation simulation
execution and data collection. This integration enables a robust
environment for users to rapidly define complex heterogeneous
simulations.
Fig. 1. C2WT Architecture
III. R AILWAY CONTROL SYSTEMS
The C2WT integration platform is used for simulation of
RCS. The railroad network control infrastructure consists of
the following main components:
Dispatch Center. The dispatch center (also known as Cen-
tral Control Center) is a centralized control center for train
management. It usually has a high bandwidth connection with
the carrier network (e.g., MPLS/IP), but it could have any IP
services.
Wayside Equipment. This is equipment located at the side of
the track, such as signal controllers, switch circuit controllers,
