Development And Test Results for a
Vision-Based Approach to AVCS 1
M. Juberts, K. Murphy, M. Nashman, H. Scheiderman, H. Scott, S. Szabo
Robot Systems Division
National Institute of Standards and Technology
Department of Commerce
Gaithersburg, MD 20899
Tel: (301) 975-3422, Fax (301) 990-9688
ABSTRACT
To improve ground transportation, the United States began a program called the Intelligent Vehicle /
Highway Systems (IVHS), sometimes called Smart Cars / Smart Roads. One of five IVHS
components, the Advanced Vehicle Control Systems (AVCS), envisions autonomous vehicles that
drive themselves, following their lane and avoiding other vehicles, all the while receiving routing
and road information from an automatic traffic surveillance and planning system. Potentially,
computer vision can fulfill many of the sensory requirements of mobility and traffic monitoring.
This paper presents the vision-based technology for AVCS developed at the National Institute of
Standards and Technology (NIST). A testbed vehicle has demonstrated autonomous road following
on a test course and on a highway at speeds of up to 90 km/hour under various weather and outdoor
light conditions. This paper describes the latest highway tests, the algorithm development for
autonomous vision-based car following, and the traffic monitoring system.
1. INTRODUCTION
The Intelligent Vehicle/ Highway Systems (IVHS) program is a major initiative of government,
industry, and academia to apply advanced technology to the operation of the Nation’s surface
transportation systems in order to improve mobility, transportation productivity, enhance safety,
maximize the use of existing transportation facilities, conserve energy resources, and reduce
adverse environmental effects. The IVHS program consists of the following five broad, interrelated
components: 1. Advanced Traffic Management Systems (ATMS); 2. Advanced Traveler
Information Systems (ATIS); 3. Commercial Vehicle Operations (CVO); 4. Advanced Public
Transportation Systems (APTS); 5. Advanced Vehicle Control Systems (AVCS).
The topic of this paper falls within the fifth IVHS component, Advanced Vehicle Control Systems.
AVCS employ advanced sensor and control technologies to assist the driver in responding to the
immediate environment on the roadway. AVCS will develop in an evolutionary manner, seeking
first to enhance the driver’s perceptions of his or her immediate environment. The long term AVCS
will provide fully automated vehicle/highway systems replacing the driver functions. Full scale
deployment of AVCS systems offer dramatic benefits, such as vastly increased highway capacity
through the use of tightly packed automated vehicles traveling at freeway speeds, and increased
safety by reducing the potential for driver error [1].
The need for autonomous vehicle lateral control (steering) and longitudinal control (braking and
throttle) on future automated highways is described in the AVCS program plans [2]. It is
envisioned that future automated highways may contain roadside and aerial traffic surveillance
systems and advanced traffic management systems with the ability to track and even control
individual or groups of vehicles in order to synchronize and coordinate their motion for various
automated driving scenarios.
The California PATH program has already demonstrated platooning of vehicles using microwave
range detectors mounted on the front of the test vehicles [3,4]. In addition, in order to improve
1
Presented at the 26th ISATA meeting on”Advanced Transport Telematics / Intelligent Vehicle
Highway Systems”, Aachen, Germany. September 1993
, response times needed for close proximity driving, each vehicle measures its acceleration and
transmits the measurements to the vehicles directly behind it. This approach could be combined
with their approach for autonomous vehicle lateral control using buried magnets in the road to
achieve fully automated platooning.
The use of vision-based perception techniques for autonomous driving are being investigated in
many programs in the United States as well as in other countries. A recent survey of these
techniques is presented in [5]. Use of machine vision as a primary sensor has promise in that the
infrastructure impact is minimized relative to other approaches. If such a low infrastructure impact
approach could be implemented safely and affordably, then deployment, operations, and
maintenance of AHS systems would be greatly simplified, increasing the viability of the overall
concept, and facilitating rapid expansion of automated travel to the entire road network in future
generation systems [6].
Over the past five years, NIST has been conducting cooperative research and development in
vision-based vehicle mobility control. This paper describes this research and the viability of using
vision to fill the long-term needs of AVCS. The initial work was in the development, integration,
and demonstration of vision processing and control algorithms needed for autonomous road
following (vehicle lateral control using lane markings), autonomous car following (vehicle lateral
and longitudinal control for platooning) and the assembly and demonstration of an automatic vision-
based surveillance system. Such a system could eventually be used as part of a higher level remote
traffic control system for providing real-time traffic information, and coordinating lateral and
longitudinal control of vehicles or groups of vehicles on instrumented highways.
2. CONTROL APPROACH
Testbed and support vehicles
The project utilizes a NIST testbed vehicle, a HMMWV, which was developed by NIST in
cooperation with the U.S. Army Research Lab as part of the DOD’s Unmanned Ground Vehicles
program [7,8]. Figure 1 is a photograph of the testbed vehicle. The vehicle controller drives
motors for steering, brake, throttle, transmission, transfer case, and park brake and monitors the
dashboard sensors such as speed, RPM, and temperature. Prior to this project, the instrumented
vehicle, which has a NIST Real-time Control System (RCS), had two modes of operation:
Teleoperation mode - where an operator remotely controls the vehicle over an RF communications
network; and Autonomous Mode called “retro-traverse” [9] - where the control system uses an
inertial navigation system (INS) to steer the vehicle along a pre-recorded path. NIST replaced the
human teleoperator control functions with intelligent vision processing elements in order to achieve
autonomous, visually guided road following. The Department of Defense (DOD) made the vehicle
available to NIST and DOT for this project.
Figure 1. Testbed vehicle followed by support van
2
