The recent rapid proliferation of transportation data resources has implemented a rich array of potential data resources for transportation agencies. The new data resources coupled with pending requirements associated with Section 1201 of SAFETEA-LU (23 CFR 940) are requiring transportation agencies nationwide to revisit their overall traffic and transportation data and information goals and requirements. The recent paradigm shift is necessitating the need for transportation agencies to conduct evaluations of existing data and information architectures, and determine future plans for data acquisition, data processing and information dissemination. New data governance disciplines will enable transportation agencies to prepare for the changing face of transportation data management, optimize the potential value of their data and information and avoid a patchwork of stove-piped data and information subsystems.
Establishing a Data and Information Framework
A successful transportation data and information framework provides for the detection, collection, aggregation and delivery of accurate, pertinent data to head-end processing and dissemination infrastructure. In order to optimize efficiencies and accuracies within the data and information network, the end user/operator should first assess the existing data and information management environment. The existing conditions survey will set a baseline for the development of the next-gen data and information framework, by establishing requirements associated with legacy infrastructure and existing user needs. Once the existing data and information survey is complete, an “existing framework” model can be generated and utilized as the foundation for the next generation framework.
A “data and information framework” identifies all functional limits, including network and system boundaries of all existing data and information management systems that support and interface with the defined comprehensive framework (and the technologies and applications that operate the framework). A detailed framework also accurately maps all data and information flows within and between transportation systems and devices, defines functional requirements associated with the framework as well as specifies technical requirements associated with the data and information environment, such as formats, protocols, polling rates, processing and storage. The framework should also identify future user needs and future system needs associated with the information framework, thus establishing a comprehensive mapping of all existing and future data and information requirements.
Considering Potential Data Resources
A formal data and information framework enables the end-user/operator to better understand existing system elements and future data needs in their entirety, and ultimately assess and understand functional and technical requirements required for their data and information services. This level of understanding is essential to the end user considering system improvements or more succinctly, looking to implement new data strategies. One of the first decisions to consider is whether or not an operator needs to buy, build and manage the data and information network in its entirety, procure private data resources to supplement all or a portion of the data needs, or develop a new framework that will satisfy the use of multiple data resources.
Traditional Means
Until recently, public agencies have traditionally designed, built and operated all data and information system components (hardware and software) required to feed their traffic management and transportation operations needs. Public data networks can be developed with hyper local understanding of benefits and constraints of specific detection and data collection strategies. Public data collection systems place complete control of data collection and data management in the public agency domain. Sharing public agency data with other internal and external agencies has proven to be easily implemented as well. Also, it is common for detection vendors to develop head-end software and system management software for the operations of detection devices, therefore alleviating the need for the public entity to develop any additional operations and management software to support a data and information network. However, public data systems require significant upfront capital investment and continuing funds for the operations and maintenance of the infrastructure, including detection devices, communications infrastructure and head-end hardware.
Private Sector Data
The last decade has seen the emergence of “Data as a Service” (DAAS) models where traffic and transportation data is provided by the private sector. The rapid proliferation of GPS-based probe data, Bluetooth detection data and peer-to-peer crowd sourced data resources are rapidly changing the transportation data landscape through the implementation of a rich network of real-time data sources. DAAS primarily originated with GPS data cultivated from fleet management applications, however, a recent paradigm shift has seen private data providers shifting to a hybrid solution of fleet-based data and crowd sourced data generated from consumer-based GPS and Bluetooth-enabled mobile devices such as smartphones and personal navigation devices (PNDs). The emergence of the concept of “probe-based people” has led the way for the explosion in data generation.
Private data sources are ideally suited for consumer-grade traffic and traveler information systems.
Private data sources are most attractive when compared to public data sources when considering the rate of deployment and rate of coverage. Private data networks are rapidly expanding coverage and data-density of the coverage, outpacing deployment capabilities of traditional public point detection data resources. Private data services also minimize operations and maintenance (O&M) costs typically tied to traditional public agency data collection systems. Also, because the data is generally delivered via the internet, public agencies do not need to operate and maintain the communications and networking infrastructure generally required to operate public data collection systems. In addition, private sector data alleviates legal issues associated with privacy, data collection and open record laws.
Although private data services drastically minimize traditional O&M costs associated with public data collection and delivery systems, new head-end hardware and new or modified software will be required. Unique, dedicated applications and middle-ware are required to ingest process and disseminate data and information culled from private data services, with the degree heavily dependent on the type of use and application of the data. Although private data has many of the same characteristics and reliance’s as that of other proprietary systems, private data more resembles. The “black box” nature of private data can be limiting in the end users understanding of the delivered data. Another present short-coming of GPS-based traffic data is its lack of precision when compared to traditional point-detection systems inability to differentiate between HOT/HOV lanes and normal travel lanes. Also, GPS-based data is currently unable to provide accurate volume data. The ability to share, manipulate or enhance private data is highly dependent on contractual terms and vendor specific criteria.
To date, private sector data has been primarily used to provide traveler information to travelers and to provide a resource for performance metrics and evaluation. In order for agencies to realize full cost benefits associated with the use of private data, private data will need to be able to fully replace data will be required to fully operate day to day traffic operations. The next big leap will require private data to fully replace existing data systems required to operate transportation systems. For example, we will need to see private data provide operational data to feed operational algorithms for systems such as ramp metering, traffic signal and incident detection systems. This represents a conceptual shift and approach to the utilization of public data sources. One early move in this direction has seen private data procured to provide the needed data to run automated travel time systems which utilize DMS for travel time dissemination. Legal hurdles may need to be addressed and policy directives may be required to fully integrate private data with traffic management systems and ultimately relieving the need for public agencies to deploy, operate and maintain their own data systems in order to obtain all necessary data. This may also require reformatting of private data streams, middle-ware or modification of existing software in order for existing traffic management applications to correctly ingest private data. IntelliDrive will generate a huge amount of data. Open data represents a transitional state of data management from the once privately held yet public data to truly “open” public data, or “Data as a utility”.
It is likely that no single type of transportation data, public or private will be able to address all transportation data needs. Public data sources are ideal for feeding and operating public transportation and traffic management systems, although there are early signs private data is slowly entering these areas and supporting these systems. How will existing public data systems and private data systems integrate with the impending data-tsunami associated with the rapid adoption of the personal mobile computing platform and new robust data-generating engines such as the data generated by the systems associated with the Connected Vehicle initiative.
Further information:
Information Management Strategic Framework
http://www.ato.gov.au/content/downloads/cor48331nat11852.pdf
Data, Information And Knowledge Management Framework
http://www.slideshare.net/alanmcsweeney/data-information-and-knowledge-management-framework-and-the-data-management-book-of-knowledge-dmbok-3366885
Data Governance Institute
http://www.datagovernance.com/
Data Governance Blog
http://datagovernanceblog.com/
Terranautix 