There is no denying “big data” and its importance to next-gen ITS applications. The emergence of a vast,
omnipresent data cloud is enabling new knowledge and wisdom to be attained, as well as facilitate new operations models for the mobility manager. Unfortunately, parochial data systems and data management strategies are quickly becoming obsolete with regards to managing this quickly evolving paradigm. As a result, the need for operating institutions and mobility managers to understand “big data” and implement new, comprehensive and overarching data management strategies has never been greater. Next-gen data and information systems will need to be autonomous, contextual, predictive and real-time. The overall impact is cascading in that now a new strategy is not only desired, but will become an essential function, as the proliferation of meaningful data sources accelerates. The time for agencies to plan, prepare, implement and transition is now. The following aggregates a few thoughts into an introductory package for agencies to consider as they get started, in hopes of widening the road to success.
THE NEED
Although all of the values of new “big data” resources are not yet fully understood, the danger of getting bogged down in the data deluge is already being felt. Before these new values can be leveraged, we must first review, research and retool, predicated on a sound understanding of existing conditions and extensive research and evaluation of likely future conditions and future capabilities. In addition, programmatic and industry changes such as MAP-21 and the Connected Vehicle are changing the operational fabric and are mandating new requirements for mobility managers, and thus, also need to be considered when developing a new data and information management strategy.
WHAT / HOW MODEL
So where to start? – The following insights are framed within the “What/How” solutions model, or “What do we want/need?”, and then,”How do we do it?” As is the case with all sound planning efforts, an accurate understanding of existing conditions is an essential first-step prior to commencing with future planning efforts.
What
Stakeholders and Champions – The first step is to identify all possible stakeholders (including champions and arbiters), both internal and external to an operating entity. It’s key to remember that the data paradigm shift will cover all departments, agencies, programs and offices within a city and/or region, therefore coordination with an overarching perspective is essential for success. Typically non-traditional stakeholders will now play important roles and become key teammates. The identification of the initial list of stakeholders should include a first draft of a new steering committee or “Data Management Team” (DMT), which should encompass all pertinent agencies and institutions.
What do we have?
Following the formation of DMT, the team should begin to assess existing conditions. Some key questions to get started include:
- What are our existing data generators?
- What systems are required to support these data generators?
- How do we currently source, transmit and aggregate data from existing data sources?
- What data and information-based goals and objectives are currently in place?
- What are our existing processes for measuring and monitoring the path towards prescribed goals?
- What values are we realizing/not realizing?
- What standards and formats do we utilize?
- What policies and regulations currently exist?
- What quality control processes and procedures are in place?
- What licensing, warranty and policy factors impact our data and information systems?
These questions will likely uncover significant new understanding as to how an agency currently handles data, and identify opportunities lost or new opportunities for functional improvements. The baseline assessment needs to include identification and mapping of existing supporting systems and infrastructure, including networking and software applications. The exploration should also begin to drill down and refine existing information such as data attributes. A list of attributes might include:
Data
- Source
- Owner
- Use rights
- Format
- Polling rates
- Current uses (realized)
- Potential uses (unrealized)
- Quality
- Cleansing/conditioning
Data Support Systems and Applications
- Infrastructure requirements
- Software dependencies
- Other OSI reference model considerations
Policies, Guidelines and Contracts
- Use policies
- Cost per byte/poll
- Licensing and Warranties
- Existing vendor contracts, limitations
- Storage and Retrieval
- Performance metrics and monitoring
- Existing staff requirements
Interim Review – Immediately following initial exploration of existing conditions, the Data Management Team should conduct an interim review of its findings. In addition, the DMT should review any and all existing goals and objectives related to data and information systems. What are we truly trying to accomplish and what are we achieving? What are we not achieving? What are the perceived initial gaps? The initial review of existing conditions will likely trigger additional exploration needs with regards to existing data and information systems. The interim review will also likely uncover additional stakeholders, both internal and external to the mobility management ecosystem.
Mapping – Map your findings. As with all good wayfinding processes, a “you are here” marker is essential. The goal is to map all exploration activities and contextualize the existing data and information system landscape. In addition to narrative and graphical mapping, a spreadsheet or database is also helpful for tracking results such as data and information attributes.
Projections and Forecasts
The next step will be to begin exploration and research of existing trends and to conduct forecasting of future trends and forecasted conditions. Predicting the future is always challenging at best. However, with a sound, comprehensive strategy in place, an organization can best plan and implement strategies that prepare an agency for potential future conditions. Trends analysis and future conditions forecasting will assist in establishing a pragmatic orientation for the foreseeable future. These assessments should be conducted in parallel, yet separate paths from the existing conditions exploration and mapping tasks. (The simultaneous work efforts will assist in finalizing the existing conditions survey task by uncovering additional gaps in the initial existing conditions survey and identify additional existing conditions research required).
Current Trends – Current trends such as cloud-computing, smartphones, mobile apps, private data sourcing, crowdsourcing, and integrated corridor management (ICM) need to be identified and included in new data management strategies. MAP-21 and other Federal requirements will mandate a new minimum acceptance level for the operating entities and also need to be immediately included in planning efforts. It’s important to look past today’s sheen of certain applications and technologies to truly understand where industries and agencies are headed.
Future Trends – Connected Vehicle, including V2X, or V2I components will directly impact operating agencies and the way they do business in the coming years. Other likely future trends such as the autonomous vehicles, City as a Platform and integration of transportation networks will directly impact the data and information framework. Additional trends such as system automation and data driven systems will amplify the need for pertinent real-time data.
Research
The “Future-Casting” task should also assign segments of industry to in-house champions (domain expertise), in order to monitor federal regulations, funding streams, the information technology and automobile sectors, university, state and federal research tracks, consumer technology markets, as well as tangential markets and adjacent internal agencies and divisions.
What do we want/need?
Immediately following the initial existing conditions survey and research and forecasting of future trends and conditions, the DMT should revisit original goals and objectives regarding data and information systems, and modify/append accordingly. At this point, a traditional “User Needs and Preferences” assessment can be conducted, and should follow a traditional Systems Engineering framework. Some of the basic questions to address include:
- Have we properly identified and defined all of our goals and objectives
- How do you plan to leverage enriched data environments?
- How will this foster enhanced wisdom and adaptive genius within our mobility ecosystem?
- How will me monitor our progress towards achieving our goals and objectives (performance measures)
- Have we instituted agency changes appropriate and sufficient to meet our goals and objectives?
To this point, you should have a pretty sound understanding of all of the existing data and information systems within the agency/region. However, it may require additional iterations of the exploration, mapping and wants and needs assessments to truly understand where you are, and where you want to be (goals).
How
Once goals and objectives have been set, we can begin to assess “How” do we get there? As with most planning efforts, an alternatives analysis and a Long Range Plan and Implementation Plan need to be developed. A scale vs. value and ROI assessment is conducted at this point as well. As is always the case with future-proofing, the key is not to plan to design for specific (undefined, and in some cases unknown) technologies, methodologies and strategies, but to identify and anticipate likely future conditions and implement a framework that is agile, flexible and capable of embracing future technologies, strategies and methodologies.
The next step is to establish a requirements-based blueprint and roadmap to transition from today to tomorrow. It’s also important to set measurable goals and identify necessary performance metrics in order to track progress towards goals and objectives, and to be able to conduct evaluatory assessments. This step should also include a traditional gap analysis as well. The Long Range Plan should also include a Concept of Operations. This step will also begin to define “rewiring” necessary for executing the new data and information management program, which should also include business rules. In addition, new data management schema needs to be integrated with the overall (typical) planning processes, including budgeting, long-range plans and regional plans.
Staffing resources and annual operations should also be assessed at this point. Domain expertise, staffing and skills requirements will need to be addressed. This should be included in the initial existing conditions exploration. A new Data Manager position is likely the most appropriate first hire. This individual may be an MPO, DOT or local agency staff person in charge of overseeing all harmonization of data and information systems across all platforms, jurisdictional and agency boundaries. A Data Scientists/Analysts will also likely be required.
Additional Challenges and Potential Impediments to Consider
Initial Buy-in and Engagement – As with most new initiatives, getting up from the “comfy couch” can be the biggest challenge to implementing new or improved strategies. Generating the initial inertia and momentum will require champions at the administrative, technical and arbiter levels, within all stakeholders, departments, agencies and regional staff (MPO).
Data use and retention policies – some data may be approved for certain uses, however, additional uses may raise privacy, licensing or ownership issues. This challenge also gives rise to additional hurdles including operational governance and regulation of the new data and information system. For example, can private data be sourced to operate public systems (signal systems, etc.) were safety is critical?
Integration and Standardization – what level of data and system integration is optimal, or will achieve the greatest Benefit/Cost ratio for an operating entity? What granularity and resolution (data density) is required for each component of the goals? Automated monitoring and performance reporting will be a key to success with regards to overall integration and standardization.
Sustainability – A new funding stream (outgoing) is likely required. However, the potential for additional revenue streams (incoming) is also likely. Funding needs to be identified for the initial capital outlay, as well as annual operations and maintenance cost for the life-cycle of the system and subsystems.
Security – As the data reservoir expands, and the network to support and manage the data and information systems expands, so will the security concerns. New policies and data management applications will be essential. Data storage, encryption, access rights, use rights as well as infrastructure and support applications should all be included in the initial security assessment and security planning efforts.
RESOURCES
Transportation Data and Information Systems – LinkedIn Working Group
http://www.linkedin.com/groups?gid=4929972&trk=myg_ugrp_ovr
USDOT Research Data Exchange
http://www.its.dot.gov/assetviewer/
Research, technology, and data drive America’s transportation system – USDOT Transportation Secretary
http://fastlane.dot.gov/2013/03/researchg.html
Real-Time Data Capture and Management
http://www.its.dot.gov/data_capture/data_capture.htm
the enhanced density in data sources is establishing a huge resource for transportation operators. The rapid proliferation of data sources from new devices such as smartphones and other newly connected devices, in conjunction with the advancement of technologies for data collection and management have manifested a sizeable inflection point in the availability of data. So what does this mean for ITS operators and the systems they currently manage? What will be required to extract and leverage values associated with “big data”?
Terranautix 