A Rural Real-Time Network

West Virginia’s hills and dales create challenges for the design, construction, and operation of a statewide RTN.
by Dr. Pete Dailey

The design and operation of real-time networks have been well documented by RTN owners such as the Ohio Department of Transportation’s Aerial Mapping Division (ODOT) and the South Carolina Geodetic Survey. We at the Rahall Appalachian Transportation Institute in West Virginia have worked closely over the years with our Ohio neighbor and modeled our RTN loosely after theirs. However, design and operation of an RTN in a rural state like West Virginia present some unique challenges.

West Virginia has been challenged from its birth. Once part of the original 13 colonies, it was cleaved from Virginia during the Civil War into its odd shape. It is the only state in the union that lies entirely within a distinct cultural region referred to as Appalachia. Its nickname, “the mountain state,” gives fair warning to the nature of its terrain.

For many RTN designers, laying out continuously operating reference station (CORS) 
locations is not particularly difficult. By simply opening a map in CAD, they can overlay 
the desired grid spacing and locate CORS at or near the grid intersections. This is not 
the case in West Virginia. We have two panhandles that jut into gaps between 
Pennsylvania and Ohio to our north and Pennsylvania, Maryland, and Virginia to 
our east. We had to take a much different tack to CORS location.

Isolating Terrain

A glance at a terrain map indicates (as my former geography professor once quipped) that if West Virginia were flattened out it would be the size of Texas. West Virginians, along with their transportation systems and businesses, tend to congregate in the flatland of river valleys. And with a population of 1.8 million, state resources are at a premium.

Rural and rugged means that building and maintaining transportation systems are expensive, resulting in economic isolation from the rest of the country. The Appalachian Regional Commission (ARC) was formed by the federal government to assist challenged Appalachian states—from Mississippi, Georgia, and Alabama through Pennsylvania and New York—to deal with the isolating aspects of terrain.

We also have extensive rail and river networks for the transportation of coal. But until late last year, trains carrying containers from East Coast ports had to spend an extra day circumnavigating the state because of 28 “Pokey Specials,” which are the layout of a bridge-tunnel-bridge combination in mountainous terrain in order to maintain a reasonable railroad grade. (The name comes from the Native American, Pocahontas.)

The Rahall Appalachian Transportation Institute (RTI) is a university transportation center on the campus of Marshall University with a mission to research transportation issues specific to the region. We study and engage in technology-transfer activities in rail, maritime, and highway modes. Norfolk Southern and CSX have an extensive railway network in West Virginia, and according to the U.S. Army Corps of Engineers, the port of Huntington is the largest inlet port in the United States, attributable to the volume of coal traffic traveling on the Ohio River.

The West Virginia Division of Highways (WVDOH) GIS group asked us to provide technical assistance in design, construction, and operation of an RTN. RTI purchased and set up three CORS sites in 2005 as part of a research project, with two of the sites supplying one-second observation data to ODOT’s RTN since 2006.

WVDOH GIS group leader Hussein Elkhansa identified a statewide RTN as an opportunity to develop greater efficiency within the WVDOT while improving position accuracies inside and outside the WVDOT. The WVDOH RTN benefits are shared by other state agencies, such as the Divisions of Forestry and Natural Resources (in their mapping work) and the Department of Environmental Protection (in better defining permit location boundaries and maintaining permit compliance; surface mines must adhere to a mining plan that approximates the original contour).

Virtualized Servers Favored

For computing power, we chose to depart from the use of specific hardware in favor of virtualized servers. Our CIO Dave Lawson argued that full computer virtualization could simulate the operation of real hardware by running an unmodified guest operating system. In our case, we designed the primary server farm as a mix of Windows 2008 Server and Linux Server operating systems. By combing virtualized servers into a few host machines—the real hardware underlying the virtualized operating system—the total computing cost is reduced to using only the computing power that we need rather than committing to the purchase of excess hardware.

This decision was a great leap of faith by our partners at WVDOH, as our choice of virtualization host was untested by our software vendor due to the numerous combinations of hypervisor software and guest OS. Virtualization not only let us pay for only the computing needed; a virtual server can be customized for the number of processors and memory, while the virtual environment makes backing up the entire server farm a matter of recording and retaining periodic snapshots of each server. Because computing, data I/O, and related power and communications redundancy to maintain system uptime aren’t tied to a physical plant, the backend becomes an outsourced commodity, with plenty of data centers around the country having a reputation for hardware and communications reliability. Virtualization reduced capital equipment, maintenance, and personnel expense, which helped to focus our efforts on managing the service, not the hardware.

CORS and user accounting information is recorded to several MS SQL databases. Data are backed up to a data center physically removed from the main computing center. Our data-retention policy keeps compressed, hourly CORS observations and troposoheric- and iononospheric-model processing results for 90 days, with RINEX files held for 30 days. By retaining the network processor results, the VRS3Net Reference Data Shop module can use physical CORS observations, apply the network-processor results to produce a virtual CORS at a user-defined coordinate anywhere within the network, and produce a RINEX file 
for use in post-processing. Post-process ing users can reduce proximity errors in the same way real-time users connect to a virtual reference site.

Our CORS receivers deliver one-second observations in two ways. A Trimble VRS3Net server manages the real-time CORS data stream, epoch synchronization, and storage. Observations are stored to a Linux server. In addition to the storage function performed by the RTN software, NetR5 receivers push hourly one-second observations to an FTP server. Because FTP is a notoriously insecure protocol, a virtual ftp site was set up on the Linux server. The CORS receivers push hourly files using virtual user login.

Other scripts running on the Linux machine copy files, perform disc management chores, and run a report for each site every 24 hours using UNAVCO’s Translation, Editing, and Quality Check utility. A script examines the data stored by VRS3Net software, first to see if it exists and then to see if the hourly CORS receivers’ file in the FTP sandbox is larger than the VRS3Net recorded file. This scheme allows us to maintain our data-delivery commitment to NGS in case the production VRS3Net server and backup system are disrupted.

CORS Requirements Not Always Easy in West Virginia

Any CORS used in an RTN needs a reliable source of power, internet connectivity, a clear sky, security, and an acceptable monument location. These are not always easy to find all together in the same place in the mountains and valleys of West Virginia.

Power: Line power was augmented with a hefty UPS capable of powering the receiver for five days. An ice storm tested this design within the first month by knocking out power to the WVDOH Oak Hill District HQ for four days. When power and internet were restored, the previously recorded data was pushed to the waiting server.

Internet: To minimize monthly service charges, we exhaust most other communication possibilities before committing to installing a DSL line or wireless data subscription. A streaming CORS doesn’t need much bandwidth; however, excessive latency results in missed observations in the synchronizer. The Governor’s Office of Technology IT network group has been helpful in alleviating latency issues.

Sky: Because the greatest population density is in river valleys, mountains may mask 15 to 20 degrees or more of the horizon.

Security: Security concerns are a primary impediment to deploying an autonomous CORS package, so we have designed a self-contained CORS, including electronics, solar panels, charging and communication equipment, and mount in a single deployable package. Locating a CORS at one of the department of corrections’ jails provides a high degree of security, with NGS respectful of prison security by witholding publication of photos that may provide unintended escape-route mapping services.

Building selection: Where appropriate to the site conditions, building mounts are preferred due to reduced construction and labor cost. Antenna mounts should be located, as NGS guidelines suggest, on two-story masonry buildings five years or older. Due to the narrow selection criteria, CORS site selection occasionally depends, as Tennessee Williams’ Blanch DuBois might say, “on the kindness of strangers.”

Sites selection took on a hierarchy of sorts. Three universities and a community college provide excellent sky, power, internet, and security. Primary locations are at WVDOH district headquarters and maintenance garages. Other state agencies benefit from the synergy, with the WV Department of Environmental Protection providing two receivers, and the Division of Natural Resources, state parks, and prisons providing excellent site locations.

At the county level we approached several boards of education, with mixed results. Initially, we believed that because all West Virginia’s secondary schools are wired with high-speed internet we could add many secondary schools to the selection queue. We found that secondary-school internet access is funded through a federal grant, with a strict restriction forbidding any non-school use of bandwidth.

Cooperative relationships with RTN operators in neighboring states have provided a mutually beneficial way to enclose the state border through CORS observation sharing.

We designed two standard GNSS antenna installations: building mounts and stand-alone monuments. NGS posts photographs of nearly every CORS accepted into the national network. The photos provide a wide range of accepted antenna mount types. Using the photos and considering material on hand, Michigan’s reinforced “light pole” design stood out as an interesting idea. Erecting light poles is a skill every state highway department excels at, and the WVDOH is no exception.

Also fortuitous to material procurement were slightly tapered, thick-wall aluminum poles that were rejected after procurement due to changed federal highway guidelines. A chop saw, scrap plate for fillets, and a skilled aluminum welder at the WVDOH Buckannon garage produced our standard “rocket boy” mount, nicknamed in honor of West Virginia native, author, and NASA astronaut trainer Homer Hickam. Base sections are a typical light pole base, with four studs anchored to the rebar to support and level the antenna mast and provide a grounding path. Fine leveling is accomplished with a SECO commercial leveling mount under the antenna.

Building mounts were likewise constructed of aluminum. We believe aluminum mounts and masts are easier and safer to handle and are corrosion resistant. Because NGS requires CORS installations to insure that the mast/mount won’t shift position throughout a 20-year design life, we drilled and bolted the mast to the building mount.

Chief of surveys for the WVDOH, Travis Long, took on the responsibility of field installation and maintenance, assembling an installation crew, procuring construction materials and tools, manufacturing building and monument mounts, and scouting CORS locations. His installation crew consisted of two young surveyors, Jimmie and Jesse, on temporary assignment. Travis commandeered a surplus WVDOH trailer to haul a small generator, ladders, a hammer drill, hand tools, building mounts, masts, leveling mounts, and common electrical components such as lightning suppressors, UPSs, receivers, antennas, and yards and yards of butyl weatherproofing tape.

Identical designs simplified later troubleshooting and replacement of lightning or surge suppressors. Communication with receivers for firmware updates uses several different routes, depending on the network manager. Most receivers can be accessed through the internet for service upgrades or configuration changes. The majority have a fixed IP address. Other units with DSL connection use a dynamic DNS forwarding service. We use dyndns.com because Trimble NetR5 receivers have a built-in DDNS client for this and other forwarding services. Those CORS not yet assigned an address outside the state network are accessed from inside the WVDOH LAN. Few require a site visit.

Some users provided feedback during our pre-commissioning phase. One of the most-difficult problems in the rugged terrain is radio propagation—how to get real-time data to the user’s rover. Ridge-to-ridge cellular data communication works well enough, but a surveyor’s real work is in the hollers (a West Virginia colloquialism for a hollow place in the mountain, a valley). RTK bridging, aka RTK rebroadcast, is used to connect to the RTN through commercial cellular circuits from the ridge top, then rebroadcast the corrector stream using VHF/UHF data radio into the valley with a high-gain Yagi antenna. WVDOH equips each survey crew with a netbook with internal cellular modem. The netbooks run GNSS Internet Radio (courtesy of the German Federal Agency for Cartography and Geodesy) and pass the corrector stream to the data radio input. The truck/radio remain on the ridge while the survey work is performed in the valley.

West Virginia’s Real-Time Network of 35 CORS provides a statewide geodetic network tied to the National Spatial Reference System. Future plans are to expand the RTN beyond a dynamic geodetic reference with additional software and analytic processes to enable civil infrastructure monitoring of bridges, dams, and earthen impoundments. The statewide RTN has expanded the scope of our research in railway track inspection. Other applications in underground mine rescue, construction, and mining machine control will fully leverage the capabilities of the WVDOH RTN investment.
Dr. Pete Dailey is with the Rahall Transportation Institute, a university transportation center on the campus of Marshall University in Huntington, West Virginia.

» Back to our June 2011 Issue