ForgeFX Simulations is honored to be a long-standing training simulator development partner of renowned Midwest research institute MRIGlobal. ForgeFX collaborated with MRIGlobal to develop Checkpoint, an augmented reality tabletop mission planning prototype application built for the Microsoft HoloLens 2. This application was recently recognized by the Joint Program Executive Office for Chemical, Biological, Radiological and Nuclear Defense (JPEO-CBRND) with a 2021 SPARK award for Multidimensional Understanding. SPARK awards are granted to identify novel approaches for delivering the best solutions for countering weapons of mass destruction (CWMD) and CBRND capabilities to the warfighter in the shortest amount of time.

SPARK Initiative

The Joint Program Executive Office for Chemical Biological Radiological and Nuclear Defense provides funding for transformative projects via the SPARK initiative, JPEO-CBRND’s idea incubator. The SPARK Innovation Initiative provides “venture capital” for emerging JPEO program areas before they become free-standing funded programs. A development team composed of ForgeFX and MRIGlobal members submitted a proposal for an augmented reality mission planner prototype which was awarded development funding via SPARK.

The Virtual Sand Table

Tactical sandboxes and tabletop exercises are traditional military mission planning and training tools with a long history. Typically, they are scale models of a terrain either built upon a table or on the ground. Objects representing troops, vehicles, buildings, and other pertinent features are positioned across the surface. This arrangement serves as a focus with which team or troop members discuss strategic and tactical operations. The US military has been interested in evolving and improving upon computer-generated virtual sand tables (VSTs) for the last 25 years. A VST solves several limitations associated with real world models: they boast immensely increased representational detail; they provide flexibility through non-destructive editing; and they expand the convenience of working collaboratively in a multiuser space.

Checkpoint Augmented Reality Mission Planner

Checkpoint was developed specifically to accommodate the type of missions that utilize the Nuclear Biological Chemical Reconnaissance Vehicle (NBCRV), a high-speed armored carrier equipped with a Sensor Suite Unit that detects multiple hazardous contaminants. Other equipment models provided by the application are a Stryker RV, an unmanned ground vehicle (UGV), and an Unmanned Aerial Vehicle (UAV). Mission planning elements such as boundary and movement lines, various personnel formations, equipment, and installations are represented by standard military symbology.

Upon launching Checkpoint, the user is presented with a virtual sand table with satellite image-quality terrain sectioned by a grid labeled with military-style coordinates. The terrain includes a wide variety of topographical features, including a section of coastline with port facilities, hills, country, towns, and roads. The terrain is scalable, ranging from a maximum of six kilometers to a zoomed-in scale of 500 meters. Above the terrain virtual table are UI panels which provide access to the library of symbols and a toolbar that enables toggling the various element layers on the map.

From the Symbols panel, the user taps an icon then drags it into any position on the terrain surface. They define the boundaries of the area of operations and the phase lines of the mission by positioning the end points of each line. Other strategic markers such as decision points, check points, named area of interest, and the suspected enemy position can be added. A third category of symbols includes several formation types and sizes, such as infantry, reconnaissance, engineer, and Special Operations Forces typical to an NBCRV mission. Finally, a line-of-sight tool can be positioned to indicate which elements are in view of each other. All these symbols, along with the NBCRV, RV, UAV, and UGV vehicle symbols, can be iteratively repositioned and resized to accommodate the needs of the mission and its planning.

Networked Multiuser Component

One of the stand-out benefits of the Checkpoint Mission Planner is its capability to provide networked multiuser interaction via the Microsoft HoloLens 2 telepresence functionality. Checkpoint accommodates multiple users per session, allowing users to see others’ avatars and speak via VoIP. While everyone sees a view of the terrain based on their virtual position at the table, each person sees only their own version of the UI panels and can rearrange them as they find convenient. Once a person drags an object from a panel, it enters shared space and can be manipulated by others. Checkpoint’s shared virtual table also provides “room” for many more people than could gather around a similarly sized table in the real world.

Just as people increasingly work from home and meet via video calls, the chief advantage of Checkpoint’s multiuser telepresence is that it overcomes the limitations of communicating and collaborating via physical presence. Additionally, its open-ended multiuser capability not only allows people to structure a meeting however they need, from a solo session to team collaboration to a lecture or structured planning meeting; they can also record any session for playback at a later time.

Checkpoint Potential

The Checkpoint Mission Planner virtual sand table offers many benefits over a traditional sand table, mainly the flexibility that comes with non-destructively editing and rescaling objects, the superior informational detail inherent in a 3D rendered terrain, and the power of networked multiuser telepresence interaction. The Checkpoint Augmented Reality Tabletop Mission Planner exemplifies the potential of augmented reality technology to improve upon an essential strategic and educational tool.