Unleashing Data to Advance Our Defense Strategy
When most of us think about moving data, we don’t think of an amount of data that requires a semi-truck to move it. But as applications at the edge are generating copious amounts of data, the calculus for this bottleneck has led One Stop Systems to build a family of storage solutions to address these needs with an eye for forward, harsh environments. The DoD data Strategy to become a data-centric enterprise requires that all DoD leaders treat data as a weapon system and manage, secure, and use data in the most effective ways possible. Being a data-centric organization, the vision is to use data at speed and scale that improves operational advantage and increased efficiency. Focusing on data for an advantage on the battlefield, to improve oversight, or to drive informed decisions up and down the command chain, are of paramount concern. This is essential to pivot on data more rapidly than our adversaries.
In the commercial world, Amazon addressed these needs by creating the Snow family of products. They come in small, medium, and large, with the Snowmobile being a 45-foot semi-truck able to transport 100 Petabytes of data. Although this type of solution might be satisfactory when transporting between a DNA sequencing lab in New York and a data center in Los Angeles that has an interstate connecting the two, it falls short when dealing with Edge positions in hostile or remote environments.
Like all storage concerns, the ability to capture and convey data to an actionable conclusion remains the same. But acknowledging that today’s sensor arrays, machine learning, satellite feeds, radar arrays, optical telescopes, and the multitude of other data sources have led to the need for data capture in ways that the physical transfer of data in a container has its advantages. At the extreme with a very expensive dedicated network supporting a 25 Gbps connection enabling the transfer of 200 terabytes each day you might expect to have a connection fee that exceeds $40,000 a month. Although this might be reasonable in some environments; the practicality is limited.
The capture and moving of data are simple enough when calculating the volumes, speeds, and costs of the link. And in those instances where cost is the regulating issue, statistics such as the one Amazon reported on indicating that a data warehouse containing 40TB can exceed $880,000 annually can easily be delineated to the time value of money. In penciling out the cost of a dedicated data link and the time value of the data, being able to transfer data physically can shave significant costs and be done faster. In an optimized network, the pipe between the sender and the receiver is maximized by keeping the in-flight data at maximum data rates including all the overhead, but when days turn into weeks, the physical transfer stands out as the solution of choice.
The Solution
In situations where the edge environment is mobile or is in a forward command center, the OSS storage solutions are designed to meet the rigors of a hostile environment – referred to as their Centauri product family. Three configurations can address situations from small, medium, or large storage capacity and throughput requirements. Using standard memory modules, the storage arrays can house multiple petabytes of data utilizing PCIe Gen 4 interconnect for lightning-fast up and downloads. Based on SWAP-C, OSS has created a balance between functionality and value to solve the age-old question of data transfer.
Using discrete, physical transport of data has significant advantages in protecting the data from corruption or malware via a commlink. Although not the primary reason that one might choose this idea of “sneakernet”, the benefits for those susceptible to security risks are comforted in knowing the possibilities have been reduced to near zero. This also goes for high-value data that might normally require high availability of commlinks that are put at risk with transfers that take hours if not days to complete. And if that isn’t enough, the memory canisters can be populated with U.2 modules that are FIPS 140-2 compliant. (Remember that adding encryption ((IPSec or SSL)) will inflate the amount of data to be transferred along with encryption and decryption delays as much as 15%).
Use Case
The ability to address the applicational needs has enabled OSS to develop semi-custom solutions that work in concert with onboard systems as well as be configured in a compact form to reduce weight and the use of power. In a recent application, the Engineering team of OSS coupled their half-width, 3U removable canister in an autonomous truck. Originally designed to sit next to their high-performance compute platforms, the application demanded that the Centauri be placed in the “saddlebags” of the truck capturing data both from the autonomous driving systems along with monitoring the truck and cargo during the trip. In this application, the use of physically transferring data was driven by the costs and the mobility of the truck. The convenience of packaging up to 8 NVMe SSDs for a capacity of up to 256 TB enables a significant cost saving as well as shaving off days in returning the data. Although this application was driven by cost, the environment of being onboard a truck made OSS uniquely position to address the problem.
Increasingly, companies are gathering and processing a large amount of data at the edge where it is generated. These environments have several unique attributes that require configurations specific to the application. Although the copious amounts of data that a movie camera might capture resembles the video feed from a reconnaissance plane, the need to address the environments and the attributes that define them is something altogether different. OSS’s history of bringing data center performance to the edge by designing to the environmental standards of Mil-STD 810 G creates a broader solution set to address almost any application. Based on PCIe Gen 4, the Centauri family of products will continue to have backward compatibility as the market evolves to future generations of the standard. OSS continues to lead the market with advanced PCIe solutions.
Collective IP
In applications involving forward command centers, remote research laboratories, or onboard a ship, OSS’s largest solution can capture petabytes of data in a form about the same size of a household brick through collaboration with Tsecond Inc., a portfolio company of AEI HorizonX, a strategic venture capital fund anchored by Boeing Tsecond, Inc. is working to accelerate the value of the global datasphere by addressing the complexities of mass data generation. Tsecond, Inc. in reviewing the complexities associated with the mass collection of data, determined that some of the methods used by the commercial sector could be advanced in support of areas utilizing AI and Machine learning. OSS and Tsecond, Inc. combined their resources to offer complete solutions for AI storage and transport applications. The combination of these two companies reduces the financial costs of moving large sums of data by reducing the capture, transport, and access. It enables technologies that previously had been limited to engineers in lab coats to be deployed to the field. OSS has combined its understanding of harsh environments with the challenges of big data.
The Benefits for the DoD
The OSS solution enables raw or structured data to be captured and physically transferred from advanced positions to data centers that can combine and index the data for greater value. The durable containers reduce the risk of the data being damaged in transit and or the data being corrupted during transmission. Composable and elastic, the ability to transport multiple petabytes of data and use canisters in parallel affirms that operational use is not compromised by downtime. Additionally, having the canister in one’s custody greatly reduces the possibility of a security breach.
It is estimated that the Pentagon produces more than 22 terabytes of data each day. Jet Propulsion Lab and Fort Huachuca create 12 terabytes each. A 2018 collaboration to build the largest radio telescope is expected to generate 700 terabytes of data every second. Although these are extreme examples of data collection, they reflect an overall explosion of data created by the connected battlefield. Everything from surveillance to inference computing and neural nets will challenge system architects to develop solutions at the edge that meet these rigors and demands.