The inefficiencies revealed by AI workloads necessitate a fundamental transformation in how data storage and computational resources interact.
For over a decade, cloud system designs have been founded on the deliberate separation of data storage from computational processing. In this framework, storage primarily served as a repository for data, while all intelligence resided exclusively within the compute layer.
This architectural approach proved effective for conventional analytics tasks that operated on structured, tabular data. Such workloads are predictable, often scheduled, and typically involve fewer processing engines working with the datasets. However, as AI transforms enterprise infrastructure and demands, redirecting data processing towards vast amounts of unstructured information, this established model is proving inadequate.
What was once a significant advantage in efficiency is progressively becoming an inherent financial burden.
How AI Uncovers the Expense of Segregation
AI introduces demands fundamentally different from the analytical workloads businesses have grown accustomed to. Instead of batches of tables and rows processed by a single engine, modern AI pipelines now manage extensive volumes of unstructured and multimodal data, simultaneously generating large quantities of embeddings, vectors, and associated metadata. Concurrently, processing has become increasingly continuous, with numerous compute engines repeatedly accessing the same data—each retrieving it from storage and reformatting it for its specific purposes.
The outcome is not merely increased data movement between storage and compute, but also greater redundant effort. The identical dataset may be read from storage, transformed for model training, then read again and reshaped for inference, and once more for testing and validation—each instance incurring the full expense of data transfer and manipulation. Consequently, it’s unsurprising that data scientists devote up to 80% of their time simply to data preparation and organization, rather than model development or performance enhancement.
While these inefficiencies might be easily overlooked on a smaller scale, they rapidly evolve into a primary economic limitation as AI workloads expand, leading to not only wasted hours but substantial infrastructure costs. For example, 93% of organizations currently report underutilized GPUs. With high-end GPUs costing several dollars per hour on major cloud platforms, this underutilization can quickly amount to tens of millions of dollars in paid-for compute resources going unused. As GPUs increasingly consume infrastructure budgets, architectures that leave them idle, awaiting I/O operations, become progressively harder to justify.
Transitioning from Static to Intelligent Storage
The inefficiencies brought to light by AI workloads indicate a pivotal transformation in how storage and compute capabilities must interact. Storage can no longer function merely as a quiescent system for records. To effectively support contemporary AI workloads and maximize the value derived from available company data, computational power needs to be integrated closer to where data resides.
Economic realities within the industry underscore this point. A terabyte of data housed in conventional storage largely represents an overhead. When that same data is moved to a platform featuring an integrated compute layer, its economic worth multiplies. The data itself remains unchanged; the only differentiator is the availability of compute resources capable of transforming and presenting that data in valuable formats.
Instead of continuously relocating data to realize its value, the optimal solution is to bring computation to the data. Data preparation ought to occur once, at the data’s source, and be reusable across various pipelines. Under this paradigm, storage evolves into an active tier where data is processed, organized, and delivered in forms optimized for subsequent systems.
This change impacts both performance and financial aspects. Pipelines operate more rapidly because data is pre-processed. Hardware maintains higher productivity as GPUs spend less time waiting on repetitive I/O operations. The expenses associated with redundant data preparation begin to diminish.
Within this updated framework, “smart storage” transforms data from a mere stored entity into a resource that is continually comprehended, enriched, and made ready for deployment across AI systems. Rather than leaving raw data confined to inert repositories and depending on external pipelines for interpretation, smart storage directly applies compute within the data layer to generate persistent transformations, metadata, and optimized representations as new data arrives.
By preparing data once and ensuring its reusability across multiple workflows, organizations empower storage to become an active platform instead of a hindrance. Without this fundamental alteration, organizations will remain entangled in cycles of repetitive data processing, constant reformatting, and escalating infrastructure costs.
Architecting for the AI Epoch
The cloud’s architectural principle of separating storage and compute was appropriate for its era. However, AI workloads have profoundly altered the economics of data and revealed the limitations of this methodology—a constraint I’ve observed derail numerous enterprise AI initiatives, and a primary motivation for founding DataPelago.
While the industry has begun concentrating on accelerating discrete stages within the data pipeline, efficiency is no longer achieved by extracting marginal improvements from existing architectures. Instead, it is now determined by constructing novel architectures that render data usable without repeated preparation, excessive relocation, or squandered computational power. As AI’s requirements become increasingly clear, it is evident that the forthcoming generation of infrastructure will be defined by the intelligent integration of storage and compute.
The enterprises that achieve success will be those that establish smart storage as a cornerstone of their AI strategy.
—
New Tech Forum offers a platform for technology leaders—including vendors and external contributors—to delve into and discuss emerging enterprise technology with unparalleled depth and breadth. The selection process is subjective, based on our identification of technologies deemed important and of highest relevance to InfoWorld readers. InfoWorld does not accept marketing materials for publication and retains the right to edit all submitted content. Please direct all queries to doug_dineley@foundryco.com.
