Seamless Cloud Migration

Current solutions for cloud infrastructure portability frequently fall short of expectations. Infrastructure as Code (IaC) tools such as Terraform often oversimplify complex infrastructure into generalized terms. Cloud provider-specific services like Azure Migrate, AWS Migration Services, and Google Cloud Migrate typically struggle to convert native workloads for rival cloud platforms. Furthermore, while governance tools effectively highlight finops and drift problems, they seldom offer practical solutions to resolve them.

I’ve detailed these challenges in a related article, which I encourage you to read first. Here, I’ll present an overview of Cloud Cloning, an infrastructure replication method developed by my team and me at FluidCloud, designed to tackle the aforementioned issues. I will now guide you through how our solution operates and what it achieves. 

Comprehensively Capturing Public Cloud Configurations

To facilitate cloud portability, Cloud Cloning’s initial step involves taking a comprehensive snapshot of the source cloud’s infrastructure. It leverages the cloud provider’s APIs to scan and record the entire infrastructure footprint, encompassing all resources, interdependencies, and critical details such as VPCs, subnets, firewall rules, and IAM (Identity and Access Management) permissions—elements frequently missed by conventional multicloud tools.

To contextualize the benefits of this comprehensive snapshot, it’s useful to consider the origins of multicloud tools. Initially, the multicloud domain developed to aid migrations from on-premises private clouds to public ones. Consequently, these tools primarily concentrated on replicating the two core components of private clouds: virtual machines (VMs) and storage. This emphasis largely persists today, with prominent solutions like AWS Migrate, Azure Migrate, Google Cloud Migrate, Nutanix Move, Veeam, and Zerto still predominantly centered on these two aspects.

However, the challenge arises when migrating between public clouds, where VMs and storage represent merely a fraction of the overall environment. Public cloud setups are built upon intricate architectures involving hundreds of services, including databases, storage buckets, IAM users and permissions, subnets, routing, firewalls, Kubernetes clusters, and their control planes, among many others.

By initiating the process with a full snapshot of the entire infrastructure ecosystem, Cloud Cloning successfully captures a significant portion of critical cloud elements that traditional tools often miss. Based on our observations, other solutions typically capture only 10% to 30% of the source cloud configuration, whereas Cloud Cloning achieves 60% or more.

Seamless Translation from One Cloud Environment to Another

Once Cloud Cloning has completely mapped the source cloud infrastructure, the subsequent phase involves adapting that infrastructure to the target cloud’s distinct services and configuration framework. Given the significantly diverse API specifications of each cloud, this translation task presents a considerable challenge. Let’s examine a few instances across three primary domains:

• Compute and resource management: While all cloud providers offer similar foundational compute and networking elements, their operational semantics vary. For example, AWS Auto Scaling Groups, Azure VM Scale Sets, and GCP Instance Groups exhibit different behaviors concerning availability, placement, and scaling. Security also differs: AWS security groups are exclusively allow-based; Azure employs prioritized, ordered allow/deny rules; and GCP utilizes separate ingress and egress firewall rules. These, among other distinctions, complicate the exact reproduction of deployments without re-interpreting the original intent for the target cloud.
• Storage and data: Storage and data services are not directly interchangeable between cloud platforms. Block volumes and file systems exhibit variations in performance, snapshot mechanisms, and consistency guarantees. Similarly, managed databases like AWS RDS, Azure SQL/PostgreSQL, and GCP Cloud SQL, despite sharing core engines, differ in extensions, limitations, backup procedures, and failover methodologies. Consequently, storage and data layers frequently necessitate re-architecting instead of direct replication.
• Identity and access: IAM (Identity and Access Management) is one of the least portable components across cloud environments. AWS employs policy-driven roles and users; Azure connects permissions to subscriptions and role assignments; and GCP enforces a hierarchical IAM structure with service accounts at various tiers. Therefore, access models and automation workflows rarely translate directly and require re-evaluation for each specific cloud.

Since leading cloud migration tools do not primarily focus on infrastructure, they are inherently unequipped to perform these complex translations. As previously discussed, even though infrastructure-as-code solutions appear to be ideal cloud-agnostic tools for overcoming interoperability gaps, IaC does not fully resolve this portability issue. Notably, hyperscaler-native IaC such as CloudFormation (AWS) and Bicep (Azure) were developed with specific cloud environments in mind. Even theoretically cloud-agnostic options like Terraform require bespoke configurations for each provider. Thus, IaC that seems cloud-neutral often becomes highly cloud-specific in practice.

As IT professionals are well aware, the conventional approach to this translation process is arduous. It entails meticulously reverse-engineering infrastructure from the source cloud to the target cloud, often necessitating collaboration between separate experts for each cloud to effectively map out the infrastructures.

Cloud Cloning addresses this translation challenge by leveraging its patented cloud mapping technology to comprehensively convert configurations from one cloud provider to another. For instance, it can take an operational, cloud-native AWS infrastructure—comprising services like EC2 instances, VPCs, subnets, security groups, Kubernetes, IAM, or databases—and transform it into an equivalent environment within Azure or GCP. In every scenario, the outcome is workloads that maintain their essential functionality while seamlessly conforming to the target cloud’s specific APIs, semantics, and operational guidelines. This enables truly multicloud applications without incurring additional engineering costs.

Crucially, Cloud Cloning automatically reverse-engineers Infrastructure as Code (IaC) from the live environment to the destination cloud, thereby removing the necessity for manual remediation or other rework. Considering that every VM destination demands its own intricate and frequently obscure network setups and configurations, this level of automation is highly beneficial. By default, Cloud Cloning provides these outcomes as Terraform configurations.

Tackling Governance Requirements: FinOps and Configuration Drift

A significant limitation of current multicloud tools lies in governance, particularly concerning FinOps and drift management. Let’s delve into how Cloud Cloning addresses these specific challenges, beginning with FinOps.

Cloud Cloning’s Approach to FinOps Management

Currently, cloud FinOps operates along two distinct pathways: optimizing within a single cloud environment and managing migrations between different clouds. Neither approach effectively supports comprehensive multicloud financial operations.

For individual clouds, FinOps largely relies on cloud-native tools like AWS Cost Explorer and Compute Optimizer, Azure Cost Management and Advisor, and Google Cloud Billing and Recommender. While these tools excel at optimizing expenses within their respective platforms, they are not designed to suggest what could be the most significant cost optimization: migrating to a different cloud. This oversight leads to substantial financial implications. Our observations indicate that IT teams can achieve considerable savings—potentially up to 50%—by replicating their configurations in alternative cloud providers or even different regions.

When IT teams choose to migrate, the FinOps effort is predominantly handled by the target cloud’s native tools and support staff. Using recent billing data from potential clients, the cloud provider offers a general estimate of the cost for a comparable setup in the proposed target environment. Theoretically, this information empowers customers to make decisions and transition to an alternative setup. However, given the intricate complexities previously described, simply knowing the potential cost of a new setup is insufficient. Teams require a precise understanding of how to translate their specific architectures into exact equivalents for the target environment. This detailed information is typically not provided by cloud providers, nor can it be adequately derived from generalized billing data.

For complete cloud cost optimization, IT teams require a detailed, actionable, and cloud-agnostic perspective on which cloud environments and configurations offer the most favorable pricing for existing functionalities. Fortunately, Cloud Cloning delivers this all-encompassing view. Employing the same translation methodologies previously outlined, Cloud Cloning facilitates accurate comparisons of functionality and pricing across various clouds and setups. Furthermore, it generates the Terraform code necessary for teams to automatically deploy their chosen new cloud configurations.

Essentially, Cloud Cloning transforms the fragmented and often ambiguous landscape of cross-cloud FinOps into the streamlined, ‘shop and click’ comparison experience it should ideally be.

How Cloud Cloning Manages Configuration Drift

Cloud deployments and migrations involve an immense number of variables. With a bewildering array of dependencies, tools, and architectural elements involved, even a seemingly simple deployment can escalate to complexities far exceeding human management capabilities, where each minor detail can subtly yet critically deviate. Considering the absence of tools capable of tracking all multi-tenancy changes within a single cloud—much less across multiple cloud providers—monitoring these changes becomes an insurmountable task. Despite rigorous DevOps hygiene and adherence to best practices, multicloud initiatives frequently encounter configuration drift that remains entirely undetected until a critical failure occurs. 

Infrastructure as Code (IaC) solutions like Terraform also fail to resolve the drift issue. Terraform operates as intended only when teams adhere to a ‘Terraform-first’ methodology—establishing Terraform as the primary source of truth from inception, diligently updating and monitoring configuration files, and ensuring the state file precisely mirrors the actual environment. Should teams introduce changes externally to Terraform or allow files and state to become unsynchronized, Terraform loses its ability to reliably manage or foresee infrastructure behavior. Again, considering the inherent complexity, this approach remains prone to drift.

Cloud Cloning addresses the drift challenge using the extensive infrastructure snapshots detailed earlier. It captures regular snapshots of the entire infrastructure based on a customizable schedule (24 hours by default). Subsequently, it compares these captured states against current configurations within a comprehensive infrastructure changelog, identifying and alerting on changes that could pose issues. This includes not only typical drift related to inventory but also deviations from established cost parameters and security controls.

From Infrastructure as Code to cloud-native migration tools and governance solutions, cloud portability has historically been plagued by significant gaps and excessive manual effort, culminating in a widespread vendor lock-in dilemma. As numerous organizations strive to rapidly diversify their cloud portfolios, there is a clear demand for a superior solution for cloud infrastructure migration and re-architecting. We are confident that Cloud Cloning provides this much-needed solution.

—

The New Tech Forum offers a platform where technology leaders—including providers and independent contributors—can delve into and discuss innovative enterprise technologies with unparalleled insight and scope. Our selection criteria are subjective, focusing on technologies we deem significant and highly relevant to InfoWorld’s audience. InfoWorld strictly declines promotional materials for publication and retains the authority to edit all submitted content. For all questions, please reach out to doug_dineley@foundryco.com.