Edgeless Methods has launched Continuum AI, a pioneering generative AI (GenAI) framework designed to keep up encrypted prompts always by way of confidential computing. This progressive answer integrates confidential digital machines (VMs) with NVIDIA H100 GPUs and safe sandboxing, in response to the NVIDIA Technical Weblog.
Making certain Knowledge Privateness and Safety
The launch of Continuum AI marks a major development in AI deployment, enabling companies to leverage highly effective massive language fashions (LLMs) with out compromising knowledge privateness and safety. By collaborating with NVIDIA, Edgeless Methods goals to empower organizations throughout numerous sectors to combine AI securely. This platform is not only a technological breakthrough but additionally a vital step in the direction of a future the place AI could be utilized securely, even with probably the most delicate knowledge.
Safety Targets of Continuum AI
Continuum AI has two major safety goals: to guard consumer knowledge and to safeguard AI mannequin weights towards infrastructure and repair suppliers. Infrastructure encompasses all of the underlying {hardware} and software program stacks that an AI utility runs on, similar to cloud platforms like Microsoft Azure. Service suppliers management the AI purposes, similar to OpenAI for ChatGPT.
How Continuum AI Operates
Continuum AI depends on two core mechanisms: confidential computing and superior sandboxing. Confidential computing is a hardware-based expertise that ensures knowledge stays encrypted even throughout processing, verifying the integrity of workloads. This method, powered by NVIDIA H100 Tensor Core GPUs, creates a safe setting that separates infrastructure and repair suppliers from knowledge and fashions. It additionally helps fashionable AI inference companies just like the NVIDIA Triton Inference Server.
Regardless of these safety measures, AI code from third events might doubtlessly leak prompts by accident or maliciously. An intensive assessment of AI code is impractical as a consequence of its complexity and frequent updates. Continuum addresses this by operating AI code inside a sandbox on a confidential computing-protected AI employee, utilizing an tailored model of Google’s gVisor sandbox. This ensures that AI code can solely deal with encrypted prompts and responses, stopping plaintext knowledge leaks.
System Structure
Continuum AI consists of two fundamental elements: the server facet, which hosts the AI service and processes prompts securely, and the consumer facet, which encrypts prompts and verifies the server. The server-side structure contains employee nodes and an attestation service.
Employee nodes, central to the backend, host AI fashions and serve inference requests. Every employee operates inside a confidential VM (CVM) operating Continuum OS, a minimal and verifiable system by way of distant attestation. The CVM hosts workloads in a sandbox and mediates community visitors by way of an encryption proxy, guaranteeing safe knowledge dealing with.
The attestation service ensures the integrity and authenticity of employee nodes, permitting each service suppliers and purchasers to confirm that they’re interacting with a safe deployment. The service runs in a CVM and manages key exchanges for immediate encryption.
Workflow and Person Interplay
Admins confirm the attestation service’s integrity by way of the CLI and configure AI code utilizing the employee API. Verified staff obtain inference secrets and techniques and might serve requests securely. Customers work together with the attestation service and employee nodes, verifying deployments and sending encrypted prompts for processing. The encryption proxy decrypts these prompts, processes them within the sandbox, and re-encrypts responses earlier than sending them again to customers.
For additional insights into this cutting-edge expertise, go to the Continuum web page.
Picture supply: Shutterstock
