{"id":79471,"date":"2026-10-22T15:42:17","date_gmt":"2026-10-22T15:42:17","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=79471"},"modified":"2026-10-22T15:42:19","modified_gmt":"2026-10-22T15:42:19","slug":"top-7-bare-metal-ai-infrastructure-providers","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/top-7-bare-metal-ai-infrastructure-providers\/","title":{"rendered":"Top 7 Bare-Metal AI Infrastructure Providers"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Bare-metal AI infrastructure providers give teams dedicated physical compute, but the buying experience varies considerably: some supply individual GPU servers that your engineers operate, while others take responsibility for an entire rack-scale deployment, including power, cooling, networking and workload planning. Comparing them requires more than a GPU price list.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Uptime Institute&#8217;s 2026 Global Data Center Survey found that more data-centre operators were reporting peak rack densities of 30 kW or higher, while power availability and capacity forecasting remained growing concerns. For AI infrastructure buyers, those findings reinforce the need to evaluate the facility, cooling and operating model alongside the GPUs themselves.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This comparison covers seven providers with dedicated physical GPU offerings, ranging from full-rack NVIDIA deployments to configurable servers for inference, fine-tuning and scientific computing. The useful shortlist depends on workload size, interconnect requirements, operating responsibilities and the amount of capacity you can keep productively occupied.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Bare-metal AI infrastructure providers compared<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td><strong>Provider<\/strong><\/td><td><strong>Strongest buyer fit<\/strong><\/td><td><strong>Relevant dedicated offering<\/strong><\/td><td><strong>What to verify before buying<\/strong><\/td><\/tr><tr><td>CambridgeNexus<\/td><td>Teams procuring operated, full-rack AI infrastructure<\/td><td>Bare-metal NVIDIA GB300 NVL72 rack leasing<\/td><td>Workload sizing, installation site, acceptance criteria and operating scope<\/td><\/tr><tr><td>Latitude.sh<\/td><td>Engineering teams that want programmable GPU infrastructure<\/td><td>Metal GPU servers and clusters, with configurations including H100 and B300<\/td><td>Hardware availability, regional pricing and the networking attached to the selected configuration<\/td><\/tr><tr><td>Scaleway<\/td><td>European deployments needing dedicated GPU servers alongside supporting services<\/td><td>Elastic Metal Titanium servers with dual L40S or dual H100 GPUs<\/td><td>Paris-zone availability, private networking and server-level capacity limits<\/td><\/tr><tr><td>OVHcloud<\/td><td>Teams building dedicated inference and AI services on established server infrastructure<\/td><td>GPU dedicated server ranges including L4 and L40S configurations<\/td><td>Orderability, setup fees, configuration and regional availability<\/td><\/tr><tr><td>Gcore<\/td><td>Teams evaluating dedicated multi-server GPU infrastructure<\/td><td>Bare Metal GPU clusters, with NVIDIA hardware and configuration-dependent interconnects<\/td><td>Region, GPU model, cluster fabric and commercial terms<\/td><\/tr><tr><td>phoenixNAP<\/td><td>Teams prepared to deploy on Intel accelerators<\/td><td>Bare Metal Cloud servers with dual Intel Max 1100 GPUs<\/td><td>Software compatibility, GPU-region availability and reservation terms<\/td><\/tr><tr><td>Cherry Servers<\/td><td>Teams needing configurable dedicated AI servers<\/td><td>Custom physical servers with optional NVIDIA accelerators<\/td><td>Complete server-plus-GPU cost, accelerator availability and deployment lead time<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">These are different procurement models, rather than interchangeable packages. A dedicated inference server may be appropriate for one application, while a full-rack deployment makes more sense for a programme that needs a larger interconnected GPU system and coordinated infrastructure operations.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How I evaluated the providers<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I compared official product pages, configuration information, documentation and published pricing where available. This is a comparison of documented infrastructure offerings and buyer fit; it does not report hands-on performance tests, account trials or independently measured uptime.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Five questions shaped the assessment:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What is physically dedicated?<\/strong> A GPU, a complete server and an entire rack represent different isolation and capacity boundaries.<\/li>\n\n\n\n<li><strong>Can the configuration support the workload?<\/strong> GPU memory, CPU architecture, storage and interconnects affect which models and training strategies are practical.<\/li>\n\n\n\n<li><strong>Who operates each layer?<\/strong> Hardware support does not automatically include operating-system maintenance, scheduling, model serving or application troubleshooting.<\/li>\n\n\n\n<li><strong>How does capacity become usable?<\/strong> Regional inventory, provisioning, data transfer and acceptance testing determine when production work can begin.<\/li>\n\n\n\n<li><strong>What does the commitment actually cost?<\/strong> Server charges, networking, storage, support and idle capacity matter alongside accelerator pricing.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The recommendations below map providers to those requirements. They do not assume that removing a hypervisor automatically makes every application faster or every deployment cheaper.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>1. CambridgeNexus: Best for operated full-rack AI deployments<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">CambridgeNexus is a Boston-based <a href=\"https:\/\/cambridgenexus.com\/\">AI Factory<\/a> operator that owns and operates bare-metal NVIDIA GB300 NVL72 racks and leases them to individual customers. Its full-rack offering suits organisations procuring dedicated production capacity with the surrounding infrastructure operated together.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The distinction is the operating model: CambridgeNexus combines power, cooling, networking, compute, orchestration, compliance and workload planning. That scope is relevant when an AI team needs responsibility for infrastructure coordination to sit with an operator rather than across several separately contracted suppliers.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its GB300 rack offering is aimed at substantial training and inference workloads, including AI video generation and enterprise research. The company describes site selection around workload, latency and compliance requirements, with deployments in the United States and Taiwan and lease terms from six months to five years or longer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For procurement, the useful discussion starts with the intended workload and required rack capacity. Buyers should establish the installation site, workload environment, support responsibilities and acceptance process before comparing the proposal with a self-managed server contract.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pricing:<\/strong> Contract-specific quotation. Compare the complete operating scope, rather than trying to infer a rack price from an unrelated GPU-hour rate.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What I like about CambridgeNexus<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Full-rack, single-tenant capacity gives buyers a clear physical allocation for sustained AI programmes.<\/li>\n\n\n\n<li>The seven-layer operating model brings power, cooling and workload infrastructure into the same supplier conversation.<\/li>\n\n\n\n<li>Site selection considers workload and residency requirements before deployment is fixed in the contract.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Where CambridgeNexus falls short<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Its full-rack GB300 offering is too large for teams whose requirement is only a small amount of compute.<\/li>\n\n\n\n<li>Bespoke proposals require a defined workload and capacity plan before buyers can make a meaningful cost comparison.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>2. Latitude.sh: Best for programmable dedicated GPU infrastructure<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"546\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2026\/10\/image-21-1024x546.png\" alt=\"\" class=\"wp-image-79473\" style=\"aspect-ratio:1.8795180722891567;width:624px;height:auto\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2026\/10\/image-21-1024x546.png 1024w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2026\/10\/image-21-300x160.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2026\/10\/image-21-768x409.png 768w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2026\/10\/image-21.png 1312w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Latitude.sh combines dedicated GPU hardware with an API-oriented provisioning model, making it relevant to engineering teams that want physical infrastructure to fit into repeatable deployment workflows rather than treating every server as a manually configured project.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its Metal GPU catalogue includes configurations ranging from a single NVIDIA H100 to larger systems with NVIDIA B300 or RTX PRO 6000 accelerators. That breadth supports different procurement sizes, although the exact hardware, location and networking must be checked together for the selected plan.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The developer tooling is a practical differentiator: Latitude.sh documents API access and Terraform integration, which can help platform teams version provisioning changes and reproduce environments. This matters when GPU resources need to join an existing engineering process for networking, access and software configuration.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The main trade-off is that catalogue breadth does not make every configuration equivalent. A single-GPU server and an eight-GPU system have different memory and communication characteristics, and a suitable training configuration still needs validation with the actual framework and workload.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pricing:<\/strong> Public configuration-based rates are available. Compare the complete server specification, billing period and traffic allowance before calculating an effective cost per GPU.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What I like about Latitude.sh<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>API and Terraform support help teams integrate physical server provisioning with infrastructure-as-code workflows.<\/li>\n\n\n\n<li>Multiple GPU server sizes provide options beyond an initial full-rack commitment.<\/li>\n\n\n\n<li>Published hardware configurations make it easier to compare GPU memory, local storage and network interfaces together.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Where Latitude.sh falls short<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Buyers must verify GPU availability for the intended location rather than assuming the entire catalogue is available everywhere.<\/li>\n\n\n\n<li>A published server configuration does not establish that its interconnect is suitable for a larger distributed training job.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>3. Scaleway: Best for dedicated GPU servers within a European infrastructure stack<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Scaleway&#8217;s Elastic Metal Titanium range offers dedicated physical GPU servers with supporting services in the broader Scaleway ecosystem. It is a useful candidate for teams that want GPU hardware in Europe while keeping associated networking, storage and application infrastructure within the same provider environment.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The published Titanium configurations include dual NVIDIA L40S servers and dual NVIDIA H100 servers, with NVMe storage and private networking. These are identifiable server products, which makes the starting architecture easier to assess than a general claim that a provider offers AI compute.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The distinction within Scaleway&#8217;s portfolio matters: Elastic Metal Titanium is the dedicated server range considered here. Its separate GPU instances and cluster offerings should be evaluated on their own terms rather than assumed to have the same architecture, price or operational boundary.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Titanium is particularly relevant when the workload fits within its available server configurations and Paris deployment location. Teams planning tightly coupled training across many servers should confirm the fabric design and scaling path instead of extrapolating from the performance of one machine.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pricing:<\/strong> Published hourly and monthly server rates. Check taxes, commitment fees, additional services and the selected availability zone when comparing the final cost.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What I like about Scaleway<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Clearly specified dual-GPU configurations support a concrete assessment of memory, storage and compute requirements.<\/li>\n\n\n\n<li>Dedicated servers can be evaluated alongside Scaleway&#8217;s private networking and supporting services.<\/li>\n\n\n\n<li>Public server pricing helps buyers build an initial budget before a detailed architecture discussion.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Where Scaleway falls short<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Titanium&#8217;s published availability is concentrated in Paris PAR2, which may not match the required location.<\/li>\n\n\n\n<li>Its dual-GPU servers should not be treated as substitutes for a rack-scale system without checking workload distribution and networking.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>4. OVHcloud: Best for dedicated inference infrastructure and configurable GPU servers<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">OVHcloud offers physical GPU dedicated servers alongside its separate virtualised compute and managed AI services. The dedicated server ranges are relevant to teams that want to run their own AI applications on identifiable hardware with configurable memory, storage and networking.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The GPU dedicated catalogue includes NVIDIA L4 configurations and HGR-AI systems using NVIDIA L40S accelerators. That makes it worth considering for inference, model-serving applications and workloads whose requirements fit those accelerator classes, rather than assuming that every AI deployment needs the largest available training platform.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Buyers should keep the product families separate during procurement. An H100 appearing elsewhere in OVHcloud&#8217;s portfolio does not establish that the same hardware is included in a bare-metal quotation; the order should identify the actual server, GPU count and accelerator model.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is also a configuration-led purchase where live availability matters. Some catalogue entries may show future availability, and setup charges can materially change the economics of a short deployment, even when the recurring monthly fee initially looks attractive.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pricing:<\/strong> Public server prices and configuration-dependent quotes. Check installation fees, tax treatment, commitment conditions and whether the selected server can be ordered in the required region.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What I like about OVHcloud<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Dedicated L4 and L40S server options give inference teams alternatives to procuring a larger rack-scale system.<\/li>\n\n\n\n<li>Published configuration details allow CPU, RAM, storage and network requirements to be assessed together.<\/li>\n\n\n\n<li>Separate public and private bandwidth specifications help clarify different traffic requirements.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Where OVHcloud falls short<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Product catalogue visibility does not guarantee immediate capacity; orderability must be confirmed.<\/li>\n\n\n\n<li>Setup fees and regional differences can make headline monthly prices an incomplete comparison.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>5. Gcore: Best for teams evaluating dedicated GPU clusters<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"555\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2026\/10\/image-21-1024x555.png\" alt=\"\" class=\"wp-image-79472\" style=\"aspect-ratio:1.8407079646017699;width:624px;height:auto\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2026\/10\/image-21-1024x555.png 1024w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2026\/10\/image-21-300x163.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2026\/10\/image-21-768x416.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Gcore offers Bare Metal GPU clusters as part of a wider infrastructure portfolio, with documentation distinguishing dedicated physical GPU servers from virtual GPU services. It is relevant when buyers are evaluating multiple-server AI capacity and want a provider with explicit cluster products.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its published GPU range includes NVIDIA H100 and H200 configurations, while networking options vary with the hardware. InfiniBand support is an important reason to investigate the offering for distributed workloads, but buyers should verify the fabric attached to the proposed configuration rather than relying on a portfolio-wide specification.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The cluster procurement discussion should cover physical placement, server count, storage access and GPU communication. A deployment can have adequate aggregate compute but still underperform if training processes repeatedly wait for data or synchronisation across a poorly matched network.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Gcore also offers different resource types and purchasing options, making precise service identification important. The proposal should state whether capacity is dedicated, its interruption conditions and which team owns software configuration and workload operations after the physical servers are provisioned.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pricing:<\/strong> Hardware-, region- and service-dependent. Request a matched cluster proposal covering compute, networking, storage, support and commitment conditions.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What I like about Gcore<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Explicit Bare Metal GPU cluster products make it a relevant candidate for multi-server AI deployments.<\/li>\n\n\n\n<li>NVIDIA accelerator options and configuration-dependent InfiniBand support allow a more detailed discussion of distributed workload requirements.<\/li>\n\n\n\n<li>Its documentation separates physical GPU resources from virtual services, helping buyers specify the intended deployment model.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Where Gcore falls short<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>GPU models and service availability vary by region, so a suitable configuration requires regional confirmation.<\/li>\n\n\n\n<li>A portfolio-level networking claim does not establish the topology or measured performance of the cluster being purchased.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>6. phoenixNAP: Best for Intel-compatible AI and HPC workloads<\/strong><strong><\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">phoenixNAP&#8217;s Bare Metal Cloud GPU offering uses dedicated servers with dual Intel Max 1100 accelerators. It provides a materially different option for teams prepared to evaluate an Intel software stack, rather than assuming every bare-metal AI procurement must use NVIDIA hardware.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The offering combines physical GPU servers with infrastructure automation, including integrations such as Terraform and Ansible. Local NVMe storage and additional network storage options make it relevant to workloads where repeatable provisioning and data access matter alongside accelerator selection.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The deciding factor is software compatibility. Teams with CUDA-specific extensions, tightly pinned dependencies or an established NVIDIA optimisation workflow should validate their application on the proposed Intel environment before treating a server-price comparison as a purchasing decision.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This option is most useful when the team can evaluate the application as a complete system: model support, precision, kernel availability, data throughput and operational effort. A different accelerator architecture is worth considering when the workload can use it effectively and the engineering work is understood.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pricing:<\/strong> Published server and reservation options. Confirm the GPU-specific purchase terms and location, because general Bare Metal Cloud availability is broader than the locations offering these GPU servers.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What I like about phoenixNAP<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Dedicated Intel accelerator servers provide an alternative for workloads that are compatible with that software ecosystem.<\/li>\n\n\n\n<li>Infrastructure automation integrations support repeatable provisioning and server-management workflows.<\/li>\n\n\n\n<li>Local NVMe and additional storage choices allow buyers to assess data movement alongside compute.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Where phoenixNAP falls short<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>CUDA-dependent applications require compatibility work or changes before this accelerator offering becomes a realistic substitute.<\/li>\n\n\n\n<li>The GPU servers&#8217; documented location coverage is narrower than the overall platform footprint.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>7. Cherry Servers: Best for custom server configurations and smaller dedicated deployments<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Cherry Servers offers configurable physical AI servers with NVIDIA accelerator options, allowing buyers to choose CPU, memory, storage and GPU resources around an application rather than committing immediately to an entire rack-scale deployment.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its AI server catalogue lists accelerators including NVIDIA A2, A40 and A100, alongside AMD EPYC and Intel-based server options. This makes it relevant to teams whose bottleneck may involve preprocessing, host memory or local storage as well as GPU compute.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The value of custom configuration is clearest when a standard GPU server leaves important resources unbalanced. An application might need substantial system memory and fast local data access, while another needs a smaller accelerator allocation for continuous inference or a specialised internal service.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The main purchasing risk is comparing an incomplete configuration with a fully specified competitor proposal. Some server prices exclude the optional GPU, and the desired accelerator, region and deployment timing need confirmation together before the infrastructure can be budgeted or scheduled accurately.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pricing:<\/strong> Configuration-based. Request the complete server price with the chosen accelerator, storage, traffic allowance and any additional support requirements.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What I like about Cherry Servers<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Configurable CPU, RAM, storage and accelerator options let buyers address bottlenecks beyond GPU count.<\/li>\n\n\n\n<li>Server-level procurement can suit workloads that do not justify a full-rack commitment.<\/li>\n\n\n\n<li>Published accelerator options make it possible to start with a defined hardware requirement before requesting a complete quote.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Where Cherry Servers falls short<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Optional-GPU pricing means an advertised base server cost may not represent a usable AI configuration.<\/li>\n\n\n\n<li>A custom dedicated server should not be assumed to include the fabric or operating model needed for a large training cluster.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How to choose bare-metal AI infrastructure for your workload<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Start with model memory and the actual workload<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Define the model, numerical precision, context length, expected concurrency and whether the deployment is for training, fine-tuning or inference. These requirements determine memory and communication needs more reliably than a general preference for whichever GPU generation has the highest advertised peak performance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For inference, model weights are only part of the memory requirement: runtime allocations and the key-value cache also matter. For training, gradients, optimiser states and activations can change the capacity calculation substantially, depending on precision, parallelisation and memory-saving techniques.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Specify whether the application must fit on one accelerator, one server or a larger interconnected system. The memory advertised across several GPUs should not be treated as one automatically interchangeable allocation without checking how the framework distributes the workload.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Distinguish hardware isolation from operational responsibility<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Dedicated hardware establishes a physical allocation, but it does not determine who patches the operating system, manages credentials, updates drivers or repairs a failed distributed training run. Those responsibilities should be assigned explicitly, especially when the organisation lacks an infrastructure team able to own every layer.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td><strong>Layer<\/strong><\/td><td><strong>Question to settle before signing<\/strong><\/td><\/tr><tr><td>Physical infrastructure<\/td><td>Who owns power, cooling, hardware maintenance and replacement?<\/td><\/tr><tr><td>Host software<\/td><td>Who maintains firmware, operating systems, drivers and accelerator libraries?<\/td><\/tr><tr><td>Scheduling<\/td><td>Who configures and supports Slurm, Kubernetes or another scheduler?<\/td><\/tr><tr><td>Data services<\/td><td>Who operates shared storage, backups, encryption and recovery?<\/td><\/tr><tr><td>Applications<\/td><td>Who handles model-serving failures, framework changes and workload tuning?<\/td><\/tr><tr><td>Incidents<\/td><td>Which team receives the alert, and what response or restoration commitment applies?<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">An operated rack model and a self-managed server can both be appropriate, but the staffing assumptions must appear in the comparison. A cheaper hardware contract can require more internal engineering work than a proposal with a broader operating scope.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Evaluate GPU communication separately from internet connectivity<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Public bandwidth determines how data enters or leaves the environment, while training performance can depend on GPU-to-GPU communication within a server, across servers and across racks. Those are different paths with different technologies, limits and contention risks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For distributed NVIDIA workloads, request communication testing with a documented configuration. NVIDIA&#8217;s <a href=\"https:\/\/github.com\/NVIDIA\/nccl-tests\">NCCL test suite<\/a> provides collective communication benchmarks that can help validate the intended deployment, but the results should be assessed alongside the application rather than treated as a substitute for it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ask for the physical topology, interconnect type, network oversubscription assumptions and whether expansion capacity can join the same fabric. Adding more GPUs in a different location does not necessarily expand the original training system in a useful way.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Check storage against training and recovery behaviour<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Request measurements for the dataset access pattern the application will actually use, including sustained reads, concurrent workers and checkpoint writes. A large NVMe capacity figure does not establish the performance of a shared filesystem or the speed of restoring a job after interruption.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Also establish whether local disks are temporary working storage or the intended home for important data. Dataset copies, checkpoints, model artefacts and backups need a persistence and recovery plan that survives a server replacement or the end of the contract.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Compare total productive cost<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A useful commercial comparison includes compute, storage, networking, support, setup charges, migration work and internal operations. For a reserved deployment, unused capacity remains part of the cost, even when the infrastructure is functioning exactly as contracted.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Use this relationship when comparing proposals:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Effective cost per productive GPU-hour = total deployment cost \u00f7 productive GPU-hours.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, if a hypothetical eight-GPU deployment costs $20,000 over 720 hours, its provisioned capacity is 5,760 GPU-hours. At 50% productive utilisation, the effective cost is about $6.94 per productive GPU-hour, before any costs omitted from that $20,000 figure.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For serving applications, cost per successful request or per generated token at the required latency can be more useful. For training, compare the cost of reaching the same target with equivalent data, precision and output quality.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Turn security requirements into deployment requirements<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Ask where data is stored, who can access management interfaces, how administrative actions are logged and what happens to local storage when hardware is replaced or released. Certifications should be checked against the actual service, operating entity and installation site being proposed.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Dedicated hardware also does not automatically provide confidential computing. If the requirement is to protect data during computation, request the specific execution protections, supported hardware, attestation process and key-management design rather than relying on a general statement about isolation.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>What to test before accepting a deployment<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Acceptance should demonstrate that the contracted environment can run the intended work, rather than merely showing that the servers power on. Agree the test workload, software versions, observation period and pass conditions before provisioning so both sides understand when the deployment is ready.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td><strong>Test area<\/strong><\/td><td><strong>Evidence to request<\/strong><\/td><\/tr><tr><td>Hardware inventory<\/td><td>Contracted GPU model, memory, CPU architecture, RAM, disks and network interfaces<\/td><\/tr><tr><td>Software compatibility<\/td><td>Working containers, drivers and framework versions for the intended application<\/td><\/tr><tr><td>Communication<\/td><td>Collective communication results at the contracted server or rack count<\/td><\/tr><tr><td>Storage<\/td><td>Dataset ingestion, sustained reads, checkpoint writes and restore behaviour<\/td><\/tr><tr><td>Application performance<\/td><td>Training throughput or inference throughput and latency under representative load<\/td><\/tr><tr><td>Stability<\/td><td>An extended workload run with health metrics and an agreed error threshold<\/td><\/tr><tr><td>Operations<\/td><td>Monitoring access, incident escalation, maintenance procedures and support ownership<\/td><\/tr><tr><td>Commercial readiness<\/td><td>Confirmed billing start, acceptance record and treatment of unresolved defects<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">A smaller trial is useful for software compatibility, but it cannot establish the scaling behaviour of a larger deployment on its own. Where procurement involves a multi-server or rack-scale environment, include testing at the contracted scale or document how the remaining validation will occur.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Frequently asked questions<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Does bare metal mean the infrastructure must be installed on premises?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No. Bare metal describes access to physical computing resources without a provider hypervisor dividing the purchased server into virtual machines. The server can be hosted in an external facility, and the surrounding management tools can still be delivered through APIs or a web console.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Can a bare-metal deployment run Kubernetes or Slurm?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Yes, but the ability to install a scheduler is different from buying a supported scheduler service. Establish who deploys it, manages upgrades, configures GPU access and handles scheduling incidents, rather than assuming those responsibilities are included with the physical hardware.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Is a GB300 rack the same as a B300 server?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No. NVIDIA&#8217;s GB300 NVL72 is a rack-scale Grace Blackwell system combining Grace CPUs and Blackwell Ultra GPUs through a rack-scale NVLink architecture. A B300 server is a different procurement unit, so software compatibility, connectivity, physical requirements and operating scope should be checked separately.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>When is a longer lease preferable to hourly capacity?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A longer commitment becomes more attractive when demand is sustained, the workload is stable and the organisation can use the reserved capacity consistently. Hourly purchasing can be more appropriate when requirements are still changing, provided the required hardware and region remain available when needed.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>How should buyers plan for expansion?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Separate a guaranteed expansion commitment from a statement that more hardware may become available. Establish the additional capacity, location, networking requirements, commercial terms and lead time, particularly when new servers must join an existing distributed training system rather than operate independently.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Which provider should you shortlist?<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Start with CambridgeNexus when the requirement is operated, full-rack NVIDIA infrastructure with power, cooling, networking and workload planning coordinated together. Consider Latitude.sh and Gcore when programmable provisioning or dedicated GPU clusters are central to the engineering plan.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Scaleway and OVHcloud deserve consideration for dedicated server deployments with surrounding infrastructure services, while Cherry Servers fits buyers who need a more customised server configuration. Evaluate phoenixNAP when the application can use Intel accelerators and the team is prepared to validate the software stack.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Before committing, compare a matched configuration and run a representative workload through an agreed acceptance process. The most useful result is a system that meets the application&#8217;s throughput, operating and cost requirements at the capacity you are actually purchasing.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Bare-metal AI infrastructure providers give teams dedicated physical compute, but the buying experience varies considerably: some supply individual GPU servers that your engineers operate, while others take&#8230; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_joinchat":[],"footnotes":""},"categories":[11138],"tags":[],"class_list":["post-79471","post","type-post","status-publish","format-standard","hentry","category-best-tools"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/79471","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=79471"}],"version-history":[{"count":1,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/79471\/revisions"}],"predecessor-version":[{"id":79474,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/79471\/revisions\/79474"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=79471"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=79471"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=79471"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}