AI infrastructure investment has to solve two problems together: bringing capacity online and allocating it to work it can perform well. A shortage of power or accelerators cannot be scheduled away. Equally, additional equipment may deliver little benefit when networking, recovery, placement, or operational constraints prevent the existing fleet from serving demand.
The portfolio approach connects those decisions. It asks which workloads the organization needs to support, which combinations of sites and systems can support them, and which investment improves useful output under the constraints that actually bind.
The workload hierarchy below is a planning framework, not a ranking of business importance. Its purpose is to make differences in deadlines, resource needs, and placement constraints visible before capital is committed.
Define useful output before optimizing utilization
Utilization remains useful for diagnosing activity. It does not, by itself, establish whether that activity met the service objective. Measure completion and quality alongside it.
For training, useful output might mean progress toward a stated evaluation target within a time and cost budget. For inference, it might mean accepted requests completed within quality and latency requirements. Retries, failed runs, queueing, and recovery should remain visible in the accounting rather than disappearing into an aggregate throughput number.
DistServe offers a concrete inference example: it evaluates request-serving capacity subject to both time-to-first-token and time-per-output-token objectives. That definition makes the service requirement part of the capacity measure. It should not be generalized into a universal goodput percentage across different workloads.
The management question is how much of a proposed investment will improve the agreed outcome. That requires a baseline, a clear denominator, and an explanation of what the measurement excludes.
Classify workloads by their constraints
| Workload class | What the capacity plan must establish | Possible flexibility to test |
|---|---|---|
| Large training runs | Model and optimizer memory, parallelism, fabric performance, checkpointing, recovery, and time to target | Start-time or placement flexibility where data, topology, and recovery costs permit |
| Fine-tuning and distillation | Data access, memory fit, experiment cadence, evaluation requirements, and deadlines | Smaller allocation shapes or scheduling windows, subject to the actual method |
| Interactive inference | Accepted-task quality, latency distribution, concurrency, memory, and proximity to users and data | Routing among eligible sites or models that satisfy the same service requirements |
| Batch inference | Completion deadline, quality, data movement, and restart behavior | Off-peak execution, interruption, or relocation where their costs fit the deadline |
| Synthetic data and evaluation | Quality controls, provenance, reproducibility, storage, and dependencies on other work | Flexible scheduling only where the pipeline can pause or restart safely |
These are starting questions. Training geography is constrained by data access, legal requirements, network topology, suitable capacity, and the cost of moving state. Interactive inference does not always require an edge site. Batch work is not automatically interruption-safe. Establish the properties of the actual job before assigning flexibility to it.
A shared fleet can benefit from complementary demand, but only when resources, software, and service requirements allow sharing. Do not assume a universal day–night pattern or that a long training run fits into every gap between inference peaks.
Make the physical capacity boundary explicit
The IEA's Energy and AI report identifies grid constraints and equipment lead times as risks to data-center expansion. Its four-to-eight-year figure concerns building new transmission lines in advanced economies; it is not a universal data-center interconnection waiting time. Individual projects need utility-specific schedules and commissioning evidence.
Within a site, distinguish the grid connection, facility power, IT power, rack density, and the equipment that can be supported through a fault. Improving one layer does not automatically release usable capacity at the next.
The DIMAGGI cooling model makes density, power-usage effectiveness, and cooling-fault ride-through explicit. Its capacity calculations describe modeled IT-power headroom behind a feed. Converting that headroom into service also requires equipment, distribution, cooling compatibility, and an operable deployment. A lower PUE alone does not establish more accepted tasks per dollar.
This is why facilities, power, network, compute, and operations should review the same demand scenarios. Their individual capacity measures must connect to one commissioned system.
Use reproducible models to compare interventions
The Usable Compute research portfolio provides public models for testing parts of this decision. The models expose assumptions and limits; they are not a claim that DIMAGGI has achieved the same results in a customer's production environment.
| Investment question | Research artifact and its role |
|---|---|
| Should capital go to networking, recovery, or more accelerators? | Network vs. More GPUs compares these interventions through a capacity-accounting framework and decision maps for training. |
| Can allocation policy make the existing fleet more usable? | Scheduler vs. More GPUs examines capacity accounting and allocation policies with a simulator and control-plane cases. |
| Which recovery policy fits the failure regime? | Reliability Economics compares checkpointing, spares, and elastic recovery through a reproducible simulator. |
| Is moving a job worth its disruption? | Compute–Power Placement compares move, stay, and capacity-related options while accounting for migration cost and destination constraints. |
| Does nominal capacity have an allocatable shape? | Slice Packer Torus examines accelerator-pod allocation, faults, isolation, and topology. |
| Does an edge location satisfy a real requirement? | Edge Continuum Placement evaluates fabric, power, latency, and data-gravity constraints across an operator footprint. |
Run the relevant model with local inputs, retain its repository commit and configuration, and record the conditions under which the preferred option changes. Keep simulated findings distinct from calibration against published data and from measurements on the proposed deployment.
Do not multiply headline gains from separate models into a portfolio return. Their constraints can overlap, their baselines can differ, and one intervention can change the value of another. The combined investment case needs a common workload and accounting boundary.
Right-size models without assuming a fleet ratio
A smaller or specialized model may be appropriate for a workload if it meets the required quality, safety, latency, and operating criteria. That has to be demonstrated against representative tasks, including difficult cases and distribution changes.
There is no general percentage of traffic that should use a smaller model, no universal cost reduction, and no direct conversion from a routing share into the same share of hardware capacity. Different requests can consume very different amounts of time, memory, and energy. Escalation to another model can add work instead of replacing it.
Evaluate routing as a system: first-pass quality, escalation rate, repeated work, latency, and total cost per accepted task. Then translate the measured demand into fleet requirements. That sequence connects model selection to capital planning without assuming the economic result in advance.
Treat placement as a constrained choice
Build a set of eligible locations for each workload before comparing prices. Include data permissions, operational authority, service availability, latency, fabric, recovery, and destination capacity. The legal-identity framework provides a companion set of questions about jurisdiction and control.
Geographic diversity can improve resilience or market access, but it can also fragment capacity and increase operating cost. A smaller number of sites may be appropriate where the service requirements allow it. Choose the portfolio from the demand and risk model, rather than treating either centralization or distribution as a universal objective.
For movable work, compare the full cost of staying with the full cost of moving. Include state transfer, interruption, restart, lost progress, and the capacity that must be available at the destination. A lower electricity price alone does not settle that comparison.
Turn the portfolio into an operating decision
I would bring a single decision record to the investment committee: workload demand, eligible sites and configurations, the current limiting constraints, options to remove them, and useful output expected under each scenario.
That record should show the baseline, downside case, dependency owners, evidence gaps, and the next commitment that can be staged. Its operating counterpart should track whether the commissioned system delivers the accepted tasks or training milestones the case promised.
This keeps the planning loop intact. When demand, power availability, model behavior, or failure rates change, the organization can revisit the allocation and the next investment using the same evidence structure.
The aim is enough capacity, in the right form and location, operated so that it delivers the required work. Portfolio discipline makes the connection between those choices explicit.
Originally published 10 April 2026. Revised 13 September 2026 to remove unsupported operating anecdotes and universal cost/routing ratios, qualify placement claims, and connect the framework to the public Usable Compute research models.