Edge AI infrastructure, placed honestly

Definition

Two different things are called “edge AI”

The first is device-level: a compact model running on a camera, a controller, or a handset, measured in watts. The second is facility-level: racks of accelerators placed at a regional site, measured in hundreds of kilowatts or megawatts. Most search results describe the first. This page is about the second — the tier where siting, power, and cooling decisions get made.

The reason the second tier exists is that compute has concentrated. US data centres consumed roughly 4.4% of national electricity in 2023, on a trajectory the federal lab's modelling puts at 6.7–12% by 2028.^[1] The IEA describes the same concentration globally, with data-centre demand rising sharply through 2030.^[2] Concentration is efficient, but it puts compute a long way from most of the places data is created. Edge placement is the deliberate, expensive exception.

Three drivers that genuinely move placement

Latency. Distance sets a floor on round-trip time that no amount of bandwidth removes. If a control loop, a robotic cell, or an interactive assistant has a deadline the round trip cannot meet, no software fix closes the gap — only geography does. The honest test is whether there is a real deadline. Many workloads called latency-sensitive are just sensitive to being slow, and those are throughput problems.

Data gravity. High-rate sensor, imaging, and video streams are usually cheaper to process where they land than to ship. When the volume produced per hour exceeds what the uplink can move in an hour, the decision is already made: compute goes to the data.

Jurisdiction and control. Some data cannot leave a country, a campus, or a classified boundary. That constraint is categorical rather than economic — it does not respond to a better price per hour, and it is the one driver that survives every round of cost-cutting.

Notably absent from that list: cost. Edge sites are usually more expensive per unit of compute, because they lose the pooling that makes central capacity cheap. Anyone selling edge placement as a savings story should be asked for the utilization assumption underneath it.

EA-01 · Power

Where edge projects most often stall

Power is where edge projects most often stall. Where firm capacity is short, on-site storage can make a site viable by shaping the load rather than raising the connection: a federal-lab demonstration of a 70 MW data centre run as a flexible grid asset dispatched 35 MW of battery storage in under five seconds with service levels intact.^[12] The technique scales down, but it turns a passive electrical service into an operated system. The power architecture page covers the distribution side in detail.

Operations

Maintenance reality: repair time is travel time

That inversion should reshape the design: redundancy that buys days rather than minutes, components that fail predictably on a schedule rather than rarely and catastrophically, and a factory-commissioned thermal system rather than one that needs field balancing. A sealed thermal enclosure is worth more at a dusty, remote, or coastal site than it is in a clean hall, and the enclosure rating conversation — IP code versus NEMA type — is a real engineering decision rather than a paperwork one.^[9]

Plan the first year as a maintenance schedule, not an incident queue: consumable intervals, firmware and certificate expiry, coolant chemistry sampling, and one rehearsed full-site restart. Then check the plan against the site survey using the readiness checklist.

The telemetry is the only instrument

Telemetry standardisation matters more here than in a staffed facility, because the data stream is the operator's only instrument.^[10] Access controls matter more because the deterrent of an occupied building is gone.^[11]And the industry's own numbers argue for humility: in a survey of more than 800 operators, roughly half reported an impactful outage within the previous three years.^[4] Those failures happen in staffed buildings. See monitoring and controls for the instrumentation layer.

In the product

A split, and where the PODOS Pod fits an edge deployment

The realistic answer for most organisations is a split: central capacity for training and batch work, edge capacity only for the inference paths that fail without it. That tradeoff is laid out in on-prem AI infrastructure vs cloud.

PODOS builds the facility layer as a factory-integrated unit rather than a field-built room. Each PODOS Pod is designed as a standardized 1 MW building block and designed for 128 GPUs, with power distribution, closed-loop liquid cooling, racks, and instrumentation specified together and commissioned before the unit leaves the factory. That matters at an edge site for one reason: the parts hardest to build correctly with a small local crew arrive already built and tested.

That approach shortens the on-site sequence to civil works, service connection, and commissioning, which is why PODOS targets a 90-day window from order to commissioning for a standard unit — a target, not a measured result. The deployment model describes that sequence; other placements for the same unit are collected under use cases, and unfamiliar terms are defined in the AI infrastructure glossary.