Starting with the operation itself
A multiply-accumulate (MAC) is the atomic unit of neural network inference. Every dot product, every attention score, every projection layer decomposes into a pile of MACs. In a transformer with 7 billion parameters running a single 512-token context, you are looking at somewhere around 7 trillion MACs per forward pass. The question we keep returning to in our Boulder lab is: what is the minimum physical energy required to perform one MAC?
In digital CMOS logic, a MAC involves switching transistors. Transistor switching charges and discharges a capacitor. The energy cost per switching event is approximately E = (1/2)CV2, where C is the node capacitance and V is the supply voltage. A modern GPU core operates around 0.7 to 0.9 V with node capacitances in the femtofarad range. That puts digital MAC energy somewhere in the range of a few to tens of femtojoules, depending on precision and microarchitecture. For 16-bit math across a whole GPU, system-level energy per MAC, including DRAM, interconnects, and control logic, can climb considerably higher than the arithmetic core alone.
The photonic approach replaces the switching transistor with optical interference. Two light beams can be made to interfere constructively or destructively in a waveguide coupler. The amplitude of the output encodes the product. The critical physical fact: photons traveling through a waveguide do not lose energy to resistive heating. Silicon waveguides at 1310 nm have propagation loss around 2 to 3 dB/cm in a well-fabricated rib waveguide geometry. For the path lengths inside a matrix-multiply chip, that is almost nothing compared to the switching energy a transistor dissipates on every clock cycle.
Where the energy actually goes in a photonic MAC
A photonic matrix multiply does not eliminate energy cost; it moves it. The energy bookkeeping has three entries: the laser source, the modulators, and the photodetectors.
The laser must run continuously to supply the optical carrier. A semiconductor laser with wall-plug efficiency around 30 to 40 percent uses energy proportional to its output power, divided across however many MACs you can pack into each photon you send through the chip. This is where parallelism pays off hard: if a single optical pulse traverses a 64-element MZI mesh and participates in 64 simultaneous multiply operations before being detected, the laser energy per MAC is 64x lower than if you processed elements one at a time.
The modulators are the next cost. Encoding an input activation or updating a weight requires an electro-optic modulator, typically a carrier-depletion or carrier-injection phase shifter in silicon. Silicon modulators have energy costs in the range of tens to hundreds of femtojoules per bit, depending on whether you are using ring resonators (lower energy, narrower bandwidth, temperature sensitive) or Mach-Zehnder modulators (broader bandwidth, more tolerant to temperature drift, higher energy per bit). This is where photonic and digital are now in direct competition at the component level.
Photodetectors, germanium on silicon for 1310 nm, consume relatively little power. A Ge-on-Si p-i-n photodiode biased at minus 1 to minus 2 V draws microamps of dark current and efficiently converts optical power to electrical current. The subsequent transimpedance amplifier and ADC are the real electronic costs in the readout chain, but these are shared across many parallel outputs in our architecture, so their per-MAC amortized cost falls with matrix size.
The MZI mesh: matrix multiply by interference
A Mach-Zehnder interferometer splits an input optical signal into two arms, applies an independently controllable phase shift to each arm, then recombines them. The output power depends on the phase difference between the arms. Two MZIs in sequence, with a phase modulator between them, form a 2x2 unitary operation. Work by Clements et al. (2016, Optica) showed that any N-by-N unitary matrix can be decomposed into a mesh of N(N-1)/2 such MZI pairs, arranged in a rectangular grid pattern.
This is how we implement the weight matrix: we set the phase values in each MZI to encode one factor of the singular value decomposition of the target matrix. An input vector (the activation vector) is fan-out encoded onto N optical inputs, the mesh performs the transformation in the time it takes light to traverse the chip, and N output photodetectors read the result simultaneously. All N elements of the output vector emerge in parallel, not sequentially.
What this buys us in energy is that the computation itself is passive propagation, not active switching. The energy to change the weights (write new phase values into the MZIs) is the modulator energy budget. But weight writes happen once per model load, not once per inference token. In a sustained inference workload where the same model processes thousands of tokens, the weight programming cost amortizes away into noise.
The comparison that matters: system level, not device level
We are not claiming that photonic MACs are cheaper than digital MACs at the transistor-vs-modulator comparison level. At the current state of silicon photonics fabrication, that comparison is genuinely mixed. What we are saying is that system-level energy per token, including the memory subsystem, is where the photonic co-processor wins on the right workloads.
In a GPU running transformer inference, the most expensive operation is not the arithmetic. It is moving the weight matrices from HBM (high-bandwidth memory) to the compute units on every forward pass. A 70-billion-parameter model has roughly 140 GB of BF16 weights. Even at H100 HBM3 bandwidth of around 3.35 TB/s, loading those weights takes meaningful time and meaningful power. That memory traffic consumes energy that shows up nowhere in a TOPS/W metric for the chip core alone.
The photonic processor holds its weights in the optical domain after initial programming. A weight that is encoded as a phase value in an MZI does not need to be fetched from DRAM before each use. The inference computation happens in place, at the point where the weight is stored. This is the fundamental architectural shift: compute-where-the-weight-lives versus fetch-weight-then-compute. For workloads that process many tokens against a fixed model, this shifts the energy per token calculation substantially in favor of the photonic path.
What the physics does not help you with
Precision is the honest qualification here. Digital logic is exact: a 16-bit MAC gives you 16 bits of output fidelity. Optical interference is limited by signal-to-noise ratio, which in practice means roughly 6 to 8 bits of effective resolution before calibration and thermal compensation techniques are applied. For many neural network inference workloads, 6 to 8 bits is sufficient, particularly for post-training quantized models. But we are not claiming equivalence with INT8 digital arithmetic in terms of numerical fidelity, and the calibration overhead is real work that must be included in any honest energy comparison.
The research direction we are following here in Boulder is weight-stationary architectures: load the model once, process as many tokens as possible before the weights need updating or recalibration. The physics strongly favors batch inference use cases over single-request latency-optimized use cases. We are direct about this tradeoff with every evaluation partner we engage with.
The energy math ultimately favors photonics when two conditions hold: the model is large enough that weight memory bandwidth dominates the per-token energy budget, and the inference workload is dense enough to amortize the weight programming cost across many tokens. For data-center operators running continuous high-volume generation workloads on large models, those conditions are often met. For edge inference or sporadic single-request scenarios, they often are not, and we would not recommend a photonic co-processor for those use cases.