TOPS/W is not the right metric for a co-processor
When accelerator vendors want to make their hardware look good in a data sheet, they report TOPS/W: tera-operations per second per watt. It is a clean number and it lets you make impressive headlines. The problem is that for a photonic co-processor sitting alongside a GPU in a PCIe slot, TOPS/W answers a question that does not match how data-center operators actually run cost accounting.
TOPS/W measures the chip in isolation. It captures how many operations the chip performs per watt of chip power. It does not capture: the power consumed by the GPU running alongside it, the energy cost of weight programming, the PCIe transfer energy, the cooling overhead attributable to the combined system, or the useful work actually performed per unit time. Two chips with identical TOPS/W numbers may be wildly different in their practical economics depending on how they integrate into the system.
For a photonic co-processor specifically, the TOPS/W comparison is even more misleading because the operation count is not directly comparable to digital operations. A photonic matrix-vector multiply on a 64-element mesh in some accounting schemes represents 64*64 = 4096 multiply-accumulate operations happening simultaneously in a single optical pass. In other schemes, you count only the effective numerical operations accounting for analog precision. The denominator in "TOPS" is not standardized across analog and digital systems.
What data-center operators actually measure
The operators we talk to are tracking a small number of metrics that map directly to their infrastructure costs and SLA commitments. These break into two categories: throughput metrics and cost metrics.
On throughput: tokens per second per server node, and tokens per second per PCIe slot are the numbers that matter. A server node has a fixed capital cost, a fixed physical footprint, and a fixed power envelope. Maximizing tokens per second per node maximizes revenue per dollar of infrastructure investment. Tokens per second per PCIe slot matters because PCIe slots are the unit of hardware addition in a running server, and the relevant question is: does adding one photonic co-processor card increase tokens per second, and by how much?
On energy cost: joules per token. Not joules per FLOP, not TOPS/W, not anything that abstracts away from the actual useful work unit. One token generated = one unit of revenue potential. The energy cost per token, measured at the server node power inlet, is what shows up in the electricity bill and in the PUE-adjusted data-center cost model.
A proposed metric: tokens per joule per PCIe slot
We have been thinking about this problem from our side as well, and the metric we keep returning to is: inference tokens per joule, measured per occupied PCIe slot. Call it TPJ/slot.
This metric is grounded in what operators actually pay for. A server node has a fixed PCIe slot budget (typically 4 to 8 double-width slots in a 2U server). Each slot that hosts a photonic co-processor adds some number of tokens per second capacity and consumes some watts from the power supply. TPJ/slot captures both sides of that tradeoff.
The measurement methodology for TPJ/slot: run a continuous inference workload on a fixed model, measure total tokens generated over the measurement window, measure total joules consumed by the co-processor (measured at the PCIe power connector, not estimated from the chip datasheet), divide, normalize per occupied slot. Do this for the baseline GPU-only configuration and the GPU-plus-photonic configuration. The ratio tells you the energy efficiency improvement per PCIe slot occupied.
We are not claiming to have published TPJ/slot numbers for our hardware against a GPU baseline. We are at the pre-production research stage and any numbers we report in this context would be from a specific test configuration that may not represent production performance. What we are arguing is that TPJ/slot is the right framework for evaluating photonic co-processors, and we expect this metric to be the basis for how we present data when we have systems ready for formal evaluation.
The latency vs. throughput split
Inference workloads stratify into two fundamentally different operating regimes, and the relevant metrics differ between them. The first is latency-sensitive: user-facing interactive inference, where the metric is time-to-first-token (TTFT) and per-token latency. The second is throughput-oriented: batch inference jobs, asynchronous workloads, document processing, where the metric is total tokens per second at a given quality level.
The photonic co-processor architecture is substantially better suited to the throughput regime. Weight programming takes time, and the weight-stationary architecture benefits from processing many tokens against the same loaded weights. In the latency regime, single-request decode latency may actually worsen slightly with the photonic path because of the PCIe round-trip overhead, particularly at batch size 1.
Any honest benchmarking of a photonic co-processor must specify which regime the benchmark is measuring. A TOPS/W number that assumes maximum throughput operation misrepresents the latency-regime performance, and vice versa. We think the field needs to adopt the MLPerf inference benchmark methodology's distinction between Server (latency-constrained) and Offline (throughput-maximizing) scenarios as a minimum standard for photonic co-processor benchmarking.
Accuracy as a required dimension
One dimension that pure throughput benchmarks omit entirely is output accuracy. An accelerator that runs faster by degrading inference quality is not a fair comparison to an accurate baseline. For photonic inference, where the analog computation introduces some numeric noise relative to digital FP16, this dimension is non-trivial.
The right accuracy metric depends on the downstream task: perplexity for language models, top-1/top-5 accuracy for classification, ROUGE or BLEU for generation quality tasks. The accuracy metric must be measured at the same quantization level as the throughput measurement. Reporting throughput at 4-bit quantization and accuracy at 8-bit quantization is a common way to cherry-pick favorable numbers, and it is something we watch for carefully in our own internal measurements.
Our position is that a complete photonic inference benchmark must report: tokens per second, joules per token, TTFT at target percentile, and task-specific quality metric, all at the same configuration and batch size. Any benchmarking framework that reports fewer than these four dimensions is giving an incomplete picture of whether the hardware is actually useful for data-center inference workloads.