Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.

Why it matters

AWS has published the opening installment of a tutorial series on profiling TorchNeuron, its open source PyTorch backend for AWS Trainium, the company's purpose-built chip for deep learning training and inference. The post teaches practitioners how to move beyond intuition about model speed and instead systematically diagnose where inference time is spent. It explains how Neuron registers with PyTorch through the PrivateUse1 extension point, then walks through configuring torch.profiler with Neuron-specific objects, NeuronConfig and NeuronProfiler, so device time propagates into the key_averages() table. Readers also learn to use Neuron Explorer, a browser-based viewer, to trace a single operation from framework dispatch through runtime scheduling to execution on a NeuronCore, aided by paired executable and instruction-trace files. The tutorial applies these techniques to RMSNorm, comparing a naive PyTorch implementation, the built-in torch.rms_norm fused kernel, and a torch.compile variant under identical inputs and settings. Reproduction scripts are available in the torch-neuronx examples directory, making the work accessible to those without prior profiling experience.