.. meta:: :description: ROCm Compute Profiler performance model: Pipeline metrics :keywords: Omniperf, ROCm Compute Profiler, ROCm, profiler, tool, Instinct, accelerator, pipeline, wavefront, metrics, launch, runtime VALU, MFMA, instruction mix, FLOPs, arithmetic, operations **************** Pipeline metrics **************** This section details the metrics available in ROCm Compute Profiler to analyze the pipelines discussed in the :doc:`pipeline-descriptions`. .. _wavefront: Wavefront ========= .. _wavefront-launch-stats: Wavefront launch stats ---------------------- The wavefront launch stats panel gives general information about the kernel launch: .. tab-set:: .. tab-item:: CDNA .. jinja:: wavefront-launch-stats-gfx908 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 2 .. jinja:: wavefront-launch-stats-gfx90a :file: _templates/metrics_table.j2 .. tab-item:: CDNA 3 .. jinja:: wavefront-launch-stats-gfx942 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 4 :selected: .. jinja:: wavefront-launch-stats-gfx950 :file: _templates/metrics_table.j2 .. _wavefront-runtime-stats: Wavefront runtime stats ----------------------- The wavefront runtime statistics gives a high-level overview of the execution of wavefronts in a kernel: .. tab-set:: .. tab-item:: CDNA .. jinja:: wavefront-runtime-stats-gfx908 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 2 .. jinja:: wavefront-runtime-stats-gfx90a :file: _templates/metrics_table.j2 .. tab-item:: CDNA 3 .. jinja:: wavefront-runtime-stats-gfx942 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 4 :selected: .. jinja:: wavefront-runtime-stats-gfx950 :file: _templates/metrics_table.j2 .. note:: The measurement of kernel cycles and time typically cannot be directly compared to other cycles, such as, wave cycles. This is due to two factors: * The kernel cycles/timings are measured using a counter that is impacted by scheduling overhead, this is particularly noticeable for short-running kernels (less than 1ms) where scheduling overhead forms a significant portion of the overall kernel runtime. * The wave cycles metric is incremented per-wavefront scheduled to a SIMD every cycle whereas the kernel cycles counter is incremented only once per-cycle when any wavefront is scheduled. .. _instruction-mix: Instruction mix =============== The instruction mix panel shows a breakdown of the various types of instructions executed by the user’s kernel, and which pipelines on the :doc:`CU ` they were executed on. In addition, ROCm Compute Profiler reports further information about the breakdown of operation types for the :ref:`VALU `, vector-memory, and :ref:`MFMA ` instructions. .. note:: All metrics in this section count the instructions issued, and not the total number of operations executed. The values reported by these metrics will not change regardless of the execution mask of the wavefront. Note that even if the execution mask is identically zero (meaning no lanes are active) the instruction will still be counted, as CDNA architecture-based GPUs still consider these as instructions issued. See :mi200-isa-pdf:`EXECute Mask, section 3.3 of the CDNA2 ISA guide<19>` for examples and further details. Overall instruction mix ----------------------- This panel shows the total number of each type of instruction issued to the :doc:`various compute pipelines ` on the :doc:`CU `. These are: .. tab-set:: .. tab-item:: CDNA .. jinja:: instruction-mix-gfx908 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 2 .. jinja:: instruction-mix-gfx90a :file: _templates/metrics_table.j2 .. tab-item:: CDNA 3 .. jinja:: instruction-mix-gfx942 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 4 :selected: .. jinja:: instruction-mix-gfx950 :file: _templates/metrics_table.j2 .. note:: Branch operations are not used for execution mask updates, but only for whole wavefront control flow changes. For more details, see :ref:`desc-branch`. .. _valu-arith-instruction-mix: VALU arithmetic instruction mix ------------------------------- .. warning:: Not all metrics in this section (such as, the floating-point instruction breakdowns) are available on CDNA architecture-based GPUs older than the :ref:`MI2XX ` series. This panel details the various types of vector instructions that were issued to the :ref:`VALU `. The metrics in this section do *not* include :ref:`MFMA ` instructions using the same precision; for instance, the “F16-ADD” metric does not include any 16-bit floating point additions executed as part of an MFMA instruction using the same precision. .. tab-set:: .. tab-item:: CDNA 2 .. jinja:: valu-arith-instruction-mix-gfx90a :file: _templates/metrics_table.j2 .. tab-item:: CDNA 3 .. jinja:: valu-arith-instruction-mix-gfx942 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 4 :selected: .. jinja:: valu-arith-instruction-mix-gfx950 :file: _templates/metrics_table.j2 For an example of these counters in action, refer to :ref:`valu-arith-instruction-mix-ex`. .. _vmem-instruction-mix: VMEM instruction mix -------------------- This section breaks down the types of vector memory (VMEM) instructions that were issued. Refer to the :ref:`Instruction Counts metrics section ` under address processor front end of the vL1D cache for descriptions of these VMEM instructions. .. _matrix-instruction-mix: Matrix instruction mix ---------------------- .. warning:: The metrics in this section are only available on CDNA2-CDNA4 architecture-based GPUs (:ref:`MI2XX `) and newer. This section details the types of Matrix Fused Multiply-Add (:ref:`MFMA `) instructions that were issued. Note that MFMA instructions are classified by the type of input data they operate on, and *not* the data type the result is accumulated to. .. tab-set:: .. tab-item:: CDNA 2 .. jinja:: matrix-instruction-mix-gfx90a :file: _templates/metrics_table.j2 .. tab-item:: CDNA 3 .. jinja:: matrix-instruction-mix-gfx942 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 4 :selected: .. jinja:: matrix-instruction-mix-gfx950 :file: _templates/metrics_table.j2 Compute pipeline ================ .. _metrics-flop-count: FLOP counting conventions ------------------------- ROCm Compute Profiler’s conventions for VALU FLOP counting are as follows: * Addition or multiplication: 1 operation * Transcendentals: 1 operation * Fused multiply-add (FMA): 2 operations Integer operations (IOPs) do not use this convention. They are counted as a single operation regardless of the instruction type. .. note:: Packed operations which operate on multiple operands in the same instruction are counted identically to the underlying instruction type. For example, the ``v_pk_add_f32`` instruction on :ref:`MI2XX `, which performs an add operation on two pairs of aligned 32-bit floating-point operands is counted only as a single addition -- that is, 1 operation. As discussed in the :ref:`instruction-mix` section, the FLOP/IOP metrics in this section do not take into account the execution mask of the operation, and will report the same value even if the execution mask is identically zero. For example, a FMA instruction operating on 32-bit floating-point operands (such as ``v_fma_f32`` on a AMD Instinct :ref:`MI2XX ` Series GPUs) would be counted as 128 total FLOPs: 2 operations (due to the instruction type) multiplied by 64 operations (because the wavefront is composed of 64 work-items). .. _compute-speed-of-light: Compute Speed-of-Light ---------------------- .. warning:: The theoretical maximum throughput for some metrics in this section are currently computed with the maximum achievable clock frequency, as reported by ``rocminfo``, for a GPU. This may not be realistic for all workloads. This section reports the number of floating-point and integer operations executed on the :ref:`VALU ` and :ref:`MFMA ` units in various precisions. We note that unlike the :ref:`VALU instruction mix ` and :ref:`Matrix instruction mix ` sections, the metrics here are reported as FLOPs and IOPs, that is, the total number of operations executed. .. tab-set:: .. tab-item:: CDNA 2 .. jinja:: compute-speed-of-light-gfx90a :file: _templates/metrics_table.j2 .. tab-item:: CDNA 3 .. jinja:: compute-speed-of-light-gfx942 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 4 :selected: .. jinja:: compute-speed-of-light-gfx950 :file: _templates/metrics_table.j2 .. _pipeline-stats: Pipeline statistics ------------------- This section reports a number of key performance characteristics of various execution units on the :doc:`CU `. Refer to :ref:`ipc-example` for a detailed dive into these metrics, and the :ref:`scheduler ` the for a high-level overview of execution units and instruction issue. .. tab-set:: .. tab-item:: CDNA .. jinja:: pipeline-stats-gfx908 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 2 .. jinja:: pipeline-stats-gfx90a :file: _templates/metrics_table.j2 .. tab-item:: CDNA 3 .. jinja:: pipeline-stats-gfx942 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 4 :selected: .. jinja:: pipeline-stats-gfx950 :file: _templates/metrics_table.j2 .. note:: The branch utilization reported in this section also includes time spent in other instruction types (namely: ``s_endpgm``) that are *typically* a very small percentage of the overall kernel execution. This complication is omitted for simplicity, but may result in small amounts of branch utilization (typically less than 1%) for otherwise branch-less kernels. .. _arithmetic-operations: Arithmetic operations --------------------- This section reports the total number of floating-point and integer operations executed in various precisions. Unlike the :ref:`compute-speed-of-light` panel, this section reports both :ref:`VALU ` and :ref:`MFMA ` operations of the same precision (e.g., F32) in the same metric. Additionally, this panel lets the user control how the data is normalized (such as, control the :ref:`normalization unit `), while the speed-of-light panel does not. For more details on how operations are counted, see :ref:`FLOP counting convention `. .. warning:: The metrics in this section do not take into account the execution mask of the operation, and will report the same value even if EXEC is identically zero. For more details, see :ref:`instruction-mix`. .. tab-set:: .. tab-item:: CDNA 2 .. jinja:: arithmetic-operations-gfx90a :file: _templates/metrics_table.j2 .. tab-item:: CDNA 3 .. jinja:: arithmetic-operations-gfx942 :file: _templates/metrics_table.j2 .. tab-item:: CDNA 4 :selected: .. jinja:: arithmetic-operations-gfx950 :file: _templates/metrics_table.j2