[v3] net/mlx5: enable PCI related counters
Checks
Commit Message
Versions of Mellanox NICs starting from CX5 have device counters
related to PCI. These counters are helpful in debugging IO
bottlenecks. For instance, the outbound_pci_stalled_rd and
outbound_pci_stalled_wr counters can help with identifying NIC
stalls due to insufficient PCI credits, which otherwise would
have required a PCI analyzer or a sophisticated PCI root port
with a PMU.
Currently none of these are available in the MLX5 PMD even
though ethtool is capable of reading some of them.
Since PMD uses the same ioctl used by ethtool (SIOCETHTOOL) and
reads via the kernel driver it is possible to add support with
ease.
There is one more PCI related counter and a device counter that
aren't implemented in the Linux driver at the moment. These two
are named outbound_pci_buffer_overflow and dev_out_of_buffer
respectively. As per Nvidia's documentation these two counters
can tell the number of packets dropped due to pci buffer
overflow and the number of times the device owned queue had not
enough buffers allocated.
Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
---
.mailmap | 1 +
drivers/net/mlx5/linux/mlx5_ethdev_os.c | 37 +++++++++++++++++++++++++
2 files changed, 38 insertions(+)
Comments
Hi,
> -----Original Message-----
> From: Wathsala Vithanage <wathsala.vithanage@arm.com>
> Sent: Thursday, February 15, 2024 8:27 PM
> To: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> Dariusz Sosnowski <dsosnowski@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>; Suanming Mou
> <suanmingm@nvidia.com>; Matan Azrad <matan@nvidia.com>
> Cc: dev@dpdk.org; nd@arm.com; Wathsala Vithanage
> <wathsala.vithanage@arm.com>; Honnappa Nagarahalli
> <honnappa.nagarahalli@arm.com>
> Subject: [PATCH v3] net/mlx5: enable PCI related counters
>
> Versions of Mellanox NICs starting from CX5 have device counters related to
> PCI. These counters are helpful in debugging IO bottlenecks. For instance, the
> outbound_pci_stalled_rd and outbound_pci_stalled_wr counters can help
> with identifying NIC stalls due to insufficient PCI credits, which otherwise
> would have required a PCI analyzer or a sophisticated PCI root port with a
> PMU.
> Currently none of these are available in the MLX5 PMD even though ethtool is
> capable of reading some of them.
> Since PMD uses the same ioctl used by ethtool (SIOCETHTOOL) and reads via
> the kernel driver it is possible to add support with ease.
> There is one more PCI related counter and a device counter that aren't
> implemented in the Linux driver at the moment. These two are named
> outbound_pci_buffer_overflow and dev_out_of_buffer respectively. As per
> Nvidia's documentation these two counters can tell the number of packets
> dropped due to pci buffer overflow and the number of times the device
> owned queue had not enough buffers allocated.
>
> Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Patch applied to next-net-mlx,
Kindest regards
Raslan Darawsheh
@@ -1510,6 +1510,7 @@ Walter Heymans <walter.heymans@corigine.com>
Wang Sheng-Hui <shhuiw@gmail.com>
Wangyu (Eric) <seven.wangyu@huawei.com>
Waterman Cao <waterman.cao@intel.com>
+Wathsala Vithanage <wathsala.vithanage@arm.com>
Weichun Chen <weichunx.chen@intel.com>
Wei Dai <wei.dai@intel.com>
Weifeng Li <liweifeng96@126.com>
@@ -1574,6 +1574,43 @@ static const struct mlx5_counter_ctrl mlx5_counters_init[] = {
.dpdk_name = "tx_vport_bytes",
.ctr_name = "vport_tx_bytes",
},
+ /**
+ * Device counters: These counters are for the
+ * entire PCI device (NIC). These counters are
+ * not counting on a per port/queue basis.
+ */
+ {
+ .dpdk_name = "rx_pci_signal_integrity",
+ .ctr_name = "rx_pci_signal_integrity",
+ },
+ {
+ .dpdk_name = "tx_pci_signal_integrity",
+ .ctr_name = "tx_pci_signal_integrity",
+ },
+ {
+ .dpdk_name = "outbound_pci_buffer_overflow",
+ .ctr_name = "outbound_pci_buffer_overflow",
+ },
+ {
+ .dpdk_name = "outbound_pci_stalled_rd",
+ .ctr_name = "outbound_pci_stalled_rd",
+ },
+ {
+ .dpdk_name = "outbound_pci_stalled_wr",
+ .ctr_name = "outbound_pci_stalled_wr",
+ },
+ {
+ .dpdk_name = "outbound_pci_stalled_rd_events",
+ .ctr_name = "outbound_pci_stalled_rd_events",
+ },
+ {
+ .dpdk_name = "outbound_pci_stalled_wr_events",
+ .ctr_name = "outbound_pci_stalled_wr_events",
+ },
+ {
+ .dpdk_name = "dev_out_of_buffer",
+ .ctr_name = "dev_out_of_buffer",
+ },
};
static const unsigned int xstats_n = RTE_DIM(mlx5_counters_init);