[1/1] net/mlx5: fix device probing for old kernel drivers
Checks
Commit Message
Retrieving network interface index via Netlink fails in
case of old mlx5 kernel drivers installed - mlx5_nl_ifindex()
routine fails due to wrong Netlink reply message sent by the
old drivers. This error was ignored in previous versions of
probing. For single devices ifindex was retrieved via sysfs
and link control was not lost, so problem just was not noticed.
In order to support MLX5 PMD functioning over old kernel drivers
this patch adds ifindex retrieving via sysfs into probing routine.
It is worth to note this method works for master/single device only.
Fixes: ad74bc619504 ("net/mlx5: support multiport IB device during probing")
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
---
drivers/net/mlx5/mlx5.c | 35 +++++++++++++++++++++++++++++++----
drivers/net/mlx5/mlx5.h | 1 +
drivers/net/mlx5/mlx5_ethdev.c | 28 ++++++++++++++--------------
3 files changed, 46 insertions(+), 18 deletions(-)
Comments
> Retrieving network interface index via Netlink fails in
> case of old mlx5 kernel drivers installedCan you put a boundary on this statement with a kernel driver
version?
Dave
> -----Original Message-----
> From: David Christensen <drc@linux.vnet.ibm.com>
> Sent: Thursday, April 4, 2019 22:06
> To: Slava Ovsiienko <viacheslavo@mellanox.com>; dev@dpdk.org
> Cc: Shahaf Shuler <shahafs@mellanox.com>
> Subject: Re: [dpdk-dev] [PATCH 1/1] net/mlx5: fix device probing for old
> kernel drivers
>
> > Retrieving network interface index via Netlink fails in case of old
> > mlx5 kernel drivers installed
>> Can you put a boundary on this statement with a kernel driver version?
>
> Dave
As far as I know this setup experiences the problem (I debugged on):
4.15.32+
Ubuntu 16.04.5 LTS
mlx5_core 5.0-0 (from Linux Upstream)
standalone ConnextX-4LX virtual function
These setups has no problem:
3.10.0-327
Red Hat7.2
mlx5_core 4.6-0.2.0 (from OFED)
standalone ConnextX-4LX physical function
5.0rc7+
Red Hat 7.5
mlx5_core 5.0-0
standalone ConnextX-4LX physical function
I'll try to get more information regarding other problematic configs.
Regards,
Slava
The patch allowing to retrieve the network interface index and name via Netlink:
https://www.spinics.net/lists/linux-rdma/msg62948.html
In Linux tree: 5b2cc79d (Leon Romanovsky 2018-03-27 20:40:49 +0300 270)
So, the problem depends on ib_core module version - 4.16 supports getting
ifindex via Netlink, 4.15 does not. Mellanox OFED brings its own ib_core
module, that's why it works over ancient 3.10.327.
I'll update log message of my patch to describe the matter.
With best regards,
Slava
> -----Original Message-----
> From: Slava Ovsiienko
> Sent: Friday, April 5, 2019 12:38
> To: 'David Christensen' <drc@linux.vnet.ibm.com>; dev@dpdk.org
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Ali Alnubani
> <alialnu@mellanox.com>
> Subject: RE: [dpdk-dev] [PATCH 1/1] net/mlx5: fix device probing for old
> kernel drivers
>
> > -----Original Message-----
> > From: David Christensen <drc@linux.vnet.ibm.com>
> > Sent: Thursday, April 4, 2019 22:06
> > To: Slava Ovsiienko <viacheslavo@mellanox.com>; dev@dpdk.org
> > Cc: Shahaf Shuler <shahafs@mellanox.com>
> > Subject: Re: [dpdk-dev] [PATCH 1/1] net/mlx5: fix device probing for
> > old kernel drivers
> >
> > > Retrieving network interface index via Netlink fails in case of old
> > > mlx5 kernel drivers installed
> >> Can you put a boundary on this statement with a kernel driver version?
> >
> > Dave
>
> As far as I know this setup experiences the problem (I debugged on):
> 4.15.32+
> Ubuntu 16.04.5 LTS
> mlx5_core 5.0-0 (from Linux Upstream)
> standalone ConnextX-4LX virtual function
>
> These setups has no problem:
> 3.10.0-327
> Red Hat7.2
> mlx5_core 4.6-0.2.0 (from OFED)
> standalone ConnextX-4LX physical function
>
> 5.0rc7+
> Red Hat 7.5
> mlx5_core 5.0-0
> standalone ConnextX-4LX physical function
>
> I'll try to get more information regarding other problematic configs.
>
> Regards,
> Slava
@@ -1772,12 +1772,39 @@ struct mlx5_dev_spawn_data {
list[ns].ifindex = mlx5_nl_ifindex
(nl_rdma, list[ns].ibv_dev->name, 1);
if (!list[ns].ifindex) {
+ char ifname[IF_NAMESIZE];
+
/*
- * No network interface index found for the
- * specified device, it means there it is not
- * a representor/master.
+ * Netlink failed, it may happen with old
+ * mlx5 kernel drivers. We can assume we have
+ * old driver because here we are processing
+ * single ports IB devices. Let's try sysfs
+ * to retrieve the ifindex. The method works
+ * for master device only.
*/
- continue;
+ if (nd > 1) {
+ /*
+ * Multiple devices found, assume
+ * representors, can not distinguish
+ * master/representor and retrieve
+ * ifindex via sysfs.
+ */
+ continue;
+ }
+ ret = mlx5_get_master_ifname
+ (ibv_match[i]->ibdev_path, &ifname);
+ if (!ret)
+ list[ns].ifindex =
+ if_nametoindex(ifname);
+ if (!list[ns].ifindex) {
+ /*
+ * No network interface index found
+ * for the specified device, it means
+ * there it is neither representor
+ * nor master.
+ */
+ continue;
+ }
}
ret = -1;
if (nl_route >= 0)
@@ -327,6 +327,7 @@ struct mlx5_priv {
/* mlx5_ethdev.c */
int mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE]);
+int mlx5_get_master_ifname(const char *ibdev_path, char (*ifname)[IF_NAMESIZE]);
unsigned int mlx5_ifindex(const struct rte_eth_dev *dev);
int mlx5_ifreq(const struct rte_eth_dev *dev, int req, struct ifreq *ifr);
int mlx5_get_mtu(struct rte_eth_dev *dev, uint16_t *mtu);
@@ -127,21 +127,18 @@ struct ethtool_link_settings {
* @return
* 0 on success, a negative errno value otherwise and rte_errno is set.
*/
-static int
-mlx5_get_master_ifname(const struct rte_eth_dev *dev,
- char (*ifname)[IF_NAMESIZE])
+int
+mlx5_get_master_ifname(const char *ibdev_path, char (*ifname)[IF_NAMESIZE])
{
- struct mlx5_priv *priv = dev->data->dev_private;
DIR *dir;
struct dirent *dent;
unsigned int dev_type = 0;
unsigned int dev_port_prev = ~0u;
char match[IF_NAMESIZE] = "";
- assert(priv);
- assert(priv->sh);
+ assert(ibdev_path);
{
- MKSTR(path, "%s/device/net", priv->sh->ibdev_path);
+ MKSTR(path, "%s/device/net", ibdev_path);
dir = opendir(path);
if (dir == NULL) {
@@ -161,7 +158,7 @@ struct ethtool_link_settings {
continue;
MKSTR(path, "%s/device/net/%s/%s",
- priv->sh->ibdev_path, name,
+ ibdev_path, name,
(dev_type ? "dev_id" : "dev_port"));
file = fopen(path, "rb");
@@ -222,15 +219,18 @@ struct ethtool_link_settings {
mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE])
{
struct mlx5_priv *priv = dev->data->dev_private;
- unsigned int ifindex =
- priv->nl_socket_rdma >= 0 ?
- mlx5_nl_ifindex(priv->nl_socket_rdma,
- priv->sh->ibdev_name,
- priv->ibv_port) : 0;
+ unsigned int ifindex;
+ assert(priv);
+ assert(priv->sh);
+ ifindex = priv->nl_socket_rdma >= 0 ?
+ mlx5_nl_ifindex(priv->nl_socket_rdma,
+ priv->sh->ibdev_name,
+ priv->ibv_port) : 0;
if (!ifindex) {
if (!priv->representor)
- return mlx5_get_master_ifname(dev, ifname);
+ return mlx5_get_master_ifname(priv->sh->ibdev_path,
+ ifname);
rte_errno = ENXIO;
return -rte_errno;
}