| Message ID | 1554300882-23990-1-git-send-email-viacheslavo@mellanox.com (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Delegated to: | Shahaf Shuler |
| Headers | show |
| Series | [1/1] net/mlx5: fix device probing for old kernel drivers | expand |
| Context | Check | Description |
|---|---|---|
| ci/checkpatch | warning | coding style issues |
| ci/Intel-compilation | success | Compilation OK |
| ci/intel-Performance-Testing | success | Performance Testing PASS |
| ci/mellanox-Performance-Testing | success | Performance Testing PASS |
> Retrieving network interface index via Netlink fails in > case of old mlx5 kernel drivers installedCan you put a boundary on this statement with a kernel driver version? Dave
> -----Original Message----- > From: David Christensen <drc@linux.vnet.ibm.com> > Sent: Thursday, April 4, 2019 22:06 > To: Slava Ovsiienko <viacheslavo@mellanox.com>; dev@dpdk.org > Cc: Shahaf Shuler <shahafs@mellanox.com> > Subject: Re: [dpdk-dev] [PATCH 1/1] net/mlx5: fix device probing for old > kernel drivers > > > Retrieving network interface index via Netlink fails in case of old > > mlx5 kernel drivers installed >> Can you put a boundary on this statement with a kernel driver version? > > Dave As far as I know this setup experiences the problem (I debugged on): 4.15.32+ Ubuntu 16.04.5 LTS mlx5_core 5.0-0 (from Linux Upstream) standalone ConnextX-4LX virtual function These setups has no problem: 3.10.0-327 Red Hat7.2 mlx5_core 4.6-0.2.0 (from OFED) standalone ConnextX-4LX physical function 5.0rc7+ Red Hat 7.5 mlx5_core 5.0-0 standalone ConnextX-4LX physical function I'll try to get more information regarding other problematic configs. Regards, Slava
The patch allowing to retrieve the network interface index and name via Netlink: https://www.spinics.net/lists/linux-rdma/msg62948.html In Linux tree: 5b2cc79d (Leon Romanovsky 2018-03-27 20:40:49 +0300 270) So, the problem depends on ib_core module version - 4.16 supports getting ifindex via Netlink, 4.15 does not. Mellanox OFED brings its own ib_core module, that's why it works over ancient 3.10.327. I'll update log message of my patch to describe the matter. With best regards, Slava > -----Original Message----- > From: Slava Ovsiienko > Sent: Friday, April 5, 2019 12:38 > To: 'David Christensen' <drc@linux.vnet.ibm.com>; dev@dpdk.org > Cc: Shahaf Shuler <shahafs@mellanox.com>; Ali Alnubani > <alialnu@mellanox.com> > Subject: RE: [dpdk-dev] [PATCH 1/1] net/mlx5: fix device probing for old > kernel drivers > > > -----Original Message----- > > From: David Christensen <drc@linux.vnet.ibm.com> > > Sent: Thursday, April 4, 2019 22:06 > > To: Slava Ovsiienko <viacheslavo@mellanox.com>; dev@dpdk.org > > Cc: Shahaf Shuler <shahafs@mellanox.com> > > Subject: Re: [dpdk-dev] [PATCH 1/1] net/mlx5: fix device probing for > > old kernel drivers > > > > > Retrieving network interface index via Netlink fails in case of old > > > mlx5 kernel drivers installed > >> Can you put a boundary on this statement with a kernel driver version? > > > > Dave > > As far as I know this setup experiences the problem (I debugged on): > 4.15.32+ > Ubuntu 16.04.5 LTS > mlx5_core 5.0-0 (from Linux Upstream) > standalone ConnextX-4LX virtual function > > These setups has no problem: > 3.10.0-327 > Red Hat7.2 > mlx5_core 4.6-0.2.0 (from OFED) > standalone ConnextX-4LX physical function > > 5.0rc7+ > Red Hat 7.5 > mlx5_core 5.0-0 > standalone ConnextX-4LX physical function > > I'll try to get more information regarding other problematic configs. > > Regards, > Slava
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 7d322b6..9aa5f0b 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1772,12 +1772,39 @@ struct mlx5_dev_spawn_data { list[ns].ifindex = mlx5_nl_ifindex (nl_rdma, list[ns].ibv_dev->name, 1); if (!list[ns].ifindex) { + char ifname[IF_NAMESIZE]; + /* - * No network interface index found for the - * specified device, it means there it is not - * a representor/master. + * Netlink failed, it may happen with old + * mlx5 kernel drivers. We can assume we have + * old driver because here we are processing + * single ports IB devices. Let's try sysfs + * to retrieve the ifindex. The method works + * for master device only. */ - continue; + if (nd > 1) { + /* + * Multiple devices found, assume + * representors, can not distinguish + * master/representor and retrieve + * ifindex via sysfs. + */ + continue; + } + ret = mlx5_get_master_ifname + (ibv_match[i]->ibdev_path, &ifname); + if (!ret) + list[ns].ifindex = + if_nametoindex(ifname); + if (!list[ns].ifindex) { + /* + * No network interface index found + * for the specified device, it means + * there it is neither representor + * nor master. + */ + continue; + } } ret = -1; if (nl_route >= 0) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 4f6c1b7..6552691 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -327,6 +327,7 @@ struct mlx5_priv { /* mlx5_ethdev.c */ int mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE]); +int mlx5_get_master_ifname(const char *ibdev_path, char (*ifname)[IF_NAMESIZE]); unsigned int mlx5_ifindex(const struct rte_eth_dev *dev); int mlx5_ifreq(const struct rte_eth_dev *dev, int req, struct ifreq *ifr); int mlx5_get_mtu(struct rte_eth_dev *dev, uint16_t *mtu); diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 7273bd9..8a251aa 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -127,21 +127,18 @@ struct ethtool_link_settings { * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ -static int -mlx5_get_master_ifname(const struct rte_eth_dev *dev, - char (*ifname)[IF_NAMESIZE]) +int +mlx5_get_master_ifname(const char *ibdev_path, char (*ifname)[IF_NAMESIZE]) { - struct mlx5_priv *priv = dev->data->dev_private; DIR *dir; struct dirent *dent; unsigned int dev_type = 0; unsigned int dev_port_prev = ~0u; char match[IF_NAMESIZE] = ""; - assert(priv); - assert(priv->sh); + assert(ibdev_path); { - MKSTR(path, "%s/device/net", priv->sh->ibdev_path); + MKSTR(path, "%s/device/net", ibdev_path); dir = opendir(path); if (dir == NULL) { @@ -161,7 +158,7 @@ struct ethtool_link_settings { continue; MKSTR(path, "%s/device/net/%s/%s", - priv->sh->ibdev_path, name, + ibdev_path, name, (dev_type ? "dev_id" : "dev_port")); file = fopen(path, "rb"); @@ -222,15 +219,18 @@ struct ethtool_link_settings { mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE]) { struct mlx5_priv *priv = dev->data->dev_private; - unsigned int ifindex = - priv->nl_socket_rdma >= 0 ? - mlx5_nl_ifindex(priv->nl_socket_rdma, - priv->sh->ibdev_name, - priv->ibv_port) : 0; + unsigned int ifindex; + assert(priv); + assert(priv->sh); + ifindex = priv->nl_socket_rdma >= 0 ? + mlx5_nl_ifindex(priv->nl_socket_rdma, + priv->sh->ibdev_name, + priv->ibv_port) : 0; if (!ifindex) { if (!priv->representor) - return mlx5_get_master_ifname(dev, ifname); + return mlx5_get_master_ifname(priv->sh->ibdev_path, + ifname); rte_errno = ENXIO; return -rte_errno; }
Retrieving network interface index via Netlink fails in case of old mlx5 kernel drivers installed - mlx5_nl_ifindex() routine fails due to wrong Netlink reply message sent by the old drivers. This error was ignored in previous versions of probing. For single devices ifindex was retrieved via sysfs and link control was not lost, so problem just was not noticed. In order to support MLX5 PMD functioning over old kernel drivers this patch adds ifindex retrieving via sysfs into probing routine. It is worth to note this method works for master/single device only. Fixes: ad74bc619504 ("net/mlx5: support multiport IB device during probing") Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> --- drivers/net/mlx5/mlx5.c | 35 +++++++++++++++++++++++++++++++---- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_ethdev.c | 28 ++++++++++++++-------------- 3 files changed, 46 insertions(+), 18 deletions(-)