From patchwork Tue Mar 1 12:15:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 108433 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7B588A0353; Tue, 1 Mar 2022 13:16:05 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0C93A426FC; Tue, 1 Mar 2022 13:16:02 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2078.outbound.protection.outlook.com [40.107.94.78]) by mails.dpdk.org (Postfix) with ESMTP id 677AC426EA; Tue, 1 Mar 2022 13:16:00 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Tp1KVVBzjzhZKyPsyX96xH2RRD+WaOKa0sly8/msR8ud/HuXou21kYeg5Nj7o+qSESEmSkxM6Lbr88XZOvbVFsgwdSERZouNF83zBLJQKlLbypJfyXdoQ/yg+wKqfTNtIef7xySMULfhNGlHRZqkyJvgXcXHUAViRkjrggjVFpWRL9P3NZIddK+fgE0lGMC6gwiv1/B8nM2k3nQWCBjSMIbfh9lDcN5vFggktojILLoqLc7OYkaV0avJwpfx9GHpuryjfzreegcCA9kor+FndEXeWBltWtFc8ed7cGF04T4tkCg/iBlljaW5wWR/11Oax6yU1uF9m4aKPhiH85NYag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qh6jpHkU4sOazZrB4TLlupwolydwqDhWox8v5jtRG+I=; b=WJSIK10uGSbZYm///EPfJiFE7Zcb8fwtF8j35xjIRSriP56krYSS3Ebw8p0q5kJg0bx4jW3OYk2nlBLT5oqDfedalcGGLrD76G8vreHTbdHDqEm0kLG/PuVPGQ7FjAFLspY7IfDtHR25J287WdSRcjM3/2d6W23Bpdx/NCGv2lBH5m4BewHFdoujdk73ioqufAIO5WU6OiJ+3Xf1lxKyA95NDBaVqqLv69GUQD2ckPgD1c+fw4JL1kLBMbdC04kx7PfGeN1/RC7SauTFy2qIzsuCkPTtRU6t9y7qWDGsxnOuf5ssKckzLHwRoLU/K+V+1Rh2y1QoTkV/OH1fFOpNFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=ashroe.eu smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qh6jpHkU4sOazZrB4TLlupwolydwqDhWox8v5jtRG+I=; b=Ac1b/eFGhJOdhIOonAl+V8j6SaDFdzEtTVjnoCoMUXuaBFJEB7Te3mtc967CMnqQbe5fRMGZyz+aUgveT9Cke+TLOhvHRCFkwo/Z0PhfgkD8CAd9f4/+XIDzH/CJ5Y7Sc6KRC/Ed+VYduYQF83fDgFi/PL1GsI/OdtlMCLYO8LZcyg2kADoEhrgscp5tu5LAvsWHSsGieT4vaUen8eeqXSWRktKTXAJ7tLyENfB8nN2ZYnC2K685loPnon9tALSK11pf0JQPfseOVpu6f3KPeb1UB/z8BcwQaPOaFKdUBccWFZD6xtImMTfrjdpZFf8WAOAGJdNAiphaYZ3/G58HJA== Received: from DM5PR22CA0011.namprd22.prod.outlook.com (2603:10b6:3:101::21) by CY4PR12MB1685.namprd12.prod.outlook.com (2603:10b6:910:11::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.24; Tue, 1 Mar 2022 12:15:57 +0000 Received: from DM6NAM11FT057.eop-nam11.prod.protection.outlook.com (2603:10b6:3:101:cafe::a) by DM5PR22CA0011.outlook.office365.com (2603:10b6:3:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.26 via Frontend Transport; Tue, 1 Mar 2022 12:15:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by DM6NAM11FT057.mail.protection.outlook.com (10.13.172.252) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Tue, 1 Mar 2022 12:15:57 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 1 Mar 2022 12:15:56 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Tue, 1 Mar 2022 04:15:54 -0800 Received: from nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Tue, 1 Mar 2022 04:15:53 -0800 From: Dmitry Kozlyuk To: CC: , Viacheslav Ovsiienko , "Matan Azrad" , Ray Kinsella Subject: [PATCH v2 1/3] common/mlx5: add Netlink event helpers Date: Tue, 1 Mar 2022 14:15:12 +0200 Message-ID: <20220301121514.41497-2-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220301121514.41497-1-dkozlyuk@nvidia.com> References: <20220223164333.3834590-1-dkozlyuk@nvidia.com> <20220301121514.41497-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cafda99e-8878-4272-365b-08d9fb7d3dbb X-MS-TrafficTypeDiagnostic: CY4PR12MB1685:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YR14YmLmyxJ5sz4D9jQgKe3BNNdWg2gESUX9LtyHz/J/NcVBm2J9diARlMm8HAvRJUMMQMiZJFmfrX2QysdPmw/IKKZiyhpPzLouOKFfni7sHY8fXzy5SUTgDsBzuwC34yI/IsM7Ivfbrpr9bF7PxuH6k2RQcWLDe8Zz3baNqQpEESxI5zFaz48PgjKE4EUBNZ7dTmkw1cRuHdfirtMlgsGil5S2gVisuFCYMJqizNfKPHnwXgg9HSdymlsgYVpf8k0xYUQFvZB9zSgC/k68hTo45mu94D1WQ1dPTsjAMg1P08QW9hmL7AQJmkpoOY2tp6dTSqKoUVgXZ1YnRKHSss1Mg0ZckkjHA/3LS0uEFZB7MeaTsHUTWtmVcDtWVEvdkPzTCaGgrmpsxsHiBLuOKaTBw+LvIgRNZtl7RdjBXetgZxD1Mv4112nLG+2QKJJDNmqZUATB4SJCoGpC6VmfYptLHEc9I1EnKJi/EYm8eZPa19pz+Wt2aLz6jtZg6p2z02SrHjnjrZwmz75aWXnc004Dbncbuyc1Y0LJC/fJxqsVYn6hPUPSwFBPivuyyVJknTBzqWTG9Df9Kv0/ney2gpCGp0SFqeElAjrq4RtTMLkEEJt0OF6LuQKjlK3k1QMrdGNnuZFidNMXLOAN8gE0pCEufu8K+lLc1+VP9X4a2iEim2NGhKa0CiLAcCZG5E/F7ykFdQUYC9Axnm8YSkrIozQn2qAG5wsaNtaMD1X7xLgTx/mMu/cu2s33+sP31YO7 X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(336012)(186003)(8676002)(81166007)(47076005)(36756003)(6666004)(7696005)(4326008)(26005)(86362001)(82310400004)(2616005)(1076003)(426003)(6286002)(83380400001)(70206006)(70586007)(55016003)(6916009)(316002)(508600001)(8936002)(54906003)(5660300002)(356005)(40460700003)(2906002)(36860700001)(36900700001)(309714004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Mar 2022 12:15:57.3080 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cafda99e-8878-4272-365b-08d9fb7d3dbb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT057.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1685 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Introduce mlx5_nl_read_events() to read Netlink events (technically, messages) from a socket that was configured to listen for them via a new mlx5_nl_init() parameter. Add mlx5_nl_parse_link_status_update() helper to extract information from link-related events. This patch is a shared base for later fixes. Cc: stable@dpdk.org Signed-off-by: Dmitry Kozlyuk Reviewed-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/mlx5_common_os.c | 2 +- drivers/common/mlx5/linux/mlx5_nl.c | 102 ++++++++++++++++++++- drivers/common/mlx5/linux/mlx5_nl.h | 8 +- drivers/common/mlx5/version.map | 2 + drivers/net/mlx5/linux/mlx5_os.c | 8 +- drivers/net/mlx5/linux/mlx5_vlan_os.c | 2 +- 6 files changed, 116 insertions(+), 8 deletions(-) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index a3c25638da..030ceb561f 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -590,7 +590,7 @@ mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) static int mlx5_nl_roce_disable(const char *addr) { - int nlsk_fd = mlx5_nl_init(NETLINK_GENERIC); + int nlsk_fd = mlx5_nl_init(NETLINK_GENERIC, 0); int devlink_id; int enable; int ret; diff --git a/drivers/common/mlx5/linux/mlx5_nl.c b/drivers/common/mlx5/linux/mlx5_nl.c index fd4c2d2625..5d04857b38 100644 --- a/drivers/common/mlx5/linux/mlx5_nl.c +++ b/drivers/common/mlx5/linux/mlx5_nl.c @@ -185,19 +185,22 @@ uint32_t atomic_sn; * * @param protocol * Netlink protocol (e.g. NETLINK_ROUTE, NETLINK_RDMA). + * @param groups + * Groups to listen (e.g. RTMGRP_LINK), can be 0. * * @return * A file descriptor on success, a negative errno value otherwise and * rte_errno is set. */ int -mlx5_nl_init(int protocol) +mlx5_nl_init(int protocol, int groups) { int fd; int buf_size; socklen_t opt_size; struct sockaddr_nl local = { .nl_family = AF_NETLINK, + .nl_groups = groups, }; int ret; @@ -1862,3 +1865,100 @@ mlx5_nl_enable_roce_set(int nlsk_fd, int family_id, const char *pci_addr, /* Now, need to reload the driver. */ return mlx5_nl_driver_reload(nlsk_fd, family_id, pci_addr); } + +/** + * Try to parse a Netlink message as a link status update. + * + * @param hdr + * Netlink message header. + * @param[out] ifindex + * Index of the updated interface. + * + * @return + * 0 on success, negative on failure. + */ +int +mlx5_nl_parse_link_status_update(struct nlmsghdr *hdr, uint32_t *ifindex) +{ + struct ifinfomsg *info; + + switch (hdr->nlmsg_type) { + case RTM_NEWLINK: + case RTM_DELLINK: + case RTM_GETLINK: + case RTM_SETLINK: + info = NLMSG_DATA(hdr); + *ifindex = info->ifi_index; + return 0; + } + return -1; +} + +/** + * Read pending events from a Netlink socket. + * + * @param nlsk_fd + * Netlink socket. + * @param cb + * Callback invoked for each of the events. + * @param cb_arg + * User data for the callback. + * + * @return + * 0 on success, including the case when there are no events. + * Negative on failure and rte_errno is set. + */ +int +mlx5_nl_read_events(int nlsk_fd, mlx5_nl_event_cb *cb, void *cb_arg) +{ + char buf[8192]; + struct sockaddr_nl addr; + struct iovec iov = { + .iov_base = buf, + .iov_len = sizeof(buf), + }; + struct msghdr msg = { + .msg_name = &addr, + .msg_namelen = sizeof(addr), + .msg_iov = &iov, + .msg_iovlen = 1, + }; + struct nlmsghdr *hdr; + ssize_t size; + + while (1) { + size = recvmsg(nlsk_fd, &msg, MSG_DONTWAIT); + if (size < 0) { + if (errno == EAGAIN) + return 0; + if (errno == EINTR) + continue; + DRV_LOG(DEBUG, "Failed to receive netlink message: %s", + strerror(errno)); + rte_errno = errno; + return -rte_errno; + } + hdr = (struct nlmsghdr *)buf; + while (size >= (ssize_t)sizeof(*hdr)) { + ssize_t msg_len = hdr->nlmsg_len; + ssize_t data_len = msg_len - sizeof(*hdr); + ssize_t aligned_len; + + if (data_len < 0) { + DRV_LOG(DEBUG, "Netlink message too short"); + rte_errno = EINVAL; + return -rte_errno; + } + aligned_len = NLMSG_ALIGN(msg_len); + if (aligned_len > size) { + DRV_LOG(DEBUG, "Netlink message too long"); + rte_errno = EINVAL; + return -rte_errno; + } + cb(hdr, cb_arg); + hdr = RTE_PTR_ADD(hdr, aligned_len); + size -= aligned_len; + } + } + return 0; +} diff --git a/drivers/common/mlx5/linux/mlx5_nl.h b/drivers/common/mlx5/linux/mlx5_nl.h index 2063c0deeb..0b7552338a 100644 --- a/drivers/common/mlx5/linux/mlx5_nl.h +++ b/drivers/common/mlx5/linux/mlx5_nl.h @@ -11,6 +11,7 @@ #include "mlx5_common.h" +typedef void (mlx5_nl_event_cb)(struct nlmsghdr *hdr, void *user_data); /* VLAN netdev for VLAN workaround. */ struct mlx5_nl_vlan_dev { @@ -30,7 +31,7 @@ struct mlx5_nl_vlan_vmwa_context { }; __rte_internal -int mlx5_nl_init(int protocol); +int mlx5_nl_init(int protocol, int groups); __rte_internal int mlx5_nl_mac_addr_add(int nlsk_fd, unsigned int iface_idx, uint64_t *mac_own, struct rte_ether_addr *mac, uint32_t index); @@ -75,4 +76,9 @@ int mlx5_nl_enable_roce_get(int nlsk_fd, int family_id, const char *pci_addr, int mlx5_nl_enable_roce_set(int nlsk_fd, int family_id, const char *pci_addr, int enable); +__rte_internal +int mlx5_nl_read_events(int nlsk_fd, mlx5_nl_event_cb *cb, void *cb_arg); +__rte_internal +int mlx5_nl_parse_link_status_update(struct nlmsghdr *hdr, uint32_t *ifindex); + #endif /* RTE_PMD_MLX5_NL_H_ */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index cb20a7d893..a23a30a6c0 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -127,9 +127,11 @@ INTERNAL { mlx5_nl_mac_addr_flush; # WINDOWS_NO_EXPORT mlx5_nl_mac_addr_remove; # WINDOWS_NO_EXPORT mlx5_nl_mac_addr_sync; # WINDOWS_NO_EXPORT + mlx5_nl_parse_link_status_update; # WINDOWS_NO_EXPORT mlx5_nl_port_state; # WINDOWS_NO_EXPORT mlx5_nl_portnum; # WINDOWS_NO_EXPORT mlx5_nl_promisc; # WINDOWS_NO_EXPORT + mlx5_nl_read_events; # WINDOWS_NO_EXPORT mlx5_nl_switch_info; # WINDOWS_NO_EXPORT mlx5_nl_vf_mac_addr_modify; # WINDOWS_NO_EXPORT mlx5_nl_vlan_vmwa_create; # WINDOWS_NO_EXPORT diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index be95095521..17e7144cc9 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1117,7 +1117,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, sh = mlx5_alloc_shared_dev_ctx(spawn, mkvlist); if (!sh) return NULL; - nl_rdma = mlx5_nl_init(NETLINK_RDMA); + nl_rdma = mlx5_nl_init(NETLINK_RDMA, 0); /* Check port status. */ if (spawn->phys_port <= UINT8_MAX) { /* Legacy Verbs api only support u8 port number. */ @@ -1180,7 +1180,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, priv->mtu = RTE_ETHER_MTU; /* Some internal functions rely on Netlink sockets, open them now. */ priv->nl_socket_rdma = nl_rdma; - priv->nl_socket_route = mlx5_nl_init(NETLINK_ROUTE); + priv->nl_socket_route = mlx5_nl_init(NETLINK_ROUTE, 0); priv->representor = !!switch_info->representor; priv->master = !!switch_info->master; priv->domain_id = RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID; @@ -1927,8 +1927,8 @@ mlx5_os_pci_probe_pf(struct mlx5_common_device *cdev, * matching ones, gathering into the list. */ struct ibv_device *ibv_match[ret + 1]; - int nl_route = mlx5_nl_init(NETLINK_ROUTE); - int nl_rdma = mlx5_nl_init(NETLINK_RDMA); + int nl_route = mlx5_nl_init(NETLINK_ROUTE, 0); + int nl_rdma = mlx5_nl_init(NETLINK_RDMA, 0); unsigned int i; while (ret-- > 0) { diff --git a/drivers/net/mlx5/linux/mlx5_vlan_os.c b/drivers/net/mlx5/linux/mlx5_vlan_os.c index 80ccd5a460..81611a8d3f 100644 --- a/drivers/net/mlx5/linux/mlx5_vlan_os.c +++ b/drivers/net/mlx5/linux/mlx5_vlan_os.c @@ -135,7 +135,7 @@ mlx5_vlan_vmwa_init(struct rte_eth_dev *dev, uint32_t ifindex) return NULL; } rte_spinlock_init(&vmwa->sl); - vmwa->nl_socket = mlx5_nl_init(NETLINK_ROUTE); + vmwa->nl_socket = mlx5_nl_init(NETLINK_ROUTE, 0); if (vmwa->nl_socket < 0) { DRV_LOG(WARNING, "Can not create Netlink socket" From patchwork Tue Mar 1 12:15:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 108434 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 32987A0353; Tue, 1 Mar 2022 13:16:11 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EA392426F9; Tue, 1 Mar 2022 13:16:04 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2086.outbound.protection.outlook.com [40.107.94.86]) by mails.dpdk.org (Postfix) with ESMTP id 510AA426EA; Tue, 1 Mar 2022 13:16:01 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PyhQq9cmYYi2BALimRQyIigkPPRgY4Sr2ipIxPTPsnVFGsn++lHixtwW5hy+E/2wgq+aFadBDPETlWcPra054vn/fL7zlfODyTacRaZD1jFFMZ7p3Jg9y+fnN9S4t8InONZPmvEi8UckWzF0g8AI4fFb9mM9iFkl7kPi1Udzv2mm47H2bryEBD9fwPUSAyU9oxCvpXuy/Ef8KNTaBTVd9/PL3Lu22VXzTs4IUP/x2HRj/Iou9ev9suOf5T7T9zekWuuT9u+jPVya3YEoHRO1szM+wyI8HgiMd8BqyVGK7DQbZAEYZxkP3J6t0A45xiwGM0VyGHQ3t/KyQuXVLUi1ZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=G5sM/U/w3J13V0Np8NxPhWgM/pTyXqRUs6zbZC5j6qk=; b=mQDU1MiQhfzkeMojyr0uWwNk3bniJqSNbA+uxIo+x2U+iYb3UC1FMAYYyDoKlVw/V1WSgvAKdyUjiHQqyP3RqGj7IlD4hlHSPhLp6AiFlAkdtMFWM6JfM+1aKdsnga/hkBbdhFRppkGU4KsRE+cFjMF/kW6iBiAL1OgXnncaObQ2wfoD9IG0PPtvcAe/DKe57Mpq7k5dYSiOgJqW1SZ/yfIdvjd9+6ndPSmsUehbe4plA50KFEGbrmr2fJLWkuuj9EW5uPmSUyM+56r208XQZR+pL/7eaSOSx4BMfSZYK/nTCtP7v8NXaiNPncxWIXsVZSt3X6kkQosNnZMxkjMJPQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=G5sM/U/w3J13V0Np8NxPhWgM/pTyXqRUs6zbZC5j6qk=; b=lRP+wD7Ck7bcOf2WIrF+9Lhk0DNJu0umbj+/GRavVu0m8rQT2Kby1B7pyGtiiSg/GvLbVMpWw9xNL56rKGPnmGQ0CQ1gqsdDuzyKlobnBIhYwLEQJh5CZOUElmcLUBBWvcfwUWBrrQJivAS65pI9tbkbIkyuKPt+XFc3U1D9pSIXnsk7m5EA5xQu/oVo+WwSqeecl0gCYfUowaINU1uDZnlk32o2RcMQU2ybbCDKR8wiQFXcsqJH9Pw92pkV+6U5pO5dKJJFPTAJMHbnP/Tfgj6cdLkaZpQL0QW5NchQHZ8C49r6kRa85vjRC+2pGykhRLh8KWnFRY3WYt1UHQorAA== Received: from MWHPR15CA0061.namprd15.prod.outlook.com (2603:10b6:301:4c::23) by DS7PR12MB5957.namprd12.prod.outlook.com (2603:10b6:8:7c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.13; Tue, 1 Mar 2022 12:15:59 +0000 Received: from CO1NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:301:4c:cafe::c0) by MWHPR15CA0061.outlook.office365.com (2603:10b6:301:4c::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.13 via Frontend Transport; Tue, 1 Mar 2022 12:15:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT035.mail.protection.outlook.com (10.13.175.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Tue, 1 Mar 2022 12:15:58 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 1 Mar 2022 12:15:58 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Tue, 1 Mar 2022 04:15:56 -0800 Received: from nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Tue, 1 Mar 2022 04:15:55 -0800 From: Dmitry Kozlyuk To: CC: , Viacheslav Ovsiienko , "Matan Azrad" Subject: [PATCH v2 2/3] net/mlx5: fix link status change detection Date: Tue, 1 Mar 2022 14:15:13 +0200 Message-ID: <20220301121514.41497-3-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220301121514.41497-1-dkozlyuk@nvidia.com> References: <20220223164333.3834590-1-dkozlyuk@nvidia.com> <20220301121514.41497-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 40705087-2d3d-406d-275d-08d9fb7d3eb7 X-MS-TrafficTypeDiagnostic: DS7PR12MB5957:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ooXUx0nMeQ1opjWCVrJjIda4+8x8Ju9Sv6PTl1lH6zDTj7cMSKQi1Gv5jEbKhIBa++j4oHKoKm8sWmcnaSIZrreN13gZYdpdNvdUerJr0H7BpyTBRKYmTMj1suGmwBlQeZJl9kfp1j3b/gy6d5tVx7muWjc9NPf/LBVGKxcw/BNRbQXl6P231p8FDZgnlcdlV9ox3CQLnTPOFBhM/QSQ2FAb0bqwArXquYcBbcrLZ2UOjos+yKqE5T5jze1zcrtPV9Sr3s3wsXzjNqFdRqKP3FnD0MNx3ITlVRqcH0Mt7JBWVUBkMFAyyKq2cOyRtwiB6Lx17xh6Kx0FF5j7YxqwukOaxF9hv/TM9GI8uGMl8HZwtFDYyy/lFU6DyW+MYLiBfts74JH88IVlYWzo8UnmAYY4e2tdWeZ/dLr0pvINmGigy0owLfXq+a80VuzWyAiwFkl3tn4rIVYJvdxzZZNzckWucloeOJYJifJ0sP3yLWIQ3lJLa94HiMICkd29xR1DXA3Mg0H77gBFq9TjgTggesKKx0G30BkdKl16mg0nBfM2jBb7F8HxqiEeIp3WsB/4hDBexHsTweV17vzotwlme0lacyFsXdZGZ0vTTDdlndg13+I6mOH8acEHkqqPYOaXfiVnBqjktewCe9otXmeAPCFHv9zB8HQExqOdnh3+9sEAsX7LmkkNl1ZZABAs3dqX00AhKbnUbyh+7Xprn9GJpw== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(40460700003)(8936002)(36860700001)(2906002)(47076005)(6666004)(6286002)(7696005)(82310400004)(83380400001)(426003)(336012)(1076003)(2616005)(186003)(508600001)(26005)(107886003)(86362001)(6916009)(450100002)(316002)(81166007)(356005)(54906003)(5660300002)(8676002)(70586007)(55016003)(70206006)(4326008)(36756003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Mar 2022 12:15:58.9903 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 40705087-2d3d-406d-275d-08d9fb7d3eb7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5957 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sometimes net/mlx5 devices did not detect link status change to "up". Each shared device was monitoring IBV_EVENT_PORT_{ACTIVE,ERR} and queried the link status upon receiving the event. IBV_EVENT_PORT_ACTIVE is delivered when the logical link status (UP flag) is set, but the physical link status (RUNNING flag) may be down at that time, in which case the new link status would be erroneously considered down. IBV interface is insufficient for the task. Monitor interface events using Netlink. Fixes: 198a3c339a8f ("mlx5: handle link status interrupts") Cc: stable@dpdk.org Signed-off-by: Dmitry Kozlyuk Reviewed-by: Viacheslav Ovsiienko --- Coverity complains about unchecked rte_intr_fd/type_set() return values. That would be useless, because rte_intr_handle is in a known state. drivers/net/mlx5/linux/mlx5_ethdev_os.c | 63 ++++++++++++++++++++----- drivers/net/mlx5/linux/mlx5_os.c | 55 +++++++++++++++++++++ drivers/net/mlx5/mlx5.c | 1 + drivers/net/mlx5/mlx5.h | 3 ++ drivers/net/mlx5/mlx5_trigger.c | 12 ++++- 5 files changed, 120 insertions(+), 14 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c index c19825ee52..8fe73f1adb 100644 --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c @@ -38,6 +38,7 @@ #include #include #include +#include #include "mlx5.h" #include "mlx5_rxtx.h" @@ -760,6 +761,56 @@ mlx5_dev_interrupt_device_fatal(struct mlx5_dev_ctx_shared *sh) } } +static void +mlx5_dev_interrupt_nl_cb(struct nlmsghdr *hdr, void *cb_arg) +{ + struct mlx5_dev_ctx_shared *sh = cb_arg; + uint32_t i; + uint32_t if_index; + + if (mlx5_nl_parse_link_status_update(hdr, &if_index) < 0) + return; + for (i = 0; i < sh->max_port; i++) { + struct mlx5_dev_shared_port *port = &sh->port[i]; + struct rte_eth_dev *dev; + struct mlx5_priv *priv; + + if (port->nl_ih_port_id >= RTE_MAX_ETHPORTS) + continue; + dev = &rte_eth_devices[port->nl_ih_port_id]; + /* Probing may initiate an LSC before configuration is done. */ + if (dev->data->dev_configured && + !dev->data->dev_conf.intr_conf.lsc) + break; + priv = dev->data->dev_private; + if (priv->if_index == if_index) { + /* Block logical LSC events. */ + uint16_t prev_status = dev->data->dev_link.link_status; + + if (mlx5_link_update(dev, 0) < 0) + DRV_LOG(ERR, "Failed to update link status: %s", + rte_strerror(rte_errno)); + else if (prev_status != dev->data->dev_link.link_status) + rte_eth_dev_callback_process + (dev, RTE_ETH_EVENT_INTR_LSC, NULL); + break; + } + } +} + +void +mlx5_dev_interrupt_handler_nl(void *arg) +{ + struct mlx5_dev_ctx_shared *sh = arg; + int nlsk_fd = rte_intr_fd_get(sh->intr_handle_nl); + + if (nlsk_fd < 0) + return; + if (mlx5_nl_read_events(nlsk_fd, mlx5_dev_interrupt_nl_cb, sh) < 0) + DRV_LOG(ERR, "Failed to process Netlink events: %s", + rte_strerror(rte_errno)); +} + /** * Handle shared asynchronous events the NIC (removal event * and link status change). Supports multiport IB device. @@ -823,18 +874,6 @@ mlx5_dev_interrupt_handler(void *cb_arg) tmp = sh->port[tmp - 1].ih_port_id; dev = &rte_eth_devices[tmp]; MLX5_ASSERT(dev); - if ((event.event_type == IBV_EVENT_PORT_ACTIVE || - event.event_type == IBV_EVENT_PORT_ERR) && - dev->data->dev_conf.intr_conf.lsc) { - mlx5_glue->ack_async_event(&event); - if (mlx5_link_update(dev, 0) == -EAGAIN) { - usleep(0); - continue; - } - rte_eth_dev_callback_process - (dev, RTE_ETH_EVENT_INTR_LSC, NULL); - continue; - } DRV_LOG(DEBUG, "port %u cannot handle an unknown event (type %d)", dev->data->port_id, event.event_type); diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 17e7144cc9..29a4890d14 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -2495,6 +2495,40 @@ mlx5_os_net_cleanup(void) mlx5_pmd_socket_uninit(); } +static int +mlx5_os_dev_shared_handler_install_lsc(struct mlx5_dev_ctx_shared *sh) +{ + int nlsk_fd, flags, ret; + + nlsk_fd = mlx5_nl_init(NETLINK_ROUTE, RTMGRP_LINK); + if (nlsk_fd < 0) { + DRV_LOG(ERR, "Failed to create a socket for Netlink events: %s", + rte_strerror(rte_errno)); + return -1; + } + flags = fcntl(nlsk_fd, F_GETFL); + ret = fcntl(nlsk_fd, F_SETFL, flags | O_NONBLOCK); + if (ret != 0) { + DRV_LOG(ERR, "Failed to make Netlink event socket non-blocking: %s", + strerror(errno)); + rte_errno = errno; + goto error; + } + rte_intr_type_set(sh->intr_handle_nl, RTE_INTR_HANDLE_EXT); + rte_intr_fd_set(sh->intr_handle_nl, nlsk_fd); + if (rte_intr_callback_register(sh->intr_handle_nl, + mlx5_dev_interrupt_handler_nl, + sh) != 0) { + DRV_LOG(ERR, "Failed to register Netlink events interrupt"); + rte_intr_fd_set(sh->intr_handle_nl, -1); + goto error; + } + return 0; +error: + close(nlsk_fd); + return -1; +} + /** * Install shared asynchronous device events handler. * This function is implemented to support event sharing @@ -2532,6 +2566,18 @@ mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh) rte_intr_fd_set(sh->intr_handle, -1); } } + sh->intr_handle_nl = rte_intr_instance_alloc + (RTE_INTR_INSTANCE_F_SHARED); + if (sh->intr_handle_nl == NULL) { + DRV_LOG(ERR, "Fail to allocate intr_handle"); + rte_errno = ENOMEM; + return; + } + rte_intr_fd_set(sh->intr_handle_nl, -1); + if (mlx5_os_dev_shared_handler_install_lsc(sh) < 0) { + DRV_LOG(INFO, "Fail to install the shared Netlink event handler."); + rte_intr_fd_set(sh->intr_handle_nl, -1); + } if (sh->cdev->config.devx) { #ifdef HAVE_IBV_DEVX_ASYNC sh->intr_handle_devx = @@ -2579,10 +2625,19 @@ mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh) void mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh) { + int nlsk_fd; + if (rte_intr_fd_get(sh->intr_handle) >= 0) mlx5_intr_callback_unregister(sh->intr_handle, mlx5_dev_interrupt_handler, sh); rte_intr_instance_free(sh->intr_handle); + nlsk_fd = rte_intr_fd_get(sh->intr_handle_nl); + if (nlsk_fd >= 0) { + mlx5_intr_callback_unregister + (sh->intr_handle_nl, mlx5_dev_interrupt_handler_nl, sh); + close(nlsk_fd); + } + rte_intr_instance_free(sh->intr_handle_nl); #ifdef HAVE_IBV_DEVX_ASYNC if (rte_intr_fd_get(sh->intr_handle_devx) >= 0) rte_intr_callback_unregister(sh->intr_handle_devx, diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 74841caaf9..09cd1367db 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1457,6 +1457,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, for (i = 0; i < sh->max_port; i++) { sh->port[i].ih_port_id = RTE_MAX_ETHPORTS; sh->port[i].devx_ih_port_id = RTE_MAX_ETHPORTS; + sh->port[i].nl_ih_port_id = RTE_MAX_ETHPORTS; } if (sh->cdev->config.devx) { sh->td = mlx5_devx_cmd_create_td(sh->cdev->ctx); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 9a4aff97cb..0f0045a2b5 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -637,6 +637,7 @@ struct mlx5_age_info { struct mlx5_dev_shared_port { uint32_t ih_port_id; uint32_t devx_ih_port_id; + uint32_t nl_ih_port_id; /* * Interrupt handler port_id. Used by shared interrupt * handler to find the corresponding rte_eth device @@ -1239,6 +1240,7 @@ struct mlx5_dev_ctx_shared { /* Shared interrupt handler section. */ struct rte_intr_handle *intr_handle; /* Interrupt handler for device. */ struct rte_intr_handle *intr_handle_devx; /* DEVX interrupt handler. */ + struct rte_intr_handle *intr_handle_nl; /* Netlink interrupt handler. */ void *devx_comp; /* DEVX async comp obj. */ struct mlx5_devx_obj *tis[16]; /* TIS object. */ struct mlx5_devx_obj *td; /* Transport domain. */ @@ -1666,6 +1668,7 @@ int mlx5_dev_set_flow_ctrl(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf); void mlx5_dev_interrupt_handler(void *arg); void mlx5_dev_interrupt_handler_devx(void *arg); +void mlx5_dev_interrupt_handler_nl(void *arg); int mlx5_set_link_down(struct rte_eth_dev *dev); int mlx5_set_link_up(struct rte_eth_dev *dev); int mlx5_is_removed(struct rte_eth_dev *dev); diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index fe8b42c414..c68b32cf14 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -1207,11 +1207,18 @@ mlx5_dev_start(struct rte_eth_dev *dev) priv->sh->port[priv->dev_port - 1].ih_port_id = (uint32_t)dev->data->port_id; } else { - DRV_LOG(INFO, "port %u starts without LSC and RMV interrupts.", + DRV_LOG(INFO, "port %u starts without RMV interrupts.", dev->data->port_id); - dev->data->dev_conf.intr_conf.lsc = 0; dev->data->dev_conf.intr_conf.rmv = 0; } + if (rte_intr_fd_get(priv->sh->intr_handle_nl) >= 0) { + priv->sh->port[priv->dev_port - 1].nl_ih_port_id = + (uint32_t)dev->data->port_id; + } else { + DRV_LOG(INFO, "port %u starts without LSC interrupts.", + dev->data->port_id); + dev->data->dev_conf.intr_conf.lsc = 0; + } if (rte_intr_fd_get(priv->sh->intr_handle_devx) >= 0) priv->sh->port[priv->dev_port - 1].devx_ih_port_id = (uint32_t)dev->data->port_id; @@ -1263,6 +1270,7 @@ mlx5_dev_stop(struct rte_eth_dev *dev) mlx5_rx_intr_vec_disable(dev); priv->sh->port[priv->dev_port - 1].ih_port_id = RTE_MAX_ETHPORTS; priv->sh->port[priv->dev_port - 1].devx_ih_port_id = RTE_MAX_ETHPORTS; + priv->sh->port[priv->dev_port - 1].nl_ih_port_id = RTE_MAX_ETHPORTS; mlx5_txq_stop(dev); mlx5_rxq_stop(dev); if (priv->obj_ops.lb_dummy_queue_release) From patchwork Tue Mar 1 12:15:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 108435 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 56353A0353; Tue, 1 Mar 2022 13:16:18 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 16C794270A; Tue, 1 Mar 2022 13:16:06 +0100 (CET) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam07on2086.outbound.protection.outlook.com [40.107.212.86]) by mails.dpdk.org (Postfix) with ESMTP id 286FD42701; Tue, 1 Mar 2022 13:16:03 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BRlQhimAEGKezNr2HRkKWDPleP9yO3Isc1yTO16PequJrhXxCmpbtvhuukY/GaVGbK9uwz57hGgge8KdCnzN1nOfcqZY0wqlC/9iRXCKCW1E5NDw0lcYjNmiu/qCzKKAxQDHhgzAit9dKLB6geskYaHuGYvQm0eeWHzJeML3Hi5pkqyBHdB2qi/INoZLYSHABEFDTBSbSX44DwUWWczxNJJ7OCLOGxrtT4Fl1dpvsL8K0JQdJxB+MX3ke1jBSsvfyCF2P+8TfJHHxkQDNhxeGlTXUinLXWF1QaSO/Ws5ilU4B73JOqc6oivSgsnb7pvDloz8NXUomPFp0HsY/JXvzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N8CSGgIMuvLSiqb9r5A49CXR/mHz8YBdA9G3vgJBpME=; b=ZudWuklgl32k1bcAyltZ0ODZSQX9HjWbwdYnfBorHKWeYmsyWPJVWSTOw3Gg6+9UJ/CGEmY3sEtOW2+CrZ0JOQlkxWmdqKZlTcy+hBytBdc0RrqNGAcNa99RvY57x2wxpxC1di43A7Bi8iSCJIzCn1jCFdiqtrobYXsYYARCb4vrKGXZmFipWz8K5Ea/sG3qDKBj/AxuE+D+eKo0Aa6NHT1f5wxjpFERm5fcK/Lw4d0pjzp9IxweRl6+ZXc6s6jwOAqLqnKnhmHZm39vRwNcMfDZz/crOhrfGyJUJTzbCFJVm1qKBHzBw//l18bb8d6Z3lC7T+HHtDaI7kypRaYmKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N8CSGgIMuvLSiqb9r5A49CXR/mHz8YBdA9G3vgJBpME=; b=HIWF1vccG/qGzMkgHXRsMxZ49g6V2J1nDIdXe9GpGiKYvvJf/fDpEx1flHUC3w7ebg/Qr6X4woJ1F6VK60+a/r/gJMdmX5VSRxhx8qRpRHDhLpvtX2dWhAtH5mkTkxY2WH6a5lwfNIttWth4mOdKDI+Tbiio6mzAmA6mw3bWFH/PBpPxK73V04T240C2SHUzlz2d2klbjODEhbnv8gNCAuocsabdfQ5OCOOAuZDs/Hj4RL/7ZLNw0CJg6TlE3Oj4RDDFgqZhuI8jzIm3LXV8J+o1a094QCnjV5lGVr+vAzeUl4YPxDBKy8wifOJ2iT+PHNV43OJyY+IANXfuz374Yg== Received: from MW4PR03CA0022.namprd03.prod.outlook.com (2603:10b6:303:8f::27) by MW3PR12MB4427.namprd12.prod.outlook.com (2603:10b6:303:52::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22; Tue, 1 Mar 2022 12:16:01 +0000 Received: from CO1NAM11FT017.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8f:cafe::81) by MW4PR03CA0022.outlook.office365.com (2603:10b6:303:8f::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.14 via Frontend Transport; Tue, 1 Mar 2022 12:16:00 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT017.mail.protection.outlook.com (10.13.175.108) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Tue, 1 Mar 2022 12:16:00 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 1 Mar 2022 12:16:00 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Tue, 1 Mar 2022 04:15:58 -0800 Received: from nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Tue, 1 Mar 2022 04:15:57 -0800 From: Dmitry Kozlyuk To: CC: , Viacheslav Ovsiienko , "Matan Azrad" Subject: [PATCH v2 3/3] net/mlx5: fix initial link status detection Date: Tue, 1 Mar 2022 14:15:14 +0200 Message-ID: <20220301121514.41497-4-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220301121514.41497-1-dkozlyuk@nvidia.com> References: <20220223164333.3834590-1-dkozlyuk@nvidia.com> <20220301121514.41497-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 23c39c19-dddb-413a-e9fd-08d9fb7d3fbd X-MS-TrafficTypeDiagnostic: MW3PR12MB4427:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F5kSW6jZjNS7yfRAIPPBMyWnA+w6K3gjSxGfvpvo8ukXVpNwPkHc4a35k6YWMzom2tmP12Ww7e5oZey1qFt3A3cSaZLZOFAMrBHekli6aS8z5god2tH9tnwf/GOayyUWtt4vQxv8heFFXIyANHljg44i7hyiIwQWah6sQhZIAXaQHRqoQSIJ8qA5MPpob3FByuJX5Kp/0/iyJXbFrJAlsppbaxMXKzbMfHMopJKlrwwt3SHwir1WpE8V29aOzZUPgJfI0yoxHBBhjTZTQhhTCEiUrzPojkdPM6kfPVKIK592wpMUIfdoIhdltC4XdYPTh5fTbR5m17vuIyAxbHuDR/mWa/whBUgXK/my5pbJvZL1+IY+zeOuHJGzyEm6gcDrlxmmGpxZFL2JtAElkftjSQZszazHzKK4YFzEIP+8CN9KNx0KQOcC0h9oroS14K4Ul0+eXUkt74BcCqllsSVb0x2Q2te08wjt26jKMY4Usr6vwQCyw3YeV9RcaCqurMnaUJ9HLw9TN6zcKzmInPY6Njlf0RB826hCqEo9G1VRPnQRx2jtD7VsgYRMwvyM7OUpkIYFfaEnylId2e0wNItyFxypGVju1ka6YhKrNDrJ+wM4drPiipQMJhAuVKzNsw+YfVECPS1QGYR5EMPszQ0Rf6GQHocdEIuGMZa7LZuvDpgt05CzRmTE58ojNe7yVFIzvxuUpsDXaE1jrsg6ApZnp4Dk4vGFZR0pX5ud2vw/y94= X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(70586007)(70206006)(450100002)(36860700001)(36756003)(6916009)(81166007)(316002)(2906002)(54906003)(356005)(82310400004)(4326008)(508600001)(6666004)(83380400001)(86362001)(7696005)(8936002)(5660300002)(8676002)(55016003)(40460700003)(107886003)(186003)(336012)(426003)(1076003)(26005)(2616005)(6286002)(47076005)(36900700001)(309714004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Mar 2022 12:16:00.7280 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 23c39c19-dddb-413a-e9fd-08d9fb7d3fbd X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT017.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4427 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Link status change takes time that depends on the HW and the kernel. It was checked immediately after the change was issued at probing. If the port had beed down before probing, a "down" state may be read, while the port would be "up" imminently. After that, DPDK reported the port as "down" mistakenly and "ifconfig $DEV up" did not trigger an LSC event, because from the system's perspective the port was "up" already. Install Netlink event handler at port probe before requesting the port to come up in order to receive LSC event even if it comes up between probe and start. Fixes: b6499434b83e ("net/mlx5: fix link status initialization") Cc: stable@dpdk.org Signed-off-by: Dmitry Kozlyuk Reviewed-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 29a4890d14..ff65efb2a2 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1483,13 +1483,12 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, /* Bring Ethernet device up. */ DRV_LOG(DEBUG, "port %u forcing Ethernet interface up", eth_dev->data->port_id); - mlx5_set_link_up(eth_dev); - /* - * Even though the interrupt handler is not installed yet, - * interrupts will still trigger on the async_fd from - * Verbs context returned by ibv_open_device(). - */ + /* Read link status in case it is up and there will be no event. */ mlx5_link_update(eth_dev, 0); + /* Watch LSC interrupts between port probe and port start. */ + priv->sh->port[priv->dev_port - 1].nl_ih_port_id = + eth_dev->data->port_id; + mlx5_set_link_up(eth_dev); for (i = 0; i < MLX5_FLOW_TYPE_MAXI; i++) { icfg[i].release_mem_en = !!sh->config.reclaim_mode; if (sh->config.reclaim_mode)