From patchwork Wed Nov 9 22:18:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 119666 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 89A2BA0093; Wed, 9 Nov 2022 23:18:22 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6ECE9400EF; Wed, 9 Nov 2022 23:18:22 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2045.outbound.protection.outlook.com [40.107.93.45]) by mails.dpdk.org (Postfix) with ESMTP id 70C04400D4; Wed, 9 Nov 2022 23:18:20 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b/+0I5tk+OXJkN27UP2J4cqXrUzwqZ+dXpxMMC3iZTfbK+/VRFJedhsjpjhLOMh3FhMR5CjKKDC3EJpCXVRToXRnsLYW7AeKMGuhWrDSGfIDCLkRKm2kZlRGqw9JCMLRbbM2limAFF3lmvgTx/dY0+zWAmFYMQqc5uX2LxzV06ygG1WRHx+PYsIoDU9g+3E7OTbvonmZpjtrWxE8mVncQOT7MFVI49DOhVkpGWQ/9vF3XhTnnQqV8as1yuMFAoM8tm50kDUtcYdCwSTkGKCYejB/vQO+FRTfyvct1NT/mYhIOtPpRIGtaQEnRnk25zUNXXLiggh1jzq9jdRyUJaztw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LhxsvRTKFYz1iwJy2ZlJCoLb/G/7/kdMDPIZ8lrJzo4=; b=Napmw8Ngh2xCZZbZs1QT8/OzhbXbE0qM69EGqXencRZOqHkYEB4yRuYyvGdSze793jYSXVaS4jP6t52ZPiEJr+XAmyoLXkFEfff2UB4I2hbAwRKIdlBumTdqZzES4AZWiNgZQn3Lx6wR4wNcgDxYK+Q+9kXpr55rDcN6uhTlxh3lkyS4mikZFsXMG3oPlcrYpCO0xwuoLAznrsm4+QZHybvOex93qDMiFNsx5qR2RSKThG9DqYAjc5u33mXpVn+tI65zNs2sowScEw1CEPv++n4/wtMkTBZREFQK1sLqmsxm54uf0fjadxgWvnfb+aaUfaABO+FzO0JeRfFgyVyx1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LhxsvRTKFYz1iwJy2ZlJCoLb/G/7/kdMDPIZ8lrJzo4=; b=WeJBOJnBf6N4XhwFK+rgJ6Ag6QArkw9BuyCV5qia+C4BCzb84DibVPSefoUiLfGUl9Txn/tWfQQqNNpQnG9tS9Q+gnDLWMaMQBg1qOvdbdPVb1aMtaFre9gnlM4mIjFKbrFa6NsVq7Gpn6FOV/k3kPhdh2WYFBiD8n6MMq5YbMT8lBIByAFkdxn8jBAsgLiQH45fFXSlAg50JTDpcQoVzQZixNcIMv5W1JTwR+xaksu2Qb+uxmCrOjxRDgGG1IXJbRtQRkQB1qLBuH54q9li23pwEx5t8Rm+nmocVtHBg/7ESjmAN6ZuADxrvck1qQvRXUpyWqifSFFv4AtB/BTxIA== Received: from BN9PR03CA0130.namprd03.prod.outlook.com (2603:10b6:408:fe::15) by BL1PR12MB5096.namprd12.prod.outlook.com (2603:10b6:208:316::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.27; Wed, 9 Nov 2022 22:18:18 +0000 Received: from BN8NAM11FT097.eop-nam11.prod.protection.outlook.com (2603:10b6:408:fe:cafe::b5) by BN9PR03CA0130.outlook.office365.com (2603:10b6:408:fe::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.12 via Frontend Transport; Wed, 9 Nov 2022 22:18:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN8NAM11FT097.mail.protection.outlook.com (10.13.176.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.12 via Frontend Transport; Wed, 9 Nov 2022 22:18:18 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Wed, 9 Nov 2022 14:18:10 -0800 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 9 Nov 2022 14:18:10 -0800 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36 via Frontend Transport; Wed, 9 Nov 2022 14:18:08 -0800 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , , Subject: [PATCH] net/mlx5: fix port's event cleaning order Date: Thu, 10 Nov 2022 00:18:02 +0200 Message-ID: <20221109221802.264768-1-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT097:EE_|BL1PR12MB5096:EE_ X-MS-Office365-Filtering-Correlation-Id: 99ce4ab2-ff58-49a2-447c-08dac2a04e19 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YxNCqdnnfqa8con5C2XsKfweD1bnwz3AdNeJdiDnzRPv1MxkLX6iQFPH+7L7xrqLaGycjWqq3sj0I7vBCRcJDDK/VTRikjqHwAu1d4wEfQOR8zgbWqqSpwoOUhC/GRS+pjkwT42prllTbRIgXqMJHTggGMh6QgS/NPJk4ED2QdB2TQUpuK6FVSXoWjt6fy534Fpm7QzsAT0fZFOC7sDvEYQXzHpeQM0K0y+QCdyU6UTxpYMv3mBLuEMif0ctUzQBCDmNsrU1H2Ww5mDwcnM8QqtkASIjODreu0CINOc0fYKlVqv1kamW6BHTCFobaKsYXDgXg05Ip4wkHAvVgb+2k8x8GvukYX0cbQdP54NNMJmktqE8LONHlhKrDtDfeO3Iv2T4T7Q+4k0RQfl/IL4mGc2BHCG/BSrXIu9E+2kaMl+Zc4rRjyWjsRHl34VqW9cG43MQ2cv/1ryMoPiPKa42MWnISsrG/ChAoulGFludAVr1GtYP0Mf7xlNENYDnkXNpJB9vSoOxtKjjBXiPPm1rB8hLtKXlL9/ckWBq/uBDCyVTzt1utcHlIzIg0FbHvttuO1sjfZkaw/bCaFG/X7Pi32zfRDskA8YpJcC9Di0GL2zweEjSkGC71we+wndKgXif3eHg+IkrJjOsRaqYYF1Uq/+iWoQ0hANWsdWNxvZCOWgNS1NrZ83+JsUXeFL5T9BXADT/N5qXUehIhH8T6/ynjWro9hYJrdfEpIk1XKMN8hXLsGxoCN0Hyo0yE15hi9Q7UfAOMkrzIbjbvN3N+V7WxA== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(346002)(136003)(396003)(39860400002)(376002)(451199015)(40470700004)(46966006)(36840700001)(40480700001)(86362001)(450100002)(8936002)(55016003)(316002)(70206006)(70586007)(54906003)(6916009)(41300700001)(5660300002)(2906002)(36860700001)(47076005)(2616005)(186003)(1076003)(478600001)(336012)(82310400005)(4326008)(26005)(40460700003)(36756003)(6286002)(426003)(7696005)(8676002)(83380400001)(356005)(82740400003)(6666004)(7636003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2022 22:18:18.5150 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 99ce4ab2-ff58-49a2-447c-08dac2a04e19 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT097.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5096 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The shared IB device (sh) has per port data with filed for interrupt handler port_id. It used by shared interrupt handler to find the corresponding rte_eth device by IB port index. If value is equal or greater RTE_MAX_ETHPORTS it means there is no subhandler installed for specified IB port index. When a few ports are created under same sh, the sh is created with the first port and the interrupt handler port_id is initialized to RTE_MAX_ETHPORTS for each port. In port creation, the interrupt handler port_id is updated with the correct value. Since this updating, the mlx5_dev_interrupt_nl_cb function uses this port and its priv structure. However, when the ports are closed, this filed isn't updated and the interrupt handler continue working untill it is uninstalled in SH destruction. If mlx5_dev_interrupt_nl_cb is called between port closing and SH destruction, it uses invalid port causing a crash. This patch adds interrupt handler port_id updating to the close function and add memory barrier to make sure it is done before priv reset. Fixes: 655c3c26c11e ("net/mlx5: fix initial link status detection") Cc: dkozlyuk@nvidia.com Cc: stable@dpdk.org Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/linux/mlx5_os.c | 3 +++ drivers/net/mlx5/mlx5.c | 6 ++++++ 2 files changed, 9 insertions(+) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 2b6741396d..a71474c90a 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1676,6 +1676,9 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, return eth_dev; error: if (priv) { + priv->sh->port[priv->dev_port - 1].nl_ih_port_id = + RTE_MAX_ETHPORTS; + rte_io_wmb(); #ifdef HAVE_MLX5_HWS_SUPPORT if (eth_dev && priv->sh && diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 1cf6df6049..95b0151fbc 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -2137,6 +2137,12 @@ mlx5_dev_close(struct rte_eth_dev *dev) if (!c) claim_zero(rte_eth_switch_domain_free(priv->domain_id)); } + priv->sh->port[priv->dev_port - 1].nl_ih_port_id = RTE_MAX_ETHPORTS; + /* + * The interrupt handler port id must be reset before priv is reset + * since 'mlx5_dev_interrupt_nl_cb' uses priv. + */ + rte_io_wmb(); memset(priv, 0, sizeof(*priv)); priv->domain_id = RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID; /*