From patchwork Fri Jun 30 12:41:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bing Zhao X-Patchwork-Id: 129162 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 59EC642D95; Fri, 30 Jun 2023 14:42:18 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E29D040EDB; Fri, 30 Jun 2023 14:42:17 +0200 (CEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2042.outbound.protection.outlook.com [40.107.220.42]) by mails.dpdk.org (Postfix) with ESMTP id 19AB9406B6; Fri, 30 Jun 2023 14:42:17 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GTV5+JUbtumVEH+ngUln76+DuuMubLXPcl4UF3cyYamkxgdCCy6AgV7KBblgNFGPG/Qppio6W6C3VS97o1U/SzWz/wmso4qcT/QB7Y9uXKIW9+XG52iKJ4KLOzzwaakISocuJus/AyYBujOQzyJYQKLBRYikn9/8D0nxzFoY/7Y4Ybe9njpeOcWnyYfte4YlPgyHcFuVQXNLdWtg08uorJZ+Ai9HbIAHGEfwhlEEmzhdL3W1BuL6qJ5mQ3ZTQuE5Mt9/IpMvw2gOPYNgFrJbLrOMZj14MP+688nPX1Rdf6TXIU0f0XUGvtQaomTGiIV1IH/6HSvOwjpXA8XH+Fh7jA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=J/1Qd488x+3OrQxwZTMAR8aKvYt9rhH2iLibamuwN38=; b=GPE245RxVH4hnOE4E1CP8hTc2BoxhOCvwx0bn8gd/vVhPSIMdBhvpXn6HqfSzffl9mDannz95wB63eoNEnIXaZii2dYJRUOTDm3TxekrXzqeuQ5HtUehPg2299kDVqC4ERgPdrRF+cX8Ufope1W9/CBK7AUZAp7HLj69QO+JAxxg5+r0GL/CAp1+1pZ2kS1vExf42M57QTnJ3AZmTUNPCb+DllFmd9xCzXzApk/DIHEVyxR8i5EaMZTRjRLuiu1G78dy0GbJfNj68QaN9Oya8jTwIsAULrQP9bimCLP1sfOisMMG9y9oukiHtNGq7gPB7iGnNtbix6kMVYyo5O/10Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J/1Qd488x+3OrQxwZTMAR8aKvYt9rhH2iLibamuwN38=; b=CoKJIpmbyZ0OkP+1m/69yMphIDbfH/RUxGqYZ63ee+Wb7uLPExOiSI1ANVu3/5jmJXQvHRXCG7/T8mzv082Tm8qfciQnsttc4KVuLDFGwA1/GJlMCJGNUP5RxTO3K3RgRV0N1BytAheL1OvEXlUI15uWcnqIsICLLFajAUKcmSRSiWfsu4jY7rDA/LCpjv60XyJuMZMDeFaUwA8cdtOBjSWGwznThtXHxieOrdp+tox8a27tAHwybWit68Z9w6rIo66lT3prHTqXEqPawzG6/cVbjGy0T2vsNtodDZdzN5wR2DLIo6By4d5vgPjAg8/QRgDI/+OZa0aXsWy+kr/m/g== Received: from BN0PR04CA0091.namprd04.prod.outlook.com (2603:10b6:408:ec::6) by DM4PR12MB6421.namprd12.prod.outlook.com (2603:10b6:8:b7::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.24; Fri, 30 Jun 2023 12:42:15 +0000 Received: from BN8NAM11FT059.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ec:cafe::50) by BN0PR04CA0091.outlook.office365.com (2603:10b6:408:ec::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.22 via Frontend Transport; Fri, 30 Jun 2023 12:42:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT059.mail.protection.outlook.com (10.13.177.120) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.43 via Frontend Transport; Fri, 30 Jun 2023 12:42:14 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Fri, 30 Jun 2023 05:42:02 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Fri, 30 Jun 2023 05:42:00 -0700 From: Bing Zhao To: , , , , CC: , , Subject: [PATCH] common/mlx5: fix obtaining IB device in LAG mode Date: Fri, 30 Jun 2023 15:41:39 +0300 Message-ID: <20230630124139.435300-1-bingz@nvidia.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT059:EE_|DM4PR12MB6421:EE_ X-MS-Office365-Filtering-Correlation-Id: 12d4a7a5-3e3b-4131-d6c3-08db79676eaf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: afOc6jlXHBEXwfXet00WTBEpNkQ12qe96VZjuex1GIfS3pkQDPRF7O2PcSwKjGJRjEJIgv+fNGBwXkU8UMge86o/4VNzEWrUGowkExJOFjG+3rcB3nwxV3KHDKAIjXOtMxmIo8UTGFyRzxIjCtZnbHKf0c6+GhTTLOPGxOAJqySv53tBOV2Bt1QfBgkUu3UnKAy9UXwOHoNRozEH/PSV2zQ1gJCRM8AN0voQYsin/8MESFmfdWfyHz6IltWLY7rQUGmfqFM8+hIjX88OCjBcMENrJuYSOoxhYSaHQVhYcWcvB7fmtrpT29pXupCDhLq/slkFr6AVXRiCABrt8glfRsYyG8SYcasd8FkBtBVckP06WaiznK2IhWveFYqv5anO5Bi/PGH1kjGj+NcOJrimqU9jtq7pxkdkjD5nM77SAnJdWFCefOS8l4TYubiUFxzar46LJWL4TZqc2yEnMovtXLf/sXEr7qGCI902Ex06GhjSjUiXKicLEha6hKXeiDz57tR/4g5AJpAn+akcHLqiWFxSR0YsN/jUh6KCqqe++5h4v82leJKFaZXPEbziA0TGdavHDWYRZzP2cbYI+dOlcaCJKFf1nCVwHiDoiAyWAyLUTVHyMYd85rEs3iEXe0woaIudW/1t5a0OF3/2IYW+2Za5b2PiTkAsaFAdK4PZwEPnS7qEFS0TEcR7V2D38qpUh2W2aTAj9MmDSKnRIzcRX4TKu/i7+V27nmRRX80ddnCn6c21Ce54/4Nro2EgHQSb X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(136003)(396003)(39860400002)(376002)(451199021)(36840700001)(40470700004)(46966006)(70206006)(70586007)(316002)(6636002)(4326008)(478600001)(8936002)(450100002)(36756003)(8676002)(5660300002)(40460700003)(6286002)(6666004)(47076005)(1076003)(55016003)(36860700001)(86362001)(40480700001)(2906002)(54906003)(110136005)(41300700001)(82310400005)(7696005)(336012)(186003)(426003)(16526019)(356005)(83380400001)(26005)(82740400003)(2616005)(7636003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jun 2023 12:42:14.6635 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 12d4a7a5-3e3b-4131-d6c3-08db79676eaf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT059.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6421 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In hardware LAG mode, both PFs are in the same E-Switch domain but the VFs are in the other domains. Moreover, VF has its own dedicated IB device. When probing a VF created on the 1st PF, usually its PCIe address is the same as the PF's except the function part. Then there would be some wrong VF BDF match on the IB "bond" device due to incomplete comparison (we do not compare the function part of BDF for bonding devices to match all bonded PFs). Adding one extra condition to check whether the current PCIe address device is a VF will solve the incorrect IB device recognition. Thus the full address comparison will be done. Fixes: f956d3d4c33c ("net/mlx5: fix probing with secondary bonding member") Cc: rongweil@nvidia.com Cc: stable@dpdk.org Signed-off-by: Bing Zhao Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/mlx5_common_os.c | 16 +++++++++------- drivers/common/mlx5/mlx5_common.h | 2 +- drivers/common/mlx5/mlx5_common_pci.c | 2 +- 3 files changed, 11 insertions(+), 9 deletions(-) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index aafff60eeb..2ebb8ac8b6 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -555,7 +555,7 @@ mlx5_os_pd_prepare(struct mlx5_common_device *cdev) } static struct ibv_device * -mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) +mlx5_os_get_ibv_device(const struct rte_pci_device *pci_dev) { int n; struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); @@ -564,6 +564,8 @@ mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) uint8_t guid2[32] = {0}; int ret1, ret2 = -1; struct rte_pci_addr paddr; + const struct rte_pci_addr *addr = &pci_dev->addr; + bool is_vf_dev = mlx5_dev_is_vf_pci(pci_dev); if (ibv_list == NULL || !n) { rte_errno = ENOSYS; @@ -579,11 +581,11 @@ mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) if (ret1 > 0) ret2 = mlx5_get_device_guid(&paddr, guid2, sizeof(guid2)); /* Bond device can bond secondary PCIe */ - if ((strstr(ibv_list[n]->name, "bond") && - ((ret1 > 0 && ret2 > 0 && !memcmp(guid1, guid2, sizeof(guid1))) || - (addr->domain == paddr.domain && addr->bus == paddr.bus && - addr->devid == paddr.devid))) || - !rte_pci_addr_cmp(addr, &paddr)) { + if ((strstr(ibv_list[n]->name, "bond") && !is_vf_dev && + ((ret1 > 0 && ret2 > 0 && !memcmp(guid1, guid2, sizeof(guid1))) || + (addr->domain == paddr.domain && addr->bus == paddr.bus && + addr->devid == paddr.devid))) || + !rte_pci_addr_cmp(addr, &paddr)) { ibv_match = ibv_list[n]; break; } @@ -697,7 +699,7 @@ mlx5_os_get_ibv_dev(const struct rte_device *dev) struct ibv_device *ibv; if (mlx5_dev_is_pci(dev)) - ibv = mlx5_os_get_ibv_device(&RTE_DEV_TO_PCI_CONST(dev)->addr); + ibv = mlx5_os_get_ibv_device(RTE_DEV_TO_PCI_CONST(dev)); else ibv = mlx5_get_aux_ibv_device(RTE_DEV_TO_AUXILIARY_CONST(dev)); if (ibv == NULL) { diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 42d938776a..28f9f41528 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -600,7 +600,7 @@ mlx5_dev_is_pci(const struct rte_device *dev); */ __rte_internal bool -mlx5_dev_is_vf_pci(struct rte_pci_device *pci_dev); +mlx5_dev_is_vf_pci(const struct rte_pci_device *pci_dev); __rte_internal int diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 5122c596bc..04aad0963c 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -109,7 +109,7 @@ mlx5_dev_is_pci(const struct rte_device *dev) } bool -mlx5_dev_is_vf_pci(struct rte_pci_device *pci_dev) +mlx5_dev_is_vf_pci(const struct rte_pci_device *pci_dev) { switch (pci_dev->id.device_id) { case PCI_DEVICE_ID_MELLANOX_CONNECTX4VF: