From patchwork Tue Aug 17 13:44:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 96998 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 55BAAA0548; Tue, 17 Aug 2021 15:46:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9C6E5411E7; Tue, 17 Aug 2021 15:45:31 +0200 (CEST) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam08on2083.outbound.protection.outlook.com [40.107.101.83]) by mails.dpdk.org (Postfix) with ESMTP id 2D8D6411DB for ; Tue, 17 Aug 2021 15:45:29 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VpC93uwiqV7zxuPXNW69VEOT7xAn68daqpnGf08HEB936YwMulSHQ2mRT/fv3pDrudifoVxbkI9XUDvwXJM9bsls9OxNdoZDPLPJO1XzSGKQjRVVnwnID/fSNkwbx5fuPMtFvCZ/8kjOceBVFHX7cK970Udv3gpBUCOJ2FWy4VEO0d0Gl+blosJM4R/pPGdAB8eGzqBKwq0Y540TAVRvhBX9enoweyeNqtAzsQ15/qhrqBXGOdbi9uRsSWvxjGKeZu5tO3IU/waSYPRtGUfUqQ/RckwWcOr+tXkrhWdRW1kR3OlQnO1S72B5knfefpxjpxE7SG7cyAHyNoxiEjGUZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tj9ETY/JgCvGGUw3bG4ji/wbs6BDHl3IRcuTGylCPVc=; b=gCQuDKWQm5oEhYKTiBPf3d6B1SNOPb3NkRK5UTNEz2f1Mk3oOuC2QVsjxTIXCWbGZ5xoqzl/Xf8femKDCKnRxbofK6+3wFGs1n2VOAkUrbD2gxDBlkcvsGqNV9Dso62B7kUIwYyVVZ79UcrBpKWQcKHfvDDSiNo8bw4QAgPpCstq3rK/w/NTP1J3yNwVYxDhmLDSMspwWPBLkXjr1yE+Yea83gPFbY4L+OKCCJyckDEtGjeR7hPI9oKljoDvmtzoMohsPwCwtdB1VFICU/+pGigK8L0onMvT7LJkYde2ea9Z/3cGjSDACi65/XaRxboIBQEgprulgKMhF34JJ7druQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tj9ETY/JgCvGGUw3bG4ji/wbs6BDHl3IRcuTGylCPVc=; b=T19MDtcQIfk2AsP6fStE91tGVEUJMufWbHocS+D9ibgxU97v+ZvnlNSSn3dWkWhB2D0ZkA9m/6mbIJoYk5y8+rTWY1msaRj8NAQOc0lUV65Xba6TFeDDbfSTDXLMSjtAIgvRKJUXGhFCixW9KYlszzgI+lsN1/lQeU0izwYU1oBQImpSsLFWd7DGI+YB3ou7gs4zwStZFfHGQqy3uK+humKNCJwBROWojlXEO4h4gDTcM3u/lyCftEE/hI6CwTkaYFbe3dXNsZjdJ7th7Cvt7hdlTKsAUijuwq7hPJvSCX3r7lVe5nyv4DemK+JuPuSLYdHn5bE7SsgLhj9vUVyJ6A== Received: from BN0PR04CA0118.namprd04.prod.outlook.com (2603:10b6:408:ec::33) by BN9PR12MB5066.namprd12.prod.outlook.com (2603:10b6:408:133::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.17; Tue, 17 Aug 2021 13:45:27 +0000 Received: from BN8NAM11FT027.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ec:cafe::cf) by BN0PR04CA0118.outlook.office365.com (2603:10b6:408:ec::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.18 via Frontend Transport; Tue, 17 Aug 2021 13:45:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by BN8NAM11FT027.mail.protection.outlook.com (10.13.177.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4415.14 via Frontend Transport; Tue, 17 Aug 2021 13:45:27 +0000 Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 17 Aug 2021 13:45:26 +0000 Received: from nvidia.com (172.20.187.6) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 17 Aug 2021 13:45:24 +0000 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Tue, 17 Aug 2021 16:44:27 +0300 Message-ID: <20210817134441.1966618-8-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210817134441.1966618-1-michaelba@nvidia.com> References: <20210817134441.1966618-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To DRHQMAIL107.nvidia.com (10.27.9.16) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 21b61ac8-1e23-492e-9e9f-08d961854586 X-MS-TrafficTypeDiagnostic: BN9PR12MB5066: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OErXcw8Ov0YgE5s0qvpKEoW+IpEWf4Bf6zK+C0I7fHL6mDDt1J8DI3S7xD+fH2xB3LBIpdbHbTjXUan3DX8HJB3CaNXSaDq4MM1xjmkNPhMfMInK8Qw+BtdiU6Lm1J9InLwwJBAid+gUmuRQzCHS2u+8C02q8M0I2FjvSWvyZh04lqpBnGUISnbr+o1wyXvn0QXSt72JlOrPNGhYI6mgyi39N7FJgQo+mlrBNkpSQdsWOo38kdyheTg9wwyC4e65Jzk0u0hjEfobhz5eYQAVK8bc7ChFM4teWNaJBXe1LW58NcayOaBKYsuyV+eKFXvKpaCNcbvHuddaj/ZiRhGq+HGemAf5gwSgyX6pfPbItfY+M9MLpyYP9NwC5XZCqz5CT4VMfTwiLxJTQzApjJyhm5P0dMYj2zifn7sxMdmvm+wwXFTw/NDAEaMvlKqKGqVFsHn4g/fNHN4HXWP9sWniLLOMrkFmOKBuZ+QR1S7U5GNpEdNnXsGYOPPlgWTwmXbay6afDoXfzBTm1XrUFYDIal9SmJ7aZhyPikKdc65fB5DdPHXemhDP9i5+8rMY7lIckIj+894aS2aTReCJoPhWR761/NdSUxwBA6AuF3bNV0uIVhcJDSH901CB9WiV6w1zypP14UchTgJ4EsgxwEZYTfY6LogBOqro8UXOVZJLcKyQIxCjOMxAiqDYSD1yBWICZbAvJKb1lUxQzaHW4iaZoQ== X-Forefront-Antispam-Report: CIP:216.228.112.35; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid04.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(136003)(39860400002)(376002)(346002)(36840700001)(46966006)(1076003)(8676002)(47076005)(36860700001)(36756003)(86362001)(316002)(54906003)(26005)(107886003)(16526019)(186003)(83380400001)(6286002)(4326008)(336012)(426003)(82310400003)(2906002)(6916009)(5660300002)(478600001)(82740400003)(356005)(70206006)(2616005)(70586007)(7636003)(55016002)(8936002)(7696005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Aug 2021 13:45:27.2382 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 21b61ac8-1e23-492e-9e9f-08d961854586 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.35]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT027.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5066 Subject: [dpdk-dev] [RFC 07/21] net/mlx5: improve probe function on Windows X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" some improvements: - use aux function to find match device. - use spawn a local variable instead of pointing to a list with a single member. Signed-off-by: Michael Baum --- drivers/common/mlx5/mlx5_common.h | 2 + drivers/common/mlx5/version.map | 1 + drivers/common/mlx5/windows/mlx5_common_os.c | 2 +- drivers/net/mlx5/windows/mlx5_os.c | 196 +++---------------- 4 files changed, 26 insertions(+), 175 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 609953b70e..10061f364f 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -465,6 +465,8 @@ int mlx5_os_devx_open_device(struct mlx5_dev_ctx *dev_ctx, struct rte_device *dev, int dbnc, uint32_t classes); int mlx5_os_pd_create(struct mlx5_dev_ctx *dev_ctx); +__rte_internal +struct devx_device_bdf *mlx5_os_get_devx_device(struct rte_device *dev); #endif /* RTE_PMD_MLX5_COMMON_H_ */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 6a88105d02..18856c198e 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -145,6 +145,7 @@ INTERNAL { mlx5_os_dealloc_pd; mlx5_os_dereg_mr; mlx5_os_get_ibv_dev; # WINDOWS_NO_EXPORT + mlx5_os_get_devx_device; mlx5_os_reg_mr; mlx5_os_umem_dereg; mlx5_os_umem_reg; diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c index 5d178b0452..12819383c1 100644 --- a/drivers/common/mlx5/windows/mlx5_common_os.c +++ b/drivers/common/mlx5/windows/mlx5_common_os.c @@ -144,7 +144,7 @@ mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf, * @return * A device match on success, NULL otherwise and rte_errno is set. */ -static struct devx_device_bdf * +struct devx_device_bdf * mlx5_os_get_devx_device(struct rte_device *dev) { int n; diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index 7e1df1c751..0ff9e70d96 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -904,68 +904,6 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable) return -ENOTSUP; } -/** - * Detect if a devx_device_bdf object has identical DBDF values to the - * rte_pci_addr found in bus/pci probing - * - * @param[in] devx_bdf - * Pointer to the devx_device_bdf structure. - * @param[in] addr - * Pointer to the rte_pci_addr structure. - * - * @return - * 1 on Device match, 0 on mismatch. - */ -static int -mlx5_match_devx_bdf_to_addr(struct devx_device_bdf *devx_bdf, - struct rte_pci_addr *addr) -{ - if (addr->domain != (devx_bdf->bus_id >> 8) || - addr->bus != (devx_bdf->bus_id & 0xff) || - addr->devid != devx_bdf->dev_id || - addr->function != devx_bdf->fnc_id) { - return 0; - } - return 1; -} - -/** - * Detect if a devx_device_bdf object matches the rte_pci_addr - * found in bus/pci probing - * Compare both the Native/PF BDF and the raw_bdf representing a VF BDF. - * - * @param[in] devx_bdf - * Pointer to the devx_device_bdf structure. - * @param[in] addr - * Pointer to the rte_pci_addr structure. - * - * @return - * 1 on Device match, 0 on mismatch, rte_errno code on failure. - */ -static int -mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf, - struct rte_pci_addr *addr) -{ - int err; - struct devx_device mlx5_dev; - - if (mlx5_match_devx_bdf_to_addr(devx_bdf, addr)) - return 1; - /** - * Didn't match on Native/PF BDF, could still - * Match a VF BDF, check it next - */ - err = mlx5_glue->query_device(devx_bdf, &mlx5_dev); - if (err) { - DRV_LOG(ERR, "query_device failed"); - rte_errno = err; - return rte_errno; - } - if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr)) - return 1; - return 0; -} - /** * DPDK callback to register a PCI device. * @@ -981,39 +919,15 @@ int mlx5_os_net_probe(struct rte_device *dev) { struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev); - struct devx_device_bdf *devx_bdf_devs, *orig_devx_bdf_devs; - /* - * Number of found IB Devices matching with requested PCI BDF. - * nd != 1 means there are multiple IB devices over the same - * PCI device and we have representors and master. - */ - unsigned int nd = 0; - /* - * Number of found IB device Ports. nd = 1 and np = 1..n means - * we have the single multiport IB device, and there may be - * representors attached to some of found ports. - * Currently not supported. - * unsigned int np = 0; - */ - - /* - * Number of DPDK ethernet devices to Spawn - either over - * multiple IB devices or multiple ports of single IB device. - * Actually this is the number of iterations to spawn. - */ - unsigned int ns = 0; - /* - * Bonding device - * < 0 - no bonding device (single one) - * >= 0 - bonding device (value is slave PF index) - */ - int bd = -1; - struct mlx5_dev_spawn_data *list = NULL; + struct mlx5_dev_spawn_data spawn = { .pf_bond = -1 }; + struct devx_device_bdf *devx_bdf_match = mlx5_os_get_devx_device(dev); struct mlx5_dev_config dev_config; unsigned int dev_config_vf; - int ret, err; + int ret; uint32_t restore; + if (devx_bdf_match == NULL) + return -rte_errno; if (rte_eal_process_type() == RTE_PROC_SECONDARY) { DRV_LOG(ERR, "Secondary process is not supported on Windows."); return -ENOTSUP; @@ -1024,67 +938,14 @@ mlx5_os_net_probe(struct rte_device *dev) strerror(rte_errno)); return -rte_errno; } - errno = 0; - devx_bdf_devs = mlx5_glue->get_device_list(&ret); - orig_devx_bdf_devs = devx_bdf_devs; - if (!devx_bdf_devs) { - rte_errno = errno ? errno : ENOSYS; - DRV_LOG(ERR, "cannot list devices, is ib_uverbs loaded?"); - return -rte_errno; - } - /* - * First scan the list of all Infiniband devices to find - * matching ones, gathering into the list. - */ - struct devx_device_bdf *devx_bdf_match[ret + 1]; - - while (ret-- > 0) { - err = mlx5_match_devx_devices_to_addr(devx_bdf_devs, - &pci_dev->addr); - if (!err) { - devx_bdf_devs++; - continue; - } - if (err != 1) { - ret = -err; - goto exit; - } - devx_bdf_match[nd++] = devx_bdf_devs; - } - devx_bdf_match[nd] = NULL; - if (!nd) { - /* No device matches, just complain and bail out. */ - DRV_LOG(WARNING, - "no DevX device matches PCI device " PCI_PRI_FMT "," - " is DevX Configured?", - pci_dev->addr.domain, pci_dev->addr.bus, - pci_dev->addr.devid, pci_dev->addr.function); - rte_errno = ENOENT; - ret = -rte_errno; - goto exit; - } - /* - * Now we can determine the maximal - * amount of devices to be spawned. - */ - list = mlx5_malloc(MLX5_MEM_ZERO, - sizeof(struct mlx5_dev_spawn_data), - RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); - if (!list) { - DRV_LOG(ERR, "spawn data array allocation failure"); - rte_errno = ENOMEM; - ret = -rte_errno; - goto exit; - } - memset(&list[ns].info, 0, sizeof(list[ns].info)); - list[ns].max_port = 1; - list[ns].phys_port = 1; - list[ns].phys_dev = devx_bdf_match[ns]; - list[ns].eth_dev = NULL; - list[ns].pci_dev = pci_dev; - list[ns].pf_bond = bd; - list[ns].ifindex = -1; /* Spawn will assign */ - list[ns].info = + memset(&spawn.info, 0, sizeof(spawn.info)); + spawn.max_port = 1; + spawn.phys_port = 1; + spawn.phys_dev = devx_bdf_match; + spawn.eth_dev = NULL; + spawn.pci_dev = pci_dev; + spawn.ifindex = -1; /* Spawn will assign */ + spawn.info = (struct mlx5_switch_info){ .master = 0, .representor = 0, @@ -1125,29 +986,16 @@ mlx5_os_net_probe(struct rte_device *dev) dev_config.dv_flow_en = 1; dev_config.decap_en = 0; dev_config.log_hp_size = MLX5_ARG_UNSET; - list[ns].numa_node = pci_dev->device.numa_node; - list[ns].eth_dev = mlx5_dev_spawn(&pci_dev->device, - &list[ns], - &dev_config); - if (!list[ns].eth_dev) - goto exit; - restore = list[ns].eth_dev->data->dev_flags; - rte_eth_copy_pci_info(list[ns].eth_dev, pci_dev); + spawn.numa_node = pci_dev->device.numa_node; + spawn.eth_dev = mlx5_dev_spawn(dev, &spawn, &dev_config); + if (!spawn.eth_dev) + return -rte_errno; + restore = spawn.eth_dev->data->dev_flags; + rte_eth_copy_pci_info(spawn.eth_dev, pci_dev); /* Restore non-PCI flags cleared by the above call. */ - list[ns].eth_dev->data->dev_flags |= restore; - rte_eth_dev_probing_finish(list[ns].eth_dev); - ret = 0; -exit: - /* - * Do the routine cleanup: - * - free allocated spawn data array - * - free the device list - */ - if (list) - mlx5_free(list); - MLX5_ASSERT(orig_devx_bdf_devs); - mlx5_glue->free_device_list(orig_devx_bdf_devs); - return ret; + spawn.eth_dev->data->dev_flags |= restore; + rte_eth_dev_probing_finish(spawn.eth_dev); + return 0; } /**