From patchwork Wed Jul 21 14:37:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96157 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3F727A0C51; Wed, 21 Jul 2021 16:38:38 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 26E0440DF7; Wed, 21 Jul 2021 16:38:38 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2071.outbound.protection.outlook.com [40.107.93.71]) by mails.dpdk.org (Postfix) with ESMTP id 7FCCE40689 for ; Wed, 21 Jul 2021 16:38:36 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HX46qJ7wsuxEzZTULILqOaX/S2hSk6kICpUusXd/HMrRGd395+unJqZWPATEp/KX+hZ6UNO7A/u68xwUE/APVIGF3ZpZjI4zF4/o0yBjdhmIYd7f1FNpFBmgkdc5YUUrfCYTpuAxQ+F2TkfllcKW+7+ETzVe0b/uu9g/RquvQtMuq27kVWeDBQ8UKoGaKkk4itUhWI9/Rc0aVHMz66E9vxA47MSuU+/FdEl3XgPUdWwb6VewJDdkSUyxwrPMZboQwm1H6so3JVItp68+wjaGC9wtC4CrhGi/lT2YcwHMV634P0cppg9/mMuUpdtD0mj47bZHxGT1CvK5c+4R2EzH3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q1B4T1lepGSitzdZQ+Ue+B3MGsI48sA0E27LcAa1YQc=; b=iTcGgK4cp0qFHTVd4p8xdIIALyb4lyql44IKMtJKBagKQ5vzcdl6+T38nX8PaOqVnMKPfWcyGfzqlM2Wx8iRg6RT8oMzpW6V0Yaov956m/x92QaGyQpMZ+CdSv5how2IYTmGuHR/lDax45DIg5K9V3WcpVg1bovCvKwDo0aiU8K+Eq2WiDIqMRYJBVije8kxaR4fzWZG7/BNLiaoyGZoTeKOp/FNseDK1Jo1xK4KFvxJB2jbXej5RjKr1asHdZ+/obD9ggOYdWViPZqX9Dic9ezKebTnxvFmT6lxwWq+kzJRtCRaXYmez2gXtvh5jC7ASmpF5/03JsTHpfLH/Mb1yA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q1B4T1lepGSitzdZQ+Ue+B3MGsI48sA0E27LcAa1YQc=; b=GGY4wRXynNGF34WoUxAY43aPFsxhw4/rXvgAoF9zclcbmq1seuN5HOv2lWQPbhHbKEyZDhjRYW8dcLwR/S0VDL2ZCvg3gXe+JII+HfERPvgCZgJ2//JC1YTX8cE6EVkVo2ICu3No33K2SH3bQQmy2zwRcyi/+W/uYM2BBI4EgM2dphhHtDK1QMclp3iScc/2NnqmQ9VVbZTgqVtQAJEIao876yoUxOBFlmyDNPHC0gRWBTkb/z2nYN0ZstBFzwMyVUDJPgTLjQ7sZepQBDdChtO40ScbjuRuEsFJ/NOTm8yViJ5VUzFRuk2aMtvD98SOi3GzWdP4qeMD3+aeIuxJag== Received: from BN0PR03CA0012.namprd03.prod.outlook.com (2603:10b6:408:e6::17) by MW2PR12MB2347.namprd12.prod.outlook.com (2603:10b6:907:7::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22; Wed, 21 Jul 2021 14:38:34 +0000 Received: from BN8NAM11FT021.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e6:cafe::7b) by BN0PR03CA0012.outlook.office365.com (2603:10b6:408:e6::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:38:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT021.mail.protection.outlook.com (10.13.177.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:38:34 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:38:31 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler Date: Wed, 21 Jul 2021 22:37:28 +0800 Message-ID: <20210721143743.24626-2-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e3f7b281-b190-4453-7afc-08d94c5537ef X-MS-TrafficTypeDiagnostic: MW2PR12MB2347: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2733; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qo01wLbipPKyRgzf8syl4zpBRU3WRoYGjZ0xKNzTx+LtyADeC1wO/q3h0j/2m5pMLYFEyD08eqX2x5IvizSB7/64NxXJKON/pBP6buJ0TCXgmUeUirXz2zRA+6x9jrjKIchyqH9l3TCNtFgrco2GzZ9nyBmIuNNROEUDDYVuU4mUErMS0CF+oxNsNZsE2/BIiOjFKnoziyB9PahlNIgqZ9jfy7oo0kOBc/28MVkpJVqNw8xXY4Nhn/CN/WI++20Tql1g3O8EIOy58MPD/JSHm/i3MZrL0Wg4XFKPSZ/JW34Tfi2Ue/vDg6UZ6wgvUfbB4fpWP8Ybmpq7CEZJLhxoYsJEFCRJppTNIVPymb8sMd40r7RLQzd+S2bRh4QGJpzfbgn+DotA+akQLRVsxByFswQcU4QPQ2pZOvqYZbHh9FmlO/D7KbhhEt8DH0loIloYgsWRjDerQBnGDfUhXaCq7ZPry1flqoWS4QqUJXiyW5dUb1sit7dYAI4foVZoPkRz4PZLIcyI+arKspnGYlB+3pXDNi2AtEq/IWoLgfi0WXAKECO71D3YR34+hMa/ap/vlukREqgqg8riJnCyNkNtAJfRp4jA1qJQ4Ya2UFMKHYLi3cMkedgtrseDcsoDP72F3xQECuVFWSOXLF2+yRCXJ2RorpPoaO6CqJqb4nF8rZbg4GkiwIbDJKmvCudAEUzmWcMqSRbJeYJgfXv7DsRSgA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(396003)(376002)(346002)(136003)(36840700001)(46966006)(8676002)(478600001)(8936002)(2616005)(6862004)(47076005)(107886003)(36860700001)(6286002)(70586007)(4326008)(2906002)(86362001)(336012)(82310400003)(356005)(6636002)(70206006)(426003)(6666004)(7636003)(16526019)(54906003)(83380400001)(186003)(7696005)(26005)(37006003)(55016002)(36906005)(316002)(82740400003)(36756003)(1076003)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:38:34.2419 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e3f7b281-b190-4453-7afc-08d94c5537ef X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT021.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR12MB2347 Subject: [dpdk-dev] [PATCH v4 01/16] common/mlx5: rename eth device class name X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To align with EAL class driver, rename internal class name from "net" to "eth" Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_common.h | 2 +- drivers/common/mlx5/mlx5_common_pci.c | 22 ++++++++++++---------- drivers/net/mlx5/mlx5.c | 2 +- 3 files changed, 14 insertions(+), 12 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 962179a5a5..05008983ea 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -212,7 +212,7 @@ int mlx5_get_ifname_sysfs(const char *ibdev_path, char *ifname); enum mlx5_class { MLX5_CLASS_INVALID, - MLX5_CLASS_NET = RTE_BIT64(0), + MLX5_CLASS_ETH = RTE_BIT64(0), MLX5_CLASS_VDPA = RTE_BIT64(1), MLX5_CLASS_REGEX = RTE_BIT64(2), MLX5_CLASS_COMPRESS = RTE_BIT64(3), diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 5547e62d6b..591054468d 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -28,34 +28,36 @@ static const struct { unsigned int driver_class; } mlx5_classes[] = { { .name = "vdpa", .driver_class = MLX5_CLASS_VDPA }, - { .name = "net", .driver_class = MLX5_CLASS_NET }, + { .name = "eth", .driver_class = MLX5_CLASS_ETH }, + /* Keep name "net" for backward compatibility. */ + { .name = "net", .driver_class = MLX5_CLASS_ETH }, { .name = "regex", .driver_class = MLX5_CLASS_REGEX }, { .name = "compress", .driver_class = MLX5_CLASS_COMPRESS }, { .name = "crypto", .driver_class = MLX5_CLASS_CRYPTO }, }; static const unsigned int mlx5_class_combinations[] = { - MLX5_CLASS_NET, + MLX5_CLASS_ETH, MLX5_CLASS_VDPA, MLX5_CLASS_REGEX, MLX5_CLASS_COMPRESS, MLX5_CLASS_CRYPTO, - MLX5_CLASS_NET | MLX5_CLASS_REGEX, + MLX5_CLASS_ETH | MLX5_CLASS_REGEX, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX, - MLX5_CLASS_NET | MLX5_CLASS_COMPRESS, + MLX5_CLASS_ETH | MLX5_CLASS_COMPRESS, MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS, MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, - MLX5_CLASS_NET | MLX5_CLASS_CRYPTO, + MLX5_CLASS_ETH | MLX5_CLASS_CRYPTO, + MLX5_CLASS_ETH | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, MLX5_CLASS_VDPA | MLX5_CLASS_CRYPTO, MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, - MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, - MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_ETH | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, - MLX5_CLASS_NET | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, + MLX5_CLASS_ETH | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, - MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | + MLX5_CLASS_ETH | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, @@ -317,7 +319,7 @@ mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, } } else { /* Default to net class. */ - user_classes = MLX5_CLASS_NET; + user_classes = MLX5_CLASS_ETH; } dev = pci_to_mlx5_device(pci_dev); if (!dev) { diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 021a34dd4d..d9c90d5ef9 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -2446,7 +2446,7 @@ static const struct rte_pci_id mlx5_pci_id_map[] = { }; static struct mlx5_pci_driver mlx5_driver = { - .driver_class = MLX5_CLASS_NET, + .driver_class = MLX5_CLASS_ETH, .pci_driver = { .driver = { .name = MLX5_PCI_DRIVER_NAME, From patchwork Wed Jul 21 14:37:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96158 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3DD2A0C51; Wed, 21 Jul 2021 16:38:44 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8DD96410DA; Wed, 21 Jul 2021 16:38:40 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2054.outbound.protection.outlook.com [40.107.244.54]) by mails.dpdk.org (Postfix) with ESMTP id 8E4DD40F35 for ; Wed, 21 Jul 2021 16:38:38 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DJC8rV4mA0e6Ydd99nbxaFTyrG8qp/LMZD9tGxDicbcmU9tjiQ/x6em1NhHmvfH8W7LMGnnU7USyrWvTI+sU4Lxu6Zav9b5JymWSKtHx2iqeP+6OvFwNW2+F3PsVVQpwM4BpwJoDGimuJc+FDLIgncJTMAyGUNbezx6lSMtDN2qSLHgDlzT0vqMnV7QbkuH5cq0wEctLGQbPIbcVva8Vj1lN5yCtVPcGiEjDHGj8BMq4lslAzqe0Fz6np3jP5RmGIPawyqSg2/OasbtKTl1FUMGAMREVsaLO0cI1iI9wYkw6zLAyj2/OlWkaoAGJNV5qcx0V6+cKO0tndl9QFsaRsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5G0eCClNTtsTWqLvHuQtLWcmc1T98Jqxd2R4BXdEzjQ=; b=f7MGUknphI4O5wDT0Sv+SwV9vDEzzS49opf1uthyLhqTKYXvGR9srKHRQn8ZAtpk7GPjt4cFQffw52Zs9h9b9hsfzHwmUxWqe1hjUxsAM7u+NHPNjcgcAE+ox6GJh0JWy4a5n+JuUL7+bfL5LweAz5PoH22+/EErt7zKheEd6TiyJ5tgaL2M1F+0AjIAA4NjyGVxeX1eJ6KksiKGdTblRuEgtsnbnHJzXfnf6anPO5tJdezR5FG8zdvQnx5hGX0e6MSzNu9zhXjHK25gnB5CqiMR7sIiR1jZt7UaFSXYTaNB/vg+bBxGuaIIRZ1e2zx7uaAsY5R3MAkC7Qw0NSVeXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=ashroe.eu smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5G0eCClNTtsTWqLvHuQtLWcmc1T98Jqxd2R4BXdEzjQ=; b=c672iKq6eHyOX38M14K7lLdygTclpuyBMSniSuFa4wemduzDmWMrGzfkgCy9YdEFiez7s25KunDOr/tCq0QZCBmN41Be/6MQC6AVuJD3NnWXCP4AZSXW6ou5MQUD7+j/d+E3YNzeUympwwbIGSa0A8JUEKbzUYDRCA4BD1a3NtpkVE0/ToBY5WaEvQesjxyKNT/L5M/aSogswFLX6PIsPfofifSTruuAwQE0dN6wGPIrfqsi0tLe1gtpGG8YPzUGjOt48FEYHluoT8hlh6nIZVhkpzStbwgAba17i9PHQ7fuGC9TTmPS6cjoDJv9BLSvKVjFEKbM2iQi6PCHHYNIQA== Received: from BN9PR03CA0643.namprd03.prod.outlook.com (2603:10b6:408:13b::18) by DM4PR12MB5117.namprd12.prod.outlook.com (2603:10b6:5:390::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.23; Wed, 21 Jul 2021 14:38:36 +0000 Received: from BN8NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13b:cafe::a6) by BN9PR03CA0643.outlook.office365.com (2603:10b6:408:13b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22 via Frontend Transport; Wed, 21 Jul 2021 14:38:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; ashroe.eu; dkim=none (message not signed) header.d=none;ashroe.eu; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT019.mail.protection.outlook.com (10.13.176.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:38:35 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:38:33 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Ray Kinsella Date: Wed, 21 Jul 2021 22:37:29 +0800 Message-ID: <20210721143743.24626-3-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 141a51b7-0a70-4733-c4ae-08d94c5538fc X-MS-TrafficTypeDiagnostic: DM4PR12MB5117: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nNB4cbgyP82AVyUWlka4wIWDuP15eT+nSi0fSyVcJzbeS2Bl8PE7wDBe6TTxIy9OSuFhfYDqzd4DAQSLKJfkknyLJKcE/I6BnmNAUIaqnHE0ERlBRfRrYaa1bFORhWzp9/i5x92IOsAYVrucSnRkJP7jmoAJ2002nWw/5FDcJk+wbp9f2/J/Gg2RL9hfOwTuIe13IOgGi5FZYAJUASJZr2xjcV1kqE7oMb50/DbNpRjvf+Ytlbnkm26X8IdRm17gizxbLrHaBWSbsHOfy6bzc7l4OkLMSf7gSiOHIc8z+th7HXSfyXQvainsXsv897lCTt0mIOeRT6R+WA7pw7k5nhBszPKVycCBk3sUTttZSXHo9Em0jfawySOzgTg2eEtg30MPbnP772lcHFSjf6+pHhi4Os0sKr2nNq1Ovh5S5KN+IlTHQSTArQ0HoB8j9DIIKIGBsIiSV9Q3HC7wlhDzsbXY3KnQ/wbmZu4xfVzb9dmryc4H9Co2gLmYojW0wHauEupOYl8ZYu+oHdIgLxEY5lqXyhrELvUEdF1zMURhnj2TAP21tLWCzPpZU7pq8rKkN02mqXNTr7kJwTcv9bYjj6xNjzRxb6optdJPunWl3j0rMp72g0g73d0+sPo5DXp5cfMKgje+vVlDrbPccfxqZHGCgGCrcdxneabySHPejAj4WfdRv71hRLtNnrc8z/257j0wmjmwMP/xSBpjkiMyPQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(376002)(346002)(39860400002)(396003)(36840700001)(46966006)(4326008)(478600001)(36906005)(316002)(83380400001)(70586007)(426003)(16526019)(8936002)(82740400003)(2906002)(6286002)(186003)(6862004)(36756003)(356005)(8676002)(54906003)(30864003)(26005)(7696005)(82310400003)(1076003)(6636002)(86362001)(47076005)(37006003)(6666004)(70206006)(336012)(55016002)(7636003)(2616005)(36860700001)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:38:35.9417 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 141a51b7-0a70-4733-c4ae-08d94c5538fc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5117 Subject: [dpdk-dev] [PATCH v4 02/16] common/mlx5: add common device driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support auxiliary bus, introduces common device driver and callbacks, suppose to replace mlx5 common PCI bus driver. Mlx5 class drivers, i.e. eth, vDPA, regex and compress normally consumes single Verbs device context to probe a device. The Verbs device comes from PCI address if the device is PCI bus device, from Auxiliary sysfs if the device is auxiliary bus device. Currently only PCI bus is supported. Common device driver is a middle layer between mlx5 class drivers and bus, resolve and abstract bus info to Verbs device for class drivers. Both PCI bus driver and Auxiliary bus driver can utilize the common driver layer to cast bus operations to mlx5 class drivers. Legacy mlx5 common PCI bus driver still being used by mlx5 eth, vDPA, regex and compress PMD, will remove once all PMD drivers migrate to new common driver. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- doc/guides/rel_notes/release_21_08.rst | 5 + drivers/common/mlx5/linux/mlx5_common_os.c | 2 +- drivers/common/mlx5/linux/mlx5_common_os.h | 7 +- drivers/common/mlx5/linux/mlx5_common_verbs.c | 21 +- drivers/common/mlx5/mlx5_common.c | 363 +++++++++++++++++- drivers/common/mlx5/mlx5_common.h | 128 ++++++ drivers/common/mlx5/mlx5_common_pci.c | 133 ++++++- drivers/common/mlx5/mlx5_common_private.h | 41 ++ drivers/common/mlx5/mlx5_common_utils.h | 2 + drivers/common/mlx5/version.map | 4 + 10 files changed, 688 insertions(+), 18 deletions(-) create mode 100644 drivers/common/mlx5/mlx5_common_private.h diff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst index 409f9980e9..27d591137e 100644 --- a/doc/guides/rel_notes/release_21_08.rst +++ b/doc/guides/rel_notes/release_21_08.rst @@ -148,6 +148,11 @@ New Features The experimental PMD power management API now supports managing multiple Ethernet Rx queues per lcore. +* **Added Sub-Function support for mlx5 PMDs** + + Added Sub-Function support based on auxiliary bus for mlx5 PMDs: + net, vDPA, compress and regex. + Removed Items ------------- diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index ea0b71e425..78a9723075 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -425,7 +425,7 @@ mlx5_glue_constructor(void) } struct ibv_device * -mlx5_os_get_ibv_device(struct rte_pci_addr *addr) +mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) { int n; struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h index 72d6bf828b..86d0cb09b0 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.h +++ b/drivers/common/mlx5/linux/mlx5_common_os.h @@ -291,6 +291,11 @@ mlx5_os_free(void *addr) __rte_internal struct ibv_device * -mlx5_os_get_ibv_device(struct rte_pci_addr *addr); +mlx5_os_get_ibv_device(const struct rte_pci_addr *addr); + +__rte_internal +struct ibv_device * +mlx5_os_get_ibv_dev(const struct rte_device *dev); + #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */ diff --git a/drivers/common/mlx5/linux/mlx5_common_verbs.c b/drivers/common/mlx5/linux/mlx5_common_verbs.c index aa560f05f2..6a6ab7a7a2 100644 --- a/drivers/common/mlx5/linux/mlx5_common_verbs.c +++ b/drivers/common/mlx5/linux/mlx5_common_verbs.c @@ -10,11 +10,31 @@ #include #include +#include +#include + +#include "mlx5_common_utils.h" +#include "mlx5_common_log.h" +#include "mlx5_common_private.h" #include "mlx5_autoconf.h" #include #include #include +struct ibv_device * +mlx5_os_get_ibv_dev(const struct rte_device *dev) +{ + struct ibv_device *ibv = NULL; + + if (mlx5_dev_is_pci(dev)) + ibv = mlx5_os_get_ibv_device(&RTE_DEV_TO_PCI_CONST(dev)->addr); + if (ibv == NULL) { + rte_errno = ENODEV; + DRV_LOG(ERR, "Verbs device not found: %s", dev->name); + } + return ibv; +} + /** * Register mr. Given protection domain pointer, pointer to addr and length * register the memory region. @@ -68,4 +88,3 @@ mlx5_common_verbs_dereg_mr(struct mlx5_pmd_mr *pmd_mr) memset(pmd_mr, 0, sizeof(*pmd_mr)); } } - diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 25e9f09108..97d3e3e60e 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -8,11 +8,14 @@ #include #include +#include +#include #include "mlx5_common.h" #include "mlx5_common_os.h" #include "mlx5_common_log.h" #include "mlx5_common_pci.h" +#include "mlx5_common_private.h" uint8_t haswell_broadwell_cpu; @@ -41,6 +44,362 @@ static inline void mlx5_cpu_id(unsigned int level, RTE_LOG_REGISTER_DEFAULT(mlx5_common_logtype, NOTICE) +/* Head of list of drivers. */ +static TAILQ_HEAD(mlx5_drivers, mlx5_class_driver) drivers_list = + TAILQ_HEAD_INITIALIZER(drivers_list); + +/* Head of devices. */ +static TAILQ_HEAD(mlx5_devices, mlx5_common_device) devices_list = + TAILQ_HEAD_INITIALIZER(devices_list); + +static const struct { + const char *name; + unsigned int drv_class; +} mlx5_classes[] = { + { .name = "vdpa", .drv_class = MLX5_CLASS_VDPA }, + { .name = "eth", .drv_class = MLX5_CLASS_ETH }, + /* Keep class "net" for backward compatibility. */ + { .name = "net", .drv_class = MLX5_CLASS_ETH }, + { .name = "regex", .drv_class = MLX5_CLASS_REGEX }, + { .name = "compress", .drv_class = MLX5_CLASS_COMPRESS }, + { .name = "crypto", .drv_class = MLX5_CLASS_CRYPTO }, +}; + +static int +class_name_to_value(const char *class_name) +{ + unsigned int i; + + for (i = 0; i < RTE_DIM(mlx5_classes); i++) { + if (strcmp(class_name, mlx5_classes[i].name) == 0) + return mlx5_classes[i].drv_class; + } + return -EINVAL; +} + +static struct mlx5_class_driver * +driver_get(uint32_t class) +{ + struct mlx5_class_driver *driver; + + TAILQ_FOREACH(driver, &drivers_list, next) { + if ((uint32_t)driver->drv_class == class) + return driver; + } + return NULL; +} + +static int +devargs_class_handler(__rte_unused const char *key, + const char *class_names, void *opaque) +{ + int *ret = opaque; + int class_val; + char *scratch; + char *found; + char *refstr = NULL; + + *ret = 0; + scratch = strdup(class_names); + if (scratch == NULL) { + *ret = -ENOMEM; + return *ret; + } + found = strtok_r(scratch, ":", &refstr); + if (found == NULL) + /* Empty string. */ + goto err; + do { + /* Extract each individual class name. Multiple + * classes can be supplied as class=net:regex:foo:bar. + */ + class_val = class_name_to_value(found); + /* Check if its a valid class. */ + if (class_val < 0) { + *ret = -EINVAL; + goto err; + } + *ret |= class_val; + found = strtok_r(NULL, ":", &refstr); + } while (found != NULL); +err: + free(scratch); + if (*ret < 0) + DRV_LOG(ERR, "Invalid mlx5 class options: %s.\n", class_names); + return *ret; +} + +static int +parse_class_options(const struct rte_devargs *devargs) +{ + struct rte_kvargs *kvlist; + int ret = 0; + + if (devargs == NULL) + return 0; + if (devargs->cls != NULL && devargs->cls->name != NULL) + /* Global syntax, only one class type. */ + return class_name_to_value(devargs->cls->name); + /* Legacy devargs support multiple classes. */ + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + return 0; + rte_kvargs_process(kvlist, RTE_DEVARGS_KEY_CLASS, + devargs_class_handler, &ret); + rte_kvargs_free(kvlist); + return ret; +} + +static const unsigned int mlx5_class_invalid_combinations[] = { + MLX5_CLASS_ETH | MLX5_CLASS_VDPA, + /* New class combination should be added here. */ +}; + +static int +is_valid_class_combination(uint32_t user_classes) +{ + unsigned int i; + + /* Verify if user specified unsupported combination. */ + for (i = 0; i < RTE_DIM(mlx5_class_invalid_combinations); i++) { + if ((mlx5_class_invalid_combinations[i] & user_classes) == + mlx5_class_invalid_combinations[i]) + return -EINVAL; + } + /* Not found any invalid class combination. */ + return 0; +} + +static bool +device_class_enabled(const struct mlx5_common_device *device, uint32_t class) +{ + return (device->classes_loaded & class) > 0; +} + +static bool +mlx5_bus_match(const struct mlx5_class_driver *drv, + const struct rte_device *dev) +{ + if (mlx5_dev_is_pci(dev)) + return mlx5_dev_pci_match(drv, dev); + return true; +} + +static struct mlx5_common_device * +to_mlx5_device(const struct rte_device *rte_dev) +{ + struct mlx5_common_device *dev; + + TAILQ_FOREACH(dev, &devices_list, next) { + if (rte_dev == dev->dev) + return dev; + } + return NULL; +} + +static void +dev_release(struct mlx5_common_device *dev) +{ + TAILQ_REMOVE(&devices_list, dev, next); + rte_free(dev); +} + +static int +drivers_remove(struct mlx5_common_device *dev, uint32_t enabled_classes) +{ + struct mlx5_class_driver *driver; + int local_ret = -ENODEV; + unsigned int i = 0; + int ret = 0; + + enabled_classes &= dev->classes_loaded; + while (enabled_classes) { + driver = driver_get(RTE_BIT64(i)); + if (driver != NULL) { + local_ret = driver->remove(dev->dev); + if (local_ret == 0) + dev->classes_loaded &= ~RTE_BIT64(i); + else if (ret == 0) + ret = local_ret; + } + enabled_classes &= ~RTE_BIT64(i); + i++; + } + if (local_ret != 0 && ret == 0) + ret = local_ret; + return ret; +} + +static int +drivers_probe(struct mlx5_common_device *dev, uint32_t user_classes) +{ + struct mlx5_class_driver *driver; + uint32_t enabled_classes = 0; + bool already_loaded; + int ret; + + TAILQ_FOREACH(driver, &drivers_list, next) { + if ((driver->drv_class & user_classes) == 0) + continue; + if (!mlx5_bus_match(driver, dev->dev)) + continue; + already_loaded = dev->classes_loaded & driver->drv_class; + if (already_loaded && driver->probe_again == 0) { + DRV_LOG(ERR, "Device %s is already probed", + dev->dev->name); + ret = -EEXIST; + goto probe_err; + } + ret = driver->probe(dev->dev); + if (ret < 0) { + DRV_LOG(ERR, "Failed to load driver %s", + driver->name); + goto probe_err; + } + enabled_classes |= driver->drv_class; + } + dev->classes_loaded |= enabled_classes; + return 0; +probe_err: + /* Only unload drivers which are enabled which were enabled + * in this probe instance. + */ + drivers_remove(dev, enabled_classes); + return ret; +} + +int +mlx5_common_dev_probe(struct rte_device *eal_dev) +{ + struct mlx5_common_device *dev; + uint32_t classes = 0; + bool new_device = false; + int ret; + + DRV_LOG(INFO, "probe device \"%s\".", eal_dev->name); + ret = parse_class_options(eal_dev->devargs); + if (ret < 0) { + DRV_LOG(ERR, "Unsupported mlx5 class type: %s", + eal_dev->devargs->args); + return ret; + } + classes = ret; + if (classes == 0) + /* Default to net class. */ + classes = MLX5_CLASS_ETH; + dev = to_mlx5_device(eal_dev); + if (!dev) { + dev = rte_zmalloc("mlx5_common_device", sizeof(*dev), 0); + if (!dev) + return -ENOMEM; + dev->dev = eal_dev; + TAILQ_INSERT_HEAD(&devices_list, dev, next); + new_device = true; + } else { + /* Validate combination here. */ + ret = is_valid_class_combination(classes | + dev->classes_loaded); + if (ret != 0) { + DRV_LOG(ERR, "Unsupported mlx5 classes combination."); + return ret; + } + } + ret = drivers_probe(dev, classes); + if (ret) + goto class_err; + return 0; +class_err: + if (new_device) + dev_release(dev); + return ret; +} + +int +mlx5_common_dev_remove(struct rte_device *eal_dev) +{ + struct mlx5_common_device *dev; + int ret; + + dev = to_mlx5_device(eal_dev); + if (!dev) + return -ENODEV; + /* Matching device found, cleanup and unload drivers. */ + ret = drivers_remove(dev, dev->classes_loaded); + if (ret != 0) + dev_release(dev); + return ret; +} + +int +mlx5_common_dev_dma_map(struct rte_device *dev, void *addr, uint64_t iova, + size_t len) +{ + struct mlx5_class_driver *driver = NULL; + struct mlx5_class_driver *temp; + struct mlx5_common_device *mdev; + int ret = -EINVAL; + + mdev = to_mlx5_device(dev); + if (!mdev) + return -ENODEV; + TAILQ_FOREACH(driver, &drivers_list, next) { + if (!device_class_enabled(mdev, driver->drv_class) || + driver->dma_map == NULL) + continue; + ret = driver->dma_map(dev, addr, iova, len); + if (ret) + goto map_err; + } + return ret; +map_err: + TAILQ_FOREACH(temp, &drivers_list, next) { + if (temp == driver) + break; + if (device_class_enabled(mdev, temp->drv_class) && + temp->dma_map && temp->dma_unmap) + temp->dma_unmap(dev, addr, iova, len); + } + return ret; +} + +int +mlx5_common_dev_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, + size_t len) +{ + struct mlx5_class_driver *driver; + struct mlx5_common_device *mdev; + int local_ret = -EINVAL; + int ret = 0; + + mdev = to_mlx5_device(dev); + if (!mdev) + return -ENODEV; + /* There is no unmap error recovery in current implementation. */ + TAILQ_FOREACH_REVERSE(driver, &drivers_list, mlx5_drivers, next) { + if (!device_class_enabled(mdev, driver->drv_class) || + driver->dma_unmap == NULL) + continue; + local_ret = driver->dma_unmap(dev, addr, iova, len); + if (local_ret && (ret == 0)) + ret = local_ret; + } + if (local_ret) + ret = local_ret; + return ret; +} + +void +mlx5_class_driver_register(struct mlx5_class_driver *driver) +{ + mlx5_common_driver_on_register_pci(driver); + TAILQ_INSERT_TAIL(&drivers_list, driver, next); +} + +static void mlx5_common_driver_init(void) +{ + mlx5_common_pci_init(); +} + static bool mlx5_common_initialized; /** @@ -55,7 +414,7 @@ mlx5_common_init(void) return; mlx5_glue_constructor(); - mlx5_common_pci_init(); + mlx5_common_driver_init(); mlx5_common_initialized = true; } @@ -214,3 +573,5 @@ mlx5_devx_alloc_uar(void *ctx, int mapping) exit: return uar; } + +RTE_PMD_EXPORT_NAME(mlx5_common_driver, __COUNTER__); diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 05008983ea..6ba1e70223 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -245,4 +245,132 @@ extern uint8_t haswell_broadwell_cpu; __rte_internal void mlx5_common_init(void); +/** + * Common Driver Interface + * ConnectX common driver supports multiple classes: net,vdpa,regex,crypto and + * compress devices. This layer enables creating such multiple class of devices + * on a single device by allowing to bind multiple class specific device + * driver to attach to common driver. + * + * ------------ ------------- -------------- ----------------- ------------ + * | mlx5 net | | mlx5 vdpa | | mlx5 regex | | mlx5 compress | | mlx5 ... | + * | driver | | driver | | driver | | driver | | drivers | + * ------------ ------------- -------------- ----------------- ------------ + * || + * ----------------- + * | mlx5 | + * | common driver | + * ----------------- + * | | + * ----------- ----------------- + * | mlx5 | | mlx5 | + * | pci dev | | auxiliary dev | + * ----------- ----------------- + * + * - mlx5 pci bus driver binds to mlx5 PCI devices defined by PCI + * ID table of all related mlx5 PCI devices. + * - mlx5 class driver such as net, vdpa, regex PMD defines its + * specific PCI ID table and mlx5 bus driver probes matching + * class drivers. + * - mlx5 common driver is central place that validates supported + * class combinations. + * - mlx5 common driver hide bus difference by resolving device address + * from devargs, locating target RDMA device and probing with it. + */ + +/** + * Initialization function for the driver called during device probing. + */ +typedef int (mlx5_class_driver_probe_t)(struct rte_device *dev); + +/** + * Uninitialization function for the driver called during hot-unplugging. + */ +typedef int (mlx5_class_driver_remove_t)(struct rte_device *dev); + +/** + * Driver-specific DMA mapping. After a successful call the device + * will be able to read/write from/to this segment. + * + * @param dev + * Pointer to the device. + * @param addr + * Starting virtual address of memory to be mapped. + * @param iova + * Starting IOVA address of memory to be mapped. + * @param len + * Length of memory segment being mapped. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int (mlx5_class_driver_dma_map_t)(struct rte_device *dev, void *addr, + uint64_t iova, size_t len); + +/** + * Driver-specific DMA un-mapping. After a successful call the device + * will not be able to read/write from/to this segment. + * + * @param dev + * Pointer to the device. + * @param addr + * Starting virtual address of memory to be unmapped. + * @param iova + * Starting IOVA address of memory to be unmapped. + * @param len + * Length of memory segment being unmapped. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int (mlx5_class_driver_dma_unmap_t)(struct rte_device *dev, void *addr, + uint64_t iova, size_t len); + +/** Device already probed can be probed again to check for new ports. */ +#define MLX5_DRV_PROBE_AGAIN 0x0004 + +/** + * A structure describing a mlx5 common class driver. + */ +struct mlx5_class_driver { + TAILQ_ENTRY(mlx5_class_driver) next; + enum mlx5_class drv_class; /**< Class of this driver. */ + const char *name; /**< Driver name. */ + mlx5_class_driver_probe_t *probe; /**< Device Probe function. */ + mlx5_class_driver_remove_t *remove; /**< Device Remove function. */ + mlx5_class_driver_dma_map_t *dma_map; /**< device dma map function. */ + mlx5_class_driver_dma_unmap_t *dma_unmap; + /**< device dma unmap function. */ + const struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ + uint32_t probe_again:1; + /**< Device already probed can be probed again to check new device. */ + uint32_t intr_lsc:1; /**< Supports link state interrupt. */ + uint32_t intr_rmv:1; /**< Supports device remove interrupt. */ +}; + +/** + * Register a mlx5 device driver. + * + * @param driver + * A pointer to a mlx5_driver structure describing the driver + * to be registered. + */ +__rte_internal +void +mlx5_class_driver_register(struct mlx5_class_driver *driver); + +/** + * Test a device is PCI bus device. + * + * @param dev + * Pointer to device. + * + * @return + * - True on device devargs is a PCI bus device. + * - False otherwise. + */ +__rte_internal +bool +mlx5_dev_is_pci(const struct rte_device *dev); + #endif /* RTE_PMD_MLX5_COMMON_H_ */ diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 591054468d..a7db6e2c11 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -3,11 +3,19 @@ */ #include + #include +#include +#include #include #include "mlx5_common_log.h" #include "mlx5_common_pci.h" +#include "mlx5_common_private.h" + +static struct rte_pci_driver mlx5_common_pci_driver; + +/********** Legacy PCI bus driver, to be removed ********/ struct mlx5_pci_device { struct rte_pci_device *pci_dev; @@ -298,8 +306,8 @@ drivers_probe(struct mlx5_pci_device *dev, struct rte_pci_driver *pci_drv, * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) +mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) { struct mlx5_pci_device *dev; uint32_t user_classes = 0; @@ -352,7 +360,7 @@ mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, * 0 on success, the function cannot fail. */ static int -mlx5_common_pci_remove(struct rte_pci_device *pci_dev) +mlx5_pci_remove(struct rte_pci_device *pci_dev) { struct mlx5_pci_device *dev; int ret; @@ -368,8 +376,8 @@ mlx5_common_pci_remove(struct rte_pci_device *pci_dev) } static int -mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) +mlx5_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) { struct mlx5_pci_driver *driver = NULL; struct mlx5_pci_driver *temp; @@ -401,8 +409,8 @@ mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, } static int -mlx5_common_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) +mlx5_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) { struct mlx5_pci_driver *driver; struct mlx5_pci_device *dev; @@ -435,10 +443,10 @@ static struct rte_pci_driver mlx5_pci_driver = { .driver = { .name = MLX5_PCI_DRIVER_NAME, }, - .probe = mlx5_common_pci_probe, - .remove = mlx5_common_pci_remove, - .dma_map = mlx5_common_pci_dma_map, - .dma_unmap = mlx5_common_pci_dma_unmap, + .probe = mlx5_pci_probe, + .remove = mlx5_pci_remove, + .dma_map = mlx5_pci_dma_map, + .dma_unmap = mlx5_pci_dma_unmap, }; static int @@ -502,7 +510,7 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) updated_table = calloc(num_ids, sizeof(*updated_table)); if (!updated_table) return -ENOMEM; - if (TAILQ_EMPTY(&drv_list)) { + if (old_table == NULL) { /* Copy the first driver's ID table. */ for (id_iter = driver_id_table; id_iter->vendor_id != 0; id_iter++, i++) @@ -518,6 +526,7 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) /* Terminate table with empty entry. */ updated_table[i].vendor_id = 0; mlx5_pci_driver.id_table = updated_table; + mlx5_common_pci_driver.id_table = updated_table; mlx5_pci_id_table = updated_table; if (old_table) free(old_table); @@ -536,6 +545,101 @@ mlx5_pci_driver_register(struct mlx5_pci_driver *driver) TAILQ_INSERT_TAIL(&drv_list, driver, next); } +/********** New common PCI bus driver ********/ + +bool +mlx5_dev_is_pci(const struct rte_device *dev) +{ + return strcmp(dev->bus->name, "pci") == 0; +} + +bool +mlx5_dev_pci_match(const struct mlx5_class_driver *drv, + const struct rte_device *dev) +{ + const struct rte_pci_device *pci_dev; + const struct rte_pci_id *id_table; + + if (!mlx5_dev_is_pci(dev)) + return false; + pci_dev = RTE_DEV_TO_PCI_CONST(dev); + for (id_table = drv->id_table; id_table->vendor_id != 0; + id_table++) { + /* Check if device's ids match the class driver's ids. */ + if (id_table->vendor_id != pci_dev->id.vendor_id && + id_table->vendor_id != RTE_PCI_ANY_ID) + continue; + if (id_table->device_id != pci_dev->id.device_id && + id_table->device_id != RTE_PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + pci_dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != RTE_PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + pci_dev->id.subsystem_device_id && + id_table->subsystem_device_id != RTE_PCI_ANY_ID) + continue; + if (id_table->class_id != pci_dev->id.class_id && + id_table->class_id != RTE_CLASS_ANY_ID) + continue; + return true; + } + return false; +} + +static int +mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) +{ + return mlx5_common_dev_probe(&pci_dev->device); +} + +static int +mlx5_common_pci_remove(struct rte_pci_device *pci_dev) +{ + return mlx5_common_dev_remove(&pci_dev->device); +} + +static int +mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_map(&pci_dev->device, addr, iova, len); +} + +static int +mlx5_common_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_unmap(&pci_dev->device, addr, iova, len); +} + +void +mlx5_common_driver_on_register_pci(struct mlx5_class_driver *driver) +{ + if (driver->id_table != NULL) { + if (pci_ids_table_update(driver->id_table) != 0) + return; + } + if (driver->probe_again) + mlx5_common_pci_driver.drv_flags |= RTE_PCI_DRV_PROBE_AGAIN; + if (driver->intr_lsc) + mlx5_common_pci_driver.drv_flags |= RTE_PCI_DRV_INTR_LSC; + if (driver->intr_rmv) + mlx5_common_pci_driver.drv_flags |= RTE_PCI_DRV_INTR_RMV; +} + +static struct rte_pci_driver mlx5_common_pci_driver = { + .driver = { + .name = MLX5_PCI_DRIVER_NAME, + }, + .probe = mlx5_common_pci_probe, + .remove = mlx5_common_pci_remove, + .dma_map = mlx5_common_pci_dma_map, + .dma_unmap = mlx5_common_pci_dma_unmap, +}; + void mlx5_common_pci_init(void) { const struct rte_pci_id empty_table[] = { @@ -551,7 +655,7 @@ void mlx5_common_pci_init(void) */ if (mlx5_pci_id_table == NULL && pci_ids_table_update(empty_table)) return; - rte_pci_register(&mlx5_pci_driver); + rte_pci_register(&mlx5_common_pci_driver); } RTE_FINI(mlx5_common_pci_finish) @@ -560,8 +664,9 @@ RTE_FINI(mlx5_common_pci_finish) /* Constructor doesn't register with PCI bus if it failed * to build the table. */ - rte_pci_unregister(&mlx5_pci_driver); + rte_pci_unregister(&mlx5_common_pci_driver); free(mlx5_pci_id_table); } } + RTE_PMD_EXPORT_NAME(mlx5_common_pci, __COUNTER__); diff --git a/drivers/common/mlx5/mlx5_common_private.h b/drivers/common/mlx5/mlx5_common_private.h new file mode 100644 index 0000000000..791eb3cd77 --- /dev/null +++ b/drivers/common/mlx5/mlx5_common_private.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Mellanox Technologies, Ltd + */ + +#ifndef _MLX5_COMMON_PRIVATE_H_ +#define _MLX5_COMMON_PRIVATE_H_ + +#include + +#include "mlx5_common.h" + +#ifdef __cplusplus +extern "C" { +#endif /* __cplusplus */ + +/* Common bus driver: */ + +struct mlx5_common_device { + struct rte_device *dev; + TAILQ_ENTRY(mlx5_common_device) next; + uint32_t classes_loaded; +}; + +int mlx5_common_dev_probe(struct rte_device *eal_dev); +int mlx5_common_dev_remove(struct rte_device *eal_dev); +int mlx5_common_dev_dma_map(struct rte_device *dev, void *addr, uint64_t iova, + size_t len); +int mlx5_common_dev_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, + size_t len); + +/* Common PCI bus driver: */ + +void mlx5_common_driver_on_register_pci(struct mlx5_class_driver *driver); +bool mlx5_dev_pci_match(const struct mlx5_class_driver *drv, + const struct rte_device *dev); + +#ifdef __cplusplus +} +#endif /* __cplusplus */ + +#endif /* _MLX5_COMMON_PRIVATE_H_ */ diff --git a/drivers/common/mlx5/mlx5_common_utils.h b/drivers/common/mlx5/mlx5_common_utils.h index 613d29de0c..98e487e7ef 100644 --- a/drivers/common/mlx5/mlx5_common_utils.h +++ b/drivers/common/mlx5/mlx5_common_utils.h @@ -5,6 +5,8 @@ #ifndef RTE_PMD_MLX5_COMMON_UTILS_H_ #define RTE_PMD_MLX5_COMMON_UTILS_H_ +#include + #include "mlx5_common.h" /************************ mlx5 list *****************************/ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index e6586d6f6f..e9d43dc1e5 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -3,6 +3,8 @@ INTERNAL { haswell_broadwell_cpu; + mlx5_class_driver_register; + mlx5_common_init; mlx5_common_verbs_reg_mr; # WINDOWS_NO_EXPORT @@ -10,6 +12,7 @@ INTERNAL { mlx5_create_mr_ext; + mlx5_dev_is_pci; # WINDOWS_NO_EXPORT mlx5_dev_to_pci_addr; # WINDOWS_NO_EXPORT mlx5_devx_alloc_uar; # WINDOWS_NO_EXPORT @@ -137,6 +140,7 @@ INTERNAL { mlx5_os_alloc_pd; mlx5_os_dealloc_pd; mlx5_os_dereg_mr; + mlx5_os_get_ibv_dev; # WINDOWS_NO_EXPORT mlx5_os_get_ibv_device; # WINDOWS_NO_EXPORT mlx5_os_reg_mr; mlx5_os_umem_dereg; From patchwork Wed Jul 21 14:37:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96159 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 44F05A0C51; Wed, 21 Jul 2021 16:38:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 24BFA4111A; Wed, 21 Jul 2021 16:38:43 +0200 (CEST) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2083.outbound.protection.outlook.com [40.107.102.83]) by mails.dpdk.org (Postfix) with ESMTP id B7604410FD for ; Wed, 21 Jul 2021 16:38:41 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bMcwmBl1RZNnYOWSBeBh3oe5qFsAUwN936wYRiAvzNa1Q160UWjUwKJ+FaD7xWHalNTA99VS+nK674QEH158965WtjuCAWPgtqXmWhWGHfBB53sF569+0vg//zAAYl1tcJgWwQ/wqS07L+LX7dr7vSZbRpSRYBTJrCF2p4yooDCsp+o2pQ51w6ant6cpW5kNCwn4DtagdueKsRdQyij34ap9P1SoMgZVudF5YcL7HVhFPND4QvVwqKAet3GJLQngkFtiYCVZTYvdazzmW/dYcTLS73OLCn012rUlbRrcjdoebu1zP4wzCeYka8ATmfm3FFpK32vaZBTcJHOuqLqodA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XQtjscw4A12R0J+oKs8bFj6NCV8/bucOpOdNXyOy6ZA=; b=c3hQ7KvBDFO6FTiF2FRNCoM8vsaFlkgtynkhU5C+CReNEMFxho1nIjDzV0WHD1AicAMVOLYdas+fcMtuHIvL6u81FOMHLbt85jf0+amGFj9UqzqRSiYCLc8Qy/ipqGmvN7QGEtTXCTH1ethiqcGH/B78/qH3g/zh49hIp3HZFqZK8Vjx0aOhly8C7CBQL7VsAqosY4knyUHxI7WJfkt6L1LOwP2I71a9ncHg016JEkVEIZNNDg8JmY+gVkh04i7z1RK01edqWB0NsSKkLkhLSZHnfJmerwyX0nCm92cO3clzeUx0zi6o6DCCgYVSs3Qffje1TqfMin0gmqvOMp2+yg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XQtjscw4A12R0J+oKs8bFj6NCV8/bucOpOdNXyOy6ZA=; b=i2LZXhyQiEHhTjAZApWgxuHE6l5i52FEX6tPU1omaQ6a/qqBpU/v27NnpAhHztbnNW4t6rhTjNSGNqzVNrMK16oWf8KrJaaCUnZix0LZWS153Cn5HhuvEkwt/FWp1XaGOZVg09HtKK4TmEfa1Y3xareZrO/wrD/7zozn75v9A8ofiz9W4q5uyJpQrPUN3R67SDdwCte8ZFO88Cg305Fh6ZP7phTdvgpJXeT4y24I+GRbegkbiLOH2ShzjeOTV444qe4hXMvNyWxV4NYDpq4m4xUjYSBAcGckrN4gXrsselomduIeMQQB/dLlxKFORSp6/MuSYAnUHKaITpAZF/c61w== Received: from BN9PR03CA0658.namprd03.prod.outlook.com (2603:10b6:408:13b::33) by BN6PR12MB1633.namprd12.prod.outlook.com (2603:10b6:405:5::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.26; Wed, 21 Jul 2021 14:38:38 +0000 Received: from BN8NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13b:cafe::20) by BN9PR03CA0658.outlook.office365.com (2603:10b6:408:13b::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:38:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT019.mail.protection.outlook.com (10.13.176.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:38:38 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:38:35 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad , Shahaf Shuler Date: Wed, 21 Jul 2021 22:37:30 +0800 Message-ID: <20210721143743.24626-4-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7c3dfb37-c255-413a-4a97-08d94c553aa0 X-MS-TrafficTypeDiagnostic: BN6PR12MB1633: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:923; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ccX7mlsAq7DHthpOX/yc6Bs+gqbnJpHCegRhrNq0UHXafhA9O9DU1zlXz5xmsuFbF3o/BZbRkdriFgS6HbHF2KFaVfUAZdvLz4sLe5PtAvRJ9sc3+9PUM7ByFcImozkCPBP2LZc7blNXnv33YcFAq7eQ656klNQwI9jrMJm2mr9/1ajXQnRVM3hWKX9qSFvk/MQif4Q3jLWopz4wirGVmk6jt/tB/OlW1dNsQrc9J/a4MIYg9rC4nGmk8eaDkI+0BgcPa/08QIouRQj87YYVo89bKFpllCjg/D4evzGwMsXgKgxNDT/uoPbuT4bJzQjTZwzedqiwciDTIkp/RYvA131p19Wfg5wvvTdV5Kv3uTpNlNtxwY3kXNm+HPWNAlDdMUgPCNEdjIMDalEHJBpO7kxk9or69jI+TwG+4l4AIVjzKqf4PQmpIgB0Zfz097TJQiM1CioVglXBfdlRsHWqMgZGkNGAPsnUXVzEYPLZ69RbxvFrb+dUv4y4BgQ7/9dO/El/3G7eZCtg5AqJcXa6nEtGhvIl0PUIb4eMDrl3ih9O9g/fwwhLHo8PxqRoXmW5RCH81rZ4ytyAjWWU++JqM827p/GufEPBP2lrAYxZq0nOGDfAQY3U7WT3MAXLPUipCP9sKHoCTG9Hn8yr+T59CsRT/APIAaa8v/aJFQ1qpTmU1LgJ/SWLbfh3zlNHBbterCbK1MjNQiZC7+HrFqrp9w== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(136003)(39860400002)(396003)(346002)(46966006)(36840700001)(356005)(1076003)(82740400003)(83380400001)(2906002)(86362001)(478600001)(54906003)(47076005)(16526019)(6666004)(36860700001)(36906005)(186003)(426003)(55016002)(7636003)(8936002)(336012)(37006003)(2616005)(316002)(36756003)(26005)(5660300002)(6636002)(6286002)(4326008)(7696005)(6862004)(82310400003)(107886003)(70586007)(8676002)(70206006); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:38:38.7581 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7c3dfb37-c255-413a-4a97-08d94c553aa0 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1633 Subject: [dpdk-dev] [PATCH v4 03/16] common/mlx5: move description of PCI sysfs functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon The Linux-specific functions mlx5_get_pci_addr() and mlx5_get_ifname_sysfs() are better described in the .h file. The requirement for using mlx5_get_pci_addr() is explicit: the node /device must exist in the provided sysfs path. Signed-off-by: Thomas Monjalon Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/mlx5_common_os.c | 22 ------------------ drivers/common/mlx5/mlx5_common.h | 26 ++++++++++++++++++++++ 2 files changed, 26 insertions(+), 22 deletions(-) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 78a9723075..337e9df8cb 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -23,17 +23,6 @@ const struct mlx5_glue *mlx5_glue; #endif -/** - * Get PCI information by sysfs device path. - * - * @param dev_path - * Pointer to device sysfs folder name. - * @param[out] pci_addr - * PCI bus address output buffer. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ int mlx5_dev_to_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr) @@ -159,17 +148,6 @@ mlx5_translate_port_name(const char *port_name_in, port_info_out->name_type = MLX5_PHYS_PORT_NAME_TYPE_UNKNOWN; } -/** - * Get kernel interface name from IB device path. - * - * @param[in] ibdev_path - * Pointer to IB device path. - * @param[out] ifname - * Interface name output buffer. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ int mlx5_get_ifname_sysfs(const char *ibdev_path, char *ifname) { diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 6ba1e70223..1b811f2509 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -204,8 +204,34 @@ check_cqe(volatile struct mlx5_cqe *cqe, const uint16_t cqes_n, return MLX5_CQE_STATUS_SW_OWN; } +/* + * Get PCI address from sysfs of a PCI-related device. + * + * @param[in] dev_path + * The sysfs path should not point to the direct plain PCI device. + * Instead, the node "/device/" is used to access the real device. + * @param[out] pci_addr + * Parsed PCI address. + * + * @return + * - 0 on success. + * - Negative value and rte_errno is set otherwise. + */ __rte_internal int mlx5_dev_to_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr); + +/* + * Get kernel network interface name from sysfs IB device path. + * + * @param[in] ibdev_path + * The sysfs path to IB device. + * @param[out] ifname + * Interface name output of size IF_NAMESIZE. + * + * @return + * - 0 on success. + * - Negative value and rte_errno is set otherwise. + */ __rte_internal int mlx5_get_ifname_sysfs(const char *ibdev_path, char *ifname); From patchwork Wed Jul 21 14:37:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96160 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A6C39A0C51; Wed, 21 Jul 2021 16:38:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3B7A44111E; Wed, 21 Jul 2021 16:38:46 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2066.outbound.protection.outlook.com [40.107.244.66]) by mails.dpdk.org (Postfix) with ESMTP id F3BAA40DF8 for ; Wed, 21 Jul 2021 16:38:42 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nCKFeKLuDdnZH9gLQ2dHTRKrLFjVgMJRJnHQs8oPei7Ut1iLoyEP2gaGSXZziHK86FKYv8N+OvZkoFh4HkwpmOjn6B+HQjnXejKvnVGCYMCZgSLLITBLvQwIh5hITCWpgnD0+jgr9IZ8EzxO5I3PuRR1TJoT6RagMgU8PF3eBvFO7HXkIBk1lrvq7ZdNjPozRkzV88dpVfraR4ebpq5h5NsRKiQQltjzxP6K6vBZKXS8Jvr6zA9mu7ZshZ3o3gk2yqsUPYcs+hsXIztFrfsrZD9BQA4sf0kkGH+gG+kCLL0G8+fu1YmgMNNy36K0OgIpXk6i+AVbRL1NlmHu7ftvhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VIPU6E8K1nfAfHxs8kHhuvhM2tbvFEDNvDBaZ0dGVCk=; b=Alx2bmOj42HoDVNMzByhf0MOMyAeJJzXq1m4p0BY007aJVOBMIpXAk3rhRrUIci7CFiI2mjf0Nzp5gFsUdFPF48LKASGU1XoUAPKteqQbk5RtqQAT40DD+aqrceKAKbdKejlHWMgIAmdfiX4Kdo0wOAkCJLmEMzMfKjO7uqhDjq8kGw6uhmIIopTcq67FJ5DIqSVtTdffjeKg2FEGTpRQBaIb2M1mB8MFiL8fRx16R7zKLfBqA3Inq111YDkLq1w0y1OjDz0ddQ5i6JPr1fmAhOmDNnYjTPDEpntvtqsg6503EPV1vgyaP/HWPhMNoT4uQalhNGWLEnHXBx5KXAoMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=ashroe.eu smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VIPU6E8K1nfAfHxs8kHhuvhM2tbvFEDNvDBaZ0dGVCk=; b=nJZAszytakTmueMvdfG7kS8ICvgzUQNVLCWD0woojrid9bwqyrP0VcR3+2AAM8LSOPsnG26wBk9lCr+Sk7YHSj9re37SKZX35gtzeMmsZvrpXEHG2yUAZRJD9PS6JqbsfaMd65sgv8n45c0EqhwNSAhEMJKarwaZePCnvluy4ForoWYuoyulRHqRQCem+p0UdPyYEiJtj5wrho7TynUNEKTnwgFw7VQmXGW44stqaV1jlt3udu9o+XhNi3v4DmHCBtvLOTfogYEb8gXy2SJw2062a5azMAc6k/4aJQ+El7SpPgq30eyP0vMEowWLJO3a1rIBgYcsfv5bkN3bUNHs1Q== Received: from BN9PR03CA0658.namprd03.prod.outlook.com (2603:10b6:408:13b::33) by CY4PR1201MB2503.namprd12.prod.outlook.com (2603:10b6:903:d1::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.29; Wed, 21 Jul 2021 14:38:40 +0000 Received: from BN8NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13b:cafe::73) by BN9PR03CA0658.outlook.office365.com (2603:10b6:408:13b::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:38:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; ashroe.eu; dkim=none (message not signed) header.d=none;ashroe.eu; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT019.mail.protection.outlook.com (10.13.176.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:38:40 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:38:38 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Ray Kinsella Date: Wed, 21 Jul 2021 22:37:31 +0800 Message-ID: <20210721143743.24626-5-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3f8a77e6-44a3-4c04-8a94-08d94c553b9a X-MS-TrafficTypeDiagnostic: CY4PR1201MB2503: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:147; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uYXNVxIWqlxMNg3heN5WrrqfhSkHQP+d+Ztl7ZoSK98NboZ8gI3zUAItMOr06GlUuTESzsEMs1vFAS5HfNbpxVbdzzxYGMnvnm8AqkitOO/mup1PTTHPzmiq0bchaVCQbmW/IZwYRc9cEyzYLEN2M6I3P3zaYG/6oZzeRTNB1TCnx4rJaYHtAzVChfv4SPIm9RGU85NP3X0ltGTjw/HXeNwPw0H5P7MJYr1GLlG/acpfnoSRxUuMaCF3g17OzZFteHzAKlRxQ8VnOWOhFRdaXrksWdrIP0QiCaEUHsXzga5qzu7TOFPVvmN704p8XuRx5NnWyO4hV/Hz4rAIiJhA8ZEueplM8Fy6LZlEYjgSeqD3hZ3xL6qh3DAM+Jzoe/Kn/0TrLFoutXl/VtSyAEtU0X7BrIDd3nTUw5Q9nQp44o9YHr0IOoFSCivzwtk4RJ4dYSlYS7J3DANgD2Os+giwj5KmuFpfRqo4u5OfGzB72Kd21ePis3jPJG1UkIEUTOFqOb83D3gsJdTcDw8fTJCS5vDWYpts2OwzNQf+328hyXUFSblxLXv3TY+iGCxssXsdLpF6lk7O+Sln62FmyXLqHq7Y8ZeRTypEKfo2iPdBtuLlE4oCwO5HEsjaO0DCbsSR0oAup9pMulHtQAFrb+5psZpf2Cm7gyYVH5vnru8FZOVvc8XuxmzPGN3MGXtbGnLYwqOIqcZlthZ90iNSEi1hCw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(376002)(39860400002)(46966006)(36840700001)(2616005)(36860700001)(70206006)(16526019)(82740400003)(7696005)(7636003)(8676002)(26005)(336012)(8936002)(6666004)(37006003)(82310400003)(186003)(2906002)(86362001)(356005)(478600001)(6286002)(47076005)(4326008)(36906005)(6862004)(70586007)(426003)(5660300002)(55016002)(36756003)(1076003)(316002)(54906003)(83380400001)(6636002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:38:40.3931 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3f8a77e6-44a3-4c04-8a94-08d94c553b9a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1201MB2503 Subject: [dpdk-dev] [PATCH v4 04/16] common/mlx5: support auxiliary bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds auxiliary bus driver and delegate to registered internal mlx5 common device drivers, i.e. eth, vdpa... Current major target is to support SubFunction on auxiliary bus. As a limitation of current driver, numa node of device is detected from PCI bus of device symbol link, will remove once numa node file available on sysfs. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/meson.build | 3 + .../common/mlx5/linux/mlx5_common_auxiliary.c | 173 ++++++++++++++++++ drivers/common/mlx5/linux/mlx5_common_verbs.c | 5 +- drivers/common/mlx5/meson.build | 2 +- drivers/common/mlx5/mlx5_common.c | 3 + drivers/common/mlx5/mlx5_common.h | 6 + drivers/common/mlx5/mlx5_common_private.h | 6 + drivers/common/mlx5/version.map | 2 + 8 files changed, 198 insertions(+), 2 deletions(-) create mode 100644 drivers/common/mlx5/linux/mlx5_common_auxiliary.c diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build index 686df26909..44446e2743 100644 --- a/drivers/common/mlx5/linux/meson.build +++ b/drivers/common/mlx5/linux/meson.build @@ -48,10 +48,13 @@ endif sources += files('mlx5_nl.c') sources += files('mlx5_common_os.c') sources += files('mlx5_common_verbs.c') +sources += files('mlx5_common_auxiliary.c') if not dlopen_ibverbs sources += files('mlx5_glue.c') endif +deps += ['bus_auxiliary'] + # To maintain the compatibility with the make build system # mlx5_autoconf.h file is still generated. # input array for meson member search: diff --git a/drivers/common/mlx5/linux/mlx5_common_auxiliary.c b/drivers/common/mlx5/linux/mlx5_common_auxiliary.c new file mode 100644 index 0000000000..4ca27cd281 --- /dev/null +++ b/drivers/common/mlx5/linux/mlx5_common_auxiliary.c @@ -0,0 +1,173 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies Ltd + */ + +#include +#include +#include +#include +#include +#include +#include "eal_filesystem.h" + +#include "mlx5_common_utils.h" +#include "mlx5_common_private.h" + +#define AUXILIARY_SYSFS_PATH "/sys/bus/auxiliary/devices" +#define MLX5_AUXILIARY_PREFIX "mlx5_core.sf." + +int +mlx5_auxiliary_get_child_name(const char *dev, const char *node, + char *child, size_t size) +{ + DIR *dir; + struct dirent *dent; + MKSTR(path, "%s/%s%s", AUXILIARY_SYSFS_PATH, dev, node); + + dir = opendir(path); + if (dir == NULL) { + rte_errno = errno; + return -rte_errno; + } + /* Get the first file name. */ + while ((dent = readdir(dir)) != NULL) { + if (dent->d_name[0] != '.') + break; + } + closedir(dir); + if (dent == NULL) { + rte_errno = ENOENT; + return -rte_errno; + } + if (rte_strscpy(child, dent->d_name, size) < 0) + return -rte_errno; + return 0; +} + +static int +mlx5_auxiliary_get_pci_path(const struct rte_auxiliary_device *dev, + char *sysfs_pci, size_t size) +{ + char sysfs_real[PATH_MAX] = { 0 }; + MKSTR(sysfs_aux, "%s/%s", AUXILIARY_SYSFS_PATH, dev->name); + char *dir; + + if (realpath(sysfs_aux, sysfs_real) == NULL) { + rte_errno = errno; + return -rte_errno; + } + dir = dirname(sysfs_real); + if (dir == NULL) { + rte_errno = errno; + return -rte_errno; + } + if (rte_strscpy(sysfs_pci, dir, size) < 0) + return -rte_errno; + return 0; +} + +static int +mlx5_auxiliary_get_numa(const struct rte_auxiliary_device *dev) +{ + unsigned long numa; + char numa_path[PATH_MAX]; + + if (mlx5_auxiliary_get_pci_path(dev, numa_path, sizeof(numa_path)) != 0) + return SOCKET_ID_ANY; + if (strcat(numa_path, "/numa_node") == NULL) { + rte_errno = ENAMETOOLONG; + return SOCKET_ID_ANY; + } + if (eal_parse_sysfs_value(numa_path, &numa) != 0) { + rte_errno = EINVAL; + return SOCKET_ID_ANY; + } + return (int)numa; +} + +struct ibv_device * +mlx5_get_aux_ibv_device(const struct rte_auxiliary_device *dev) +{ + int n; + char ib_name[64] = { 0 }; + struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); + struct ibv_device *ibv_match = NULL; + + if (!ibv_list) { + rte_errno = ENOSYS; + return NULL; + } + if (mlx5_auxiliary_get_child_name(dev->name, "/infiniband", + ib_name, sizeof(ib_name)) != 0) + goto out; + while (n-- > 0) { + if (strcmp(ibv_list[n]->name, ib_name) != 0) + continue; + ibv_match = ibv_list[n]; + break; + } + if (ibv_match == NULL) + rte_errno = ENOENT; +out: + mlx5_glue->free_device_list(ibv_list); + return ibv_match; +} + +static bool +mlx5_common_auxiliary_match(const char *name) +{ + return strncmp(name, MLX5_AUXILIARY_PREFIX, + strlen(MLX5_AUXILIARY_PREFIX)) == 0; +} + +static int +mlx5_common_auxiliary_probe(struct rte_auxiliary_driver *drv __rte_unused, + struct rte_auxiliary_device *dev) +{ + dev->device.numa_node = mlx5_auxiliary_get_numa(dev); + return mlx5_common_dev_probe(&dev->device); +} + +static int +mlx5_common_auxiliary_remove(struct rte_auxiliary_device *auxiliary_dev) +{ + return mlx5_common_dev_remove(&auxiliary_dev->device); +} + +static int +mlx5_common_auxiliary_dma_map(struct rte_auxiliary_device *auxiliary_dev, + void *addr, uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_map(&auxiliary_dev->device, addr, iova, len); +} + +static int +mlx5_common_auxiliary_dma_unmap(struct rte_auxiliary_device *auxiliary_dev, + void *addr, uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_unmap(&auxiliary_dev->device, addr, iova, + len); +} + +static struct rte_auxiliary_driver mlx5_auxiliary_driver = { + .driver = { + .name = MLX5_AUXILIARY_DRIVER_NAME, + }, + .match = mlx5_common_auxiliary_match, + .probe = mlx5_common_auxiliary_probe, + .remove = mlx5_common_auxiliary_remove, + .dma_map = mlx5_common_auxiliary_dma_map, + .dma_unmap = mlx5_common_auxiliary_dma_unmap, +}; + +void mlx5_common_auxiliary_init(void) +{ + if (mlx5_auxiliary_driver.bus == NULL) + rte_auxiliary_register(&mlx5_auxiliary_driver); +} + +RTE_FINI(mlx5_common_auxiliary_driver_finish) +{ + if (mlx5_auxiliary_driver.bus != NULL) + rte_auxiliary_unregister(&mlx5_auxiliary_driver); +} diff --git a/drivers/common/mlx5/linux/mlx5_common_verbs.c b/drivers/common/mlx5/linux/mlx5_common_verbs.c index 6a6ab7a7a2..9080bd3e87 100644 --- a/drivers/common/mlx5/linux/mlx5_common_verbs.c +++ b/drivers/common/mlx5/linux/mlx5_common_verbs.c @@ -12,6 +12,7 @@ #include #include +#include #include "mlx5_common_utils.h" #include "mlx5_common_log.h" @@ -24,10 +25,12 @@ struct ibv_device * mlx5_os_get_ibv_dev(const struct rte_device *dev) { - struct ibv_device *ibv = NULL; + struct ibv_device *ibv; if (mlx5_dev_is_pci(dev)) ibv = mlx5_os_get_ibv_device(&RTE_DEV_TO_PCI_CONST(dev)->addr); + else + ibv = mlx5_get_aux_ibv_device(RTE_DEV_TO_AUXILIARY_CONST(dev)); if (ibv == NULL) { rte_errno = ENODEV; DRV_LOG(ERR, "Verbs device not found: %s", dev->name); diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build index fa2b8b9834..6ddbde7e8f 100644 --- a/drivers/common/mlx5/meson.build +++ b/drivers/common/mlx5/meson.build @@ -7,7 +7,7 @@ if not (is_linux or (is_windows and is_ms_linker)) subdir_done() endif -deps += ['hash', 'pci', 'bus_pci', 'net', 'eal', 'kvargs'] +deps += ['hash', 'pci', 'bus_pci', 'bus_auxiliary', 'net', 'eal', 'kvargs'] sources += files( 'mlx5_devx_cmds.c', 'mlx5_common.c', diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 97d3e3e60e..94f858a9bd 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -398,6 +398,9 @@ mlx5_class_driver_register(struct mlx5_class_driver *driver) static void mlx5_common_driver_init(void) { mlx5_common_pci_init(); +#ifdef RTE_EXEC_ENV_LINUX + mlx5_common_auxiliary_init(); +#endif } static bool mlx5_common_initialized; diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 1b811f2509..3cb961f495 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -24,6 +24,7 @@ /* Reported driver name. */ #define MLX5_PCI_DRIVER_NAME "mlx5_pci" +#define MLX5_AUXILIARY_DRIVER_NAME "mlx5_auxiliary" /* Bit-field manipulation. */ #define BITFIELD_DECLARE(bf, type, size) \ @@ -109,6 +110,7 @@ pmd_drv_log_basename(const char *s) int mkstr_size_##name = snprintf(NULL, 0, "" __VA_ARGS__); \ char name[mkstr_size_##name + 1]; \ \ + memset(name, 0, mkstr_size_##name + 1); \ snprintf(name, sizeof(name), "" __VA_ARGS__) enum { @@ -136,6 +138,10 @@ enum { PCI_DEVICE_ID_MELLANOX_CONNECTX7BF = 0Xa2dc, }; + +__rte_internal +int mlx5_auxiliary_get_child_name(const char *dev, const char *node, + char *child, size_t size); /* Maximum number of simultaneous unicast MAC addresses. */ #define MLX5_MAX_UC_MAC_ADDRESSES 128 /* Maximum number of simultaneous Multicast MAC addresses. */ diff --git a/drivers/common/mlx5/mlx5_common_private.h b/drivers/common/mlx5/mlx5_common_private.h index 791eb3cd77..9f00a6c54d 100644 --- a/drivers/common/mlx5/mlx5_common_private.h +++ b/drivers/common/mlx5/mlx5_common_private.h @@ -6,6 +6,7 @@ #define _MLX5_COMMON_PRIVATE_H_ #include +#include #include "mlx5_common.h" @@ -34,6 +35,11 @@ void mlx5_common_driver_on_register_pci(struct mlx5_class_driver *driver); bool mlx5_dev_pci_match(const struct mlx5_class_driver *drv, const struct rte_device *dev); +/* Common auxiliary bus driver: */ +void mlx5_common_auxiliary_init(void); +struct ibv_device *mlx5_get_aux_ibv_device( + const struct rte_auxiliary_device *dev); + #ifdef __cplusplus } #endif /* __cplusplus */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index e9d43dc1e5..b47d73b425 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -3,6 +3,8 @@ INTERNAL { haswell_broadwell_cpu; + mlx5_auxiliary_get_child_name; # WINDOWS_NO_EXPORT + mlx5_class_driver_register; mlx5_common_init; From patchwork Wed Jul 21 14:37:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96161 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C1479A0C51; Wed, 21 Jul 2021 16:39:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 59BCE410E1; Wed, 21 Jul 2021 16:39:04 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by mails.dpdk.org (Postfix) with ESMTP id DD840410DB for ; Wed, 21 Jul 2021 16:39:02 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=if9sQ3fL3SJCcMCQ+odHWryIKRaeLt48Wf7egXCcde5iuI2oKTRVRvRf94jY++ciEt8gJc1mZRt7b4O7NH8w14q0inqSPb4YHVoP7e0ylMRy3vOdRgkzvUL9TEZ1yFTmMnBcODupcF11sor1maEQRlf6YcPc5lu0hxTKeD+AlpIUfz9g3xB2R8XLZGPIhdpsNeEZb8W2vIlfPdDOJGsNrhm1gDNSSm/BrlNHsCEj4orVfaW0UHFv+8lqkXQryD9bgXu8Cp1P27hYXrJEeOTUKf27+CSxE6018gexPtVKfi7GTMD1KaIoBbQS2RbpIaBjA8mgqYpWv22YOWOqlvGDTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U3IsrHB9RAtOGxRs5J3xi6MtDZIhoUxBfEBFBApCmEY=; b=SlULtB2xkgm/7A1V+vcARCEmEb/SX8rXuDjKLY6NrOq0zj4+g0JW8wHFAdRTkG+01oOJolBQOEZrcnin3mkRMNSv2DWtcNlZRwu+LG87o5jhQq1mi3raXCI3KrJyEaUoHbA2OLVnzEkbVY3DTSupEmS6Au3VPy6CPsvpUTYACKl1DuaFl1MedBMhS/a9i/6QyA1ty868JGl8hjhKX7b3pHCnflyf3N8bMCCBTI1q46shtyc6qkcVHkxeiXCJrhs5OqgFm6rFO/eA9qUeLxk4hh5ZONMyUfC4OGlvJ2RUw353LEfVKuWrzFMdPFu8YryW3yJLoF0LeoYPyKOIlwB6Sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=ashroe.eu smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U3IsrHB9RAtOGxRs5J3xi6MtDZIhoUxBfEBFBApCmEY=; b=nzDqSnatgjCnw8YtYJrVB5eFusL1rrHQuBOHfaMjd7xqyRsJM1eO2Kf9UxqOcSGOzZKGqM1DbTOEeIPjq/fNIkB3R2Sgba3WaDWtgaynwJ2VTTgiYhDYXOdSjAe584XPYujmlM3TxiP+U35XKvZ7x+3iR8dR8B7w0DmUiD+vlqwkWHNY6MPpy0+Hjxn1pBVCSndECFczc+hF/TXI0ctAksdpUK0lQYRMLeOSqp+Yq9FWMbpggoB4n1cqATGqo+eDiGyFvxVinleByxj57rkcvZ6Ky2z5XqxBJySYg0dBSiF26dDpWGHsjky5B3IHcErOUtfOBxGYDTltCYGblHf91w== Received: from BN9PR03CA0275.namprd03.prod.outlook.com (2603:10b6:408:f5::10) by BY5PR12MB3697.namprd12.prod.outlook.com (2603:10b6:a03:1a2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.23; Wed, 21 Jul 2021 14:39:01 +0000 Received: from BN8NAM11FT005.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f5:cafe::3d) by BN9PR03CA0275.outlook.office365.com (2603:10b6:408:f5::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; ashroe.eu; dkim=none (message not signed) header.d=none;ashroe.eu; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT005.mail.protection.outlook.com (10.13.176.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:00 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:38:58 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad , Shahaf Shuler , Ray Kinsella Date: Wed, 21 Jul 2021 22:37:32 +0800 Message-ID: <20210721143743.24626-6-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3704efa9-a8c9-4e16-a15a-08d94c5547ad X-MS-TrafficTypeDiagnostic: BY5PR12MB3697: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:568; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MgUZtUjg38cvco78yfVsS6jiyryJzH4bs1DmAXJgLik3Tq7vKPsCNPndP//i63+lpyDssO5ulTesyb9ZA6eTjMUGIPY9S/Ulp0LCAWYMuU8d57BcvxUzl/V9MzVX5J9ESgF3RoheFkeM5SQo1HKpeceBVcNpAMFp9WhIRkB1Q7O1OO1jwkfuWPuQgBuVNQ6sfcqftTsZeSo1dqH9U3geULiEBGVamKs14jmqx2HkdzXoERUWIqnMgJzJhtC5ZZYq5IQ72K1p8miOdKoGhc+fyUKGqAVb6hgrtc/gvY4MdxfR4hy4mMKZRRUK3i7p6xVoA1PVGdtZ75TsD7AnOf8tbSfIDhu9JsCyYltGUL7s5hvOy1FpwTPg0opYeOA9piZqO6uucp8DSdn1NCvPvJvM28R6DtwPRtlcyUP1fMPB0JFIXbA4bSQi9+jd5sc3j/yRUwFwQiC0JXCkAQZT1lbsuszQSDJpYueacRJcnZpoilUnPmX8qiLlnYX7uxtxGwmkjteoVUc8C1ulFhsq6Z+EgC2/ualW6qowi/NsuBJwuXoGjLG0UlbUUR4biGS4dVTuVP7/7Ln6+q/Ya/2CAL5q8ASmA4taVi4JSSj/nlLMvoU2/JxBLs6VS/dDh40vL/anIlsRkAFHkzQKMyiBgkFP0XXg9PU++jAMUSOfH7dEmbBp5lz/+Pe1U4f4uM+uWMtD4emm5uxUETC3xrualNia9g== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(36840700001)(46966006)(82310400003)(316002)(37006003)(83380400001)(6286002)(8936002)(36906005)(336012)(16526019)(426003)(1076003)(186003)(2616005)(70206006)(47076005)(478600001)(55016002)(54906003)(70586007)(356005)(7636003)(26005)(4326008)(36860700001)(5660300002)(86362001)(2906002)(6862004)(36756003)(7696005)(6666004)(8676002)(82740400003)(6636002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:00.6509 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3704efa9-a8c9-4e16-a15a-08d94c5547ad X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT005.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB3697 Subject: [dpdk-dev] [PATCH v4 05/16] common/mlx5: get PCI device address from any bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon A function is exported to allow retrieving the PCI address of the parent PCI device of a Sub-Function in auxiliary bus sysfs. The function mlx5_dev_to_pci_str() is accepting both PCI and auxiliary devices. In case of a PCI device, it is simply using the device name. The function mlx5_dev_to_pci_addr(), which is based on sysfs path and do not use any device object, is renamed to mlx5_get_pci_addr() for clarity purpose. Signed-off-by: Thomas Monjalon Acked-by: Viacheslav Ovsiienko --- .../common/mlx5/linux/mlx5_common_auxiliary.c | 19 +++++++++++++++ drivers/common/mlx5/linux/mlx5_common_os.c | 5 ++-- drivers/common/mlx5/mlx5_common.c | 23 +++++++++++++++++++ drivers/common/mlx5/mlx5_common.h | 16 ++++++++++++- drivers/common/mlx5/mlx5_common_private.h | 2 ++ drivers/common/mlx5/version.map | 3 ++- drivers/net/mlx5/linux/mlx5_os.c | 6 ++--- 7 files changed, 66 insertions(+), 8 deletions(-) diff --git a/drivers/common/mlx5/linux/mlx5_common_auxiliary.c b/drivers/common/mlx5/linux/mlx5_common_auxiliary.c index 4ca27cd281..6584aeb18e 100644 --- a/drivers/common/mlx5/linux/mlx5_common_auxiliary.c +++ b/drivers/common/mlx5/linux/mlx5_common_auxiliary.c @@ -4,6 +4,8 @@ #include #include +#include + #include #include #include @@ -66,6 +68,23 @@ mlx5_auxiliary_get_pci_path(const struct rte_auxiliary_device *dev, return 0; } +int +mlx5_auxiliary_get_pci_str(const struct rte_auxiliary_device *dev, + char *addr, size_t size) +{ + char sysfs_pci[PATH_MAX]; + char *base; + + if (mlx5_auxiliary_get_pci_path(dev, sysfs_pci, sizeof(sysfs_pci)) != 0) + return -ENODEV; + base = basename(sysfs_pci); + if (base == NULL) + return -errno; + if (rte_strscpy(addr, base, size) < 0) + return -rte_errno; + return 0; +} + static int mlx5_auxiliary_get_numa(const struct rte_auxiliary_device *dev) { diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 337e9df8cb..9e0c823c97 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -24,8 +24,7 @@ const struct mlx5_glue *mlx5_glue; #endif int -mlx5_dev_to_pci_addr(const char *dev_path, - struct rte_pci_addr *pci_addr) +mlx5_get_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr) { FILE *file; char line[32]; @@ -417,7 +416,7 @@ mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) struct rte_pci_addr paddr; DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name); - if (mlx5_dev_to_pci_addr(ibv_list[n]->ibdev_path, &paddr) != 0) + if (mlx5_get_pci_addr(ibv_list[n]->ibdev_path, &paddr) != 0) continue; if (rte_pci_addr_cmp(addr, &paddr) != 0) continue; diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 94f858a9bd..8fe36f7077 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -197,6 +197,29 @@ to_mlx5_device(const struct rte_device *rte_dev) return NULL; } +int +mlx5_dev_to_pci_str(const struct rte_device *dev, char *addr, size_t size) +{ + struct rte_pci_addr pci_addr = { 0 }; + int ret; + + if (mlx5_dev_is_pci(dev)) { + /* Input might be , format PCI address to . */ + ret = rte_pci_addr_parse(dev->name, &pci_addr); + if (ret != 0) + return -ENODEV; + rte_pci_device_name(&pci_addr, addr, size); + return 0; + } +#ifdef RTE_EXEC_ENV_LINUX + return mlx5_auxiliary_get_pci_str(RTE_DEV_TO_AUXILIARY_CONST(dev), + addr, size); +#else + rte_errno = ENODEV; + return -rte_errno; +#endif +} + static void dev_release(struct mlx5_common_device *dev) { diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 3cb961f495..66c83047a1 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -210,6 +210,20 @@ check_cqe(volatile struct mlx5_cqe *cqe, const uint16_t cqes_n, return MLX5_CQE_STATUS_SW_OWN; } +/* + * Get PCI address string from EAL device. + * + * @param[out] addr + * The output address buffer string + * @param[in] size + * The output buffer size + * @return + * - 0 on success. + * - Negative value and rte_errno is set otherwise. + */ +__rte_internal +int mlx5_dev_to_pci_str(const struct rte_device *dev, char *addr, size_t size); + /* * Get PCI address from sysfs of a PCI-related device. * @@ -224,7 +238,7 @@ check_cqe(volatile struct mlx5_cqe *cqe, const uint16_t cqes_n, * - Negative value and rte_errno is set otherwise. */ __rte_internal -int mlx5_dev_to_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr); +int mlx5_get_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr); /* * Get kernel network interface name from sysfs IB device path. diff --git a/drivers/common/mlx5/mlx5_common_private.h b/drivers/common/mlx5/mlx5_common_private.h index 9f00a6c54d..1096fa85e7 100644 --- a/drivers/common/mlx5/mlx5_common_private.h +++ b/drivers/common/mlx5/mlx5_common_private.h @@ -39,6 +39,8 @@ bool mlx5_dev_pci_match(const struct mlx5_class_driver *drv, void mlx5_common_auxiliary_init(void); struct ibv_device *mlx5_get_aux_ibv_device( const struct rte_auxiliary_device *dev); +int mlx5_auxiliary_get_pci_str(const struct rte_auxiliary_device *dev, + char *addr, size_t size); #ifdef __cplusplus } diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index b47d73b425..a47c86e354 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -15,7 +15,7 @@ INTERNAL { mlx5_create_mr_ext; mlx5_dev_is_pci; # WINDOWS_NO_EXPORT - mlx5_dev_to_pci_addr; # WINDOWS_NO_EXPORT + mlx5_dev_to_pci_str; # WINDOWS_NO_EXPORT mlx5_devx_alloc_uar; # WINDOWS_NO_EXPORT @@ -75,6 +75,7 @@ INTERNAL { mlx5_free; mlx5_get_ifname_sysfs; # WINDOWS_NO_EXPORT + mlx5_get_pci_addr; # WINDOWS_NO_EXPORT mlx5_glue; diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index aa5210fa45..05b1761f6b 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1995,7 +1995,7 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, /* Process slave interface names in the loop. */ snprintf(tmp_str, sizeof(tmp_str), "/sys/class/net/%s", ifname); - if (mlx5_dev_to_pci_addr(tmp_str, &pci_addr)) { + if (mlx5_get_pci_addr(tmp_str, &pci_addr)) { DRV_LOG(WARNING, "can not get PCI address" " for netdev \"%s\"", ifname); continue; @@ -2172,8 +2172,8 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, break; } else { /* Bonding device not found. */ - if (mlx5_dev_to_pci_addr - (ibv_list[ret]->ibdev_path, &pci_addr)) + if (mlx5_get_pci_addr(ibv_list[ret]->ibdev_path, + &pci_addr)) continue; if (owner_pci.domain != pci_addr.domain || owner_pci.bus != pci_addr.bus || From patchwork Wed Jul 21 14:37:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96162 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BD9E4A0C51; Wed, 21 Jul 2021 16:39:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 58127410FC; Wed, 21 Jul 2021 16:39:05 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2042.outbound.protection.outlook.com [40.107.223.42]) by mails.dpdk.org (Postfix) with ESMTP id 40A5E410DB for ; Wed, 21 Jul 2021 16:39:04 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vj2QqdFygZe9tSKvTK8jZHk1PquKoLXZ6/a99SWhdKY7WK6hpQgr2dRqsrS+ksF21z1v6nmfwz2MI8rUAn7UPMa5i1yR5tI3u9voSx+Jo1CutMqjukevGP+ZsQfNoE8YULeOdGltbaAtJzEv5XhihrVcAR/o2uZefpvLdJshvAwVIvAbHMxM/ihHL4MBAFeUqlpeGxsjZ6AK/TjZ8UDjiRpclMgi9DD+gqPFzDZLe+MPuccc6ZGjR9r67VYerQpW7k5Ch0HJEx734FTDx2kH/hE/vQAJwkqYRHuRSniJRKx2+S7f24ooZ0pvXm2VdYYP0rHqHjkBy8DSkCsCHbWmkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tToz3V+4BB+3auy2OXLRYoJMso4ACVOStFBAiq3lx+s=; b=j2R8YGUSNdZXWA2JwxEqXG/Lrmxovz1DFrNPZSFDwqrmOcoF1wISV0yPZB7zFn8Z7QUTMEZWIE/40vvIfxrZ9XS01UiYOivJHNnnplvlXHM3hL7TCrKysG23Dcqb5v27nrqqixjrS1BIU/xy0xWlp0LSFqL7g+1bVGvkHCKzNQw4K9vnMK3fPUoXGgXx4+k4T4DbljC8eAK4gzo9/AwU4LJ7rEhB4qr7VZfV53yzOiuUAoZKAOrA7kq0CxjoJO0NVeEsoinPPyD6id1r33tcPuDUl/y8vGc7OHNzWgBKIaJLniWr8UJFw8fd6EnALPD7qxZ+Z/ziB9UAtMF6ybu7Xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tToz3V+4BB+3auy2OXLRYoJMso4ACVOStFBAiq3lx+s=; b=PfdxP27NivehowruQPTvwsqemmUbBm72+9Bp6XzUv4CP+5JsjzAUTwbN3+/lR8JG5o6QW/stQovYGEcGrAqDX7ci99e+KRKN8dl4Tqt5sYQE+kEcwETEVoImuZWccCH0ChPK1xeuBF/FPQm48qnx+OKuMDp9MIr3TWuIie92/D1GGYZDwdoLLZ8O3OAtxQrKNTXBmaRIKyMJh5jHxVefWCftWm5vUZEpDiZIDgu4cj6hGQlvnIT3aO37IrA9dk8mVGDnOQvjTkVh9b+ivuKW9Q/E30O8oxP0mTjvwXosOmWCK+kuOp0kfL/doonxCgS3cR14tI4Jb3p/y/Qq7rj7zg== Received: from BN9P220CA0030.NAMP220.PROD.OUTLOOK.COM (2603:10b6:408:13e::35) by DM6PR12MB3466.namprd12.prod.outlook.com (2603:10b6:5:3b::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.24; Wed, 21 Jul 2021 14:39:02 +0000 Received: from BN8NAM11FT046.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13e:cafe::f5) by BN9P220CA0030.outlook.office365.com (2603:10b6:408:13e::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT046.mail.protection.outlook.com (10.13.177.127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:02 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:00 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler Date: Wed, 21 Jul 2021 22:37:33 +0800 Message-ID: <20210721143743.24626-7-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3f65e8e1-5ab5-44f2-ceb2-08d94c5548e2 X-MS-TrafficTypeDiagnostic: DM6PR12MB3466: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:48; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GrSnqHXdGRjH8cxD1uqdT8dRohkyHK2YJ7w2aadvDIsqJtTSCEii4RmEIOUCtV22Y4Vg4GVitWucatnMBSNsJvkjAUauRCK6dq538Uzti8jMXZMIln/yHCXVxsPEoNS0ul5RR+aLHbswTl0+6HKq4yFOv6GnW4T+6PGDG0+8KnmhCYKLGRyWqGj87WOkGISK8cCRzPnwK5yJcgFo2wNUAY2peRh/lMOwTkOutQh9BehYLc6A3K7L7GDYIoV/M53CblTGWFiDB1Ea2CJkmIIhPo09xUHYDTvKBgExfQkhKdp+hZoNyb9D0n86NAL3kOs9E4dXleEQm1/2GjujgN7X11Q4eu/WgMjSQNzkLYiKI2TIah0UeOwxcfig+CetrcJxTmv59JtrLQE5kruNwXYwV/PNAT4Y5xsbNo4Dlgoj6nt8tolB3J2iJusMYW/rDYdFbpe23HuwArW0hZRx4z7d4tQ7Aa+GkZGjLp5tpqPbVjLmi6inhQnKmkve9JiS9NBrJ0D1VMW1tRDAU02a6IyF52Kf/ltW+XpPSB1q3NrENJDbb9G6U/n0oW+d2QWXnBld9tnRlK60sRE/RJExwhAMcBrE4hC18L/j6lw1tv/CZZzG8oLJxTIDiXQGBiZGZCfbVDMf2cSLjPbwF8TYXuhXDGeax3zvMnrpswyGLHpGLGnnFwmc+M6RFdHhIXFf3ge8vR1cpDcqg0DHtipqQIMt6w== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(136003)(39860400002)(376002)(346002)(36840700001)(46966006)(1076003)(86362001)(55016002)(54906003)(36906005)(36756003)(356005)(47076005)(83380400001)(7636003)(5660300002)(30864003)(26005)(82740400003)(16526019)(2906002)(316002)(336012)(7696005)(8676002)(186003)(6286002)(2616005)(426003)(70586007)(6666004)(107886003)(8936002)(36860700001)(6636002)(82310400003)(37006003)(6862004)(70206006)(478600001)(4326008); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:02.6161 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3f65e8e1-5ab5-44f2-ceb2-08d94c5548e2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT046.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3466 Subject: [dpdk-dev] [PATCH v4 06/16] net/mlx5: remove PCI dependency X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support more bus types, remove PCI dependency where possible. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_ethdev_os.c | 2 +- drivers/net/mlx5/linux/mlx5_os.c | 4 +-- drivers/net/mlx5/mlx5.c | 45 ++++++++++++++----------- drivers/net/mlx5/mlx5.h | 9 ++--- drivers/net/mlx5/mlx5_ethdev.c | 2 +- drivers/net/mlx5/mlx5_mr.c | 14 ++++---- drivers/net/mlx5/mlx5_trigger.c | 12 +++---- drivers/net/mlx5/mlx5_txq.c | 3 +- drivers/net/mlx5/windows/mlx5_os.c | 3 +- 9 files changed, 51 insertions(+), 43 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c index ddc1371aa9..b05b9fc950 100644 --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c @@ -346,7 +346,7 @@ mlx5_find_master_dev(struct rte_eth_dev *dev) priv = dev->data->dev_private; domain_id = priv->domain_id; MLX5_ASSERT(priv->representor); - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, dev->device) { struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; if (opriv && diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 05b1761f6b..4ab30fd244 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1252,7 +1252,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, * Look for sibling devices in order to reuse their switch domain * if any, otherwise allocate one. */ - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, NULL) { const struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; @@ -2511,6 +2511,7 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, dev_config.decap_en = 1; dev_config.log_hp_size = MLX5_ARG_UNSET; dev_config.allow_duplicate_pattern = 1; + list[i].numa_node = pci_dev->device.numa_node; list[i].eth_dev = mlx5_dev_spawn(&pci_dev->device, &list[i], &dev_config, @@ -2708,7 +2709,6 @@ mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn, int dbmap_env; int err = 0; - sh->numa_node = spawn->pci_dev->device.numa_node; pthread_mutex_init(&sh->txpp.mutex, NULL); /* * Configure environment variable "MLX5_BF_SHUT_UP" diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index d9c90d5ef9..8e64bf955b 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1120,6 +1120,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, rte_errno = ENOMEM; goto exit; } + sh->numa_node = spawn->numa_node; if (spawn->bond_info) sh->bond = *spawn->bond_info; err = mlx5_os_open_device(spawn, config, sh); @@ -1197,7 +1198,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, */ err = mlx5_mr_btree_init(&sh->share_cache.cache, MLX5_MR_BTREE_CACHE_N * 2, - spawn->pci_dev->device.numa_node); + sh->numa_node); if (err) { err = rte_errno; goto error; @@ -1635,7 +1636,7 @@ mlx5_dev_close(struct rte_eth_dev *dev) unsigned int c = 0; uint16_t port_id; - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, dev->device) { struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; @@ -2077,18 +2078,20 @@ mlx5_set_min_inline(struct mlx5_dev_spawn_data *spawn, { if (config->txq_inline_min != MLX5_ARG_UNSET) { /* Application defines size of inlined data explicitly. */ - switch (spawn->pci_dev->id.device_id) { - case PCI_DEVICE_ID_MELLANOX_CONNECTX4: - case PCI_DEVICE_ID_MELLANOX_CONNECTX4VF: - if (config->txq_inline_min < - (int)MLX5_INLINE_HSIZE_L2) { - DRV_LOG(DEBUG, - "txq_inline_mix aligned to minimal" - " ConnectX-4 required value %d", - (int)MLX5_INLINE_HSIZE_L2); - config->txq_inline_min = MLX5_INLINE_HSIZE_L2; + if (spawn->pci_dev != NULL) { + switch (spawn->pci_dev->id.device_id) { + case PCI_DEVICE_ID_MELLANOX_CONNECTX4: + case PCI_DEVICE_ID_MELLANOX_CONNECTX4VF: + if (config->txq_inline_min < + (int)MLX5_INLINE_HSIZE_L2) { + DRV_LOG(DEBUG, + "txq_inline_mix aligned to minimal ConnectX-4 required value %d", + (int)MLX5_INLINE_HSIZE_L2); + config->txq_inline_min = + MLX5_INLINE_HSIZE_L2; + } + break; } - break; } goto exit; } @@ -2142,6 +2145,10 @@ mlx5_set_min_inline(struct mlx5_dev_spawn_data *spawn, } } } + if (spawn->pci_dev == NULL) { + config->txq_inline_min = MLX5_INLINE_HSIZE_NONE; + goto exit; + } /* * We get here if we are unable to deduce * inline data size with DevX. Try PCI ID @@ -2276,7 +2283,7 @@ mlx5_dev_check_sibling_config(struct mlx5_priv *priv, if (sh->refcnt == 1) return 0; /* Find the device with shared context. */ - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, NULL) { struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; @@ -2307,25 +2314,25 @@ mlx5_dev_check_sibling_config(struct mlx5_priv *priv, * * @param[in] port_id * port_id to start looking for device. - * @param[in] pci_dev - * Pointer to the hint PCI device. When device is being probed + * @param[in] odev + * Pointer to the hint device. When device is being probed * the its siblings (master and preceding representors might * not have assigned driver yet (because the mlx5_os_pci_probe() - * is not completed yet, for this case match on hint PCI + * is not completed yet, for this case match on hint * device may be used to detect sibling device. * * @return * port_id of found device, RTE_MAX_ETHPORT if not found. */ uint16_t -mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev) +mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) { while (port_id < RTE_MAX_ETHPORTS) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; if (dev->state != RTE_ETH_DEV_UNUSED && dev->device && - (dev->device == &pci_dev->device || + (dev->device == odev || (dev->device->driver && dev->device->driver->name && !strcmp(dev->device->driver->name, MLX5_PCI_DRIVER_NAME)))) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 94618e10fa..749a9e95d4 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -134,6 +134,7 @@ struct mlx5_dev_spawn_data { uint32_t max_port; /**< Device maximal port index. */ uint32_t phys_port; /**< Device physical port index. */ int pf_bond; /**< bonding device PF index. < 0 - no bonding */ + int numa_node; /**< Device numa node. */ struct mlx5_switch_info info; /**< Switch information. */ void *phys_dev; /**< Associated physical device. */ struct rte_eth_dev *eth_dev; /**< Associated Ethernet device. */ @@ -1462,16 +1463,16 @@ int mlx5_proc_priv_init(struct rte_eth_dev *dev); void mlx5_proc_priv_uninit(struct rte_eth_dev *dev); int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev, struct rte_eth_udp_tunnel *udp_tunnel); -uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev); +uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev); int mlx5_dev_close(struct rte_eth_dev *dev); bool mlx5_is_hpf(struct rte_eth_dev *dev); void mlx5_age_event_prepare(struct mlx5_dev_ctx_shared *sh); /* Macro to iterate over all valid ports for mlx5 driver. */ -#define MLX5_ETH_FOREACH_DEV(port_id, pci_dev) \ - for (port_id = mlx5_eth_find_next(0, pci_dev); \ +#define MLX5_ETH_FOREACH_DEV(port_id, dev) \ + for (port_id = mlx5_eth_find_next(0, dev); \ port_id < RTE_MAX_ETHPORTS; \ - port_id = mlx5_eth_find_next(port_id + 1, pci_dev)) + port_id = mlx5_eth_find_next(port_id + 1, dev)) int mlx5_args(struct mlx5_dev_config *config, struct rte_devargs *devargs); struct mlx5_dev_ctx_shared * mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index c32eeefd88..82e2284d98 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -335,7 +335,7 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) if (priv->representor) { uint16_t port_id; - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, dev->device) { struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index 89c43fc9e9..87fb4371a4 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -199,23 +199,23 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, } /** - * Finds the first ethdev that match the pci device. + * Finds the first ethdev that match the device. * The existence of multiple ethdev per pci device is only with representors. * On such case, it is enough to get only one of the ports as they all share * the same ibv context. * - * @param pdev - * Pointer to the PCI device. + * @param dev + * Pointer to the device. * * @return * Pointer to the ethdev if found, NULL otherwise. */ static struct rte_eth_dev * -pci_dev_to_eth_dev(struct rte_pci_device *pdev) +dev_to_eth_dev(struct rte_device *dev) { uint16_t port_id; - port_id = rte_eth_find_next_of(0, &pdev->device); + port_id = rte_eth_find_next_of(0, dev); if (port_id == RTE_MAX_ETHPORTS) return NULL; return &rte_eth_devices[port_id]; @@ -245,7 +245,7 @@ mlx5_dma_map(struct rte_pci_device *pdev, void *addr, struct mlx5_priv *priv; struct mlx5_dev_ctx_shared *sh; - dev = pci_dev_to_eth_dev(pdev); + dev = dev_to_eth_dev(&pdev->device); if (!dev) { DRV_LOG(WARNING, "unable to find matching ethdev " "to PCI device %p", (void *)pdev); @@ -295,7 +295,7 @@ mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, struct mlx5_mr *mr; struct mr_cache_entry entry; - dev = pci_dev_to_eth_dev(pdev); + dev = dev_to_eth_dev(&pdev->device); if (!dev) { DRV_LOG(WARNING, "unable to find matching ethdev " "to PCI device %p", (void *)pdev); diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index 7cb8920d6b..6d2351f5a8 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -697,7 +697,7 @@ mlx5_hairpin_bind_single_port(struct rte_eth_dev *dev, uint16_t rx_port) uint32_t explicit; uint16_t rx_queue; - if (mlx5_eth_find_next(rx_port, priv->pci_dev) != rx_port) { + if (mlx5_eth_find_next(rx_port, dev->device) != rx_port) { rte_errno = ENODEV; DRV_LOG(ERR, "Rx port %u does not belong to mlx5", rx_port); return -rte_errno; @@ -835,7 +835,7 @@ mlx5_hairpin_unbind_single_port(struct rte_eth_dev *dev, uint16_t rx_port) int ret; uint16_t cur_port = priv->dev_data->port_id; - if (mlx5_eth_find_next(rx_port, priv->pci_dev) != rx_port) { + if (mlx5_eth_find_next(rx_port, dev->device) != rx_port) { rte_errno = ENODEV; DRV_LOG(ERR, "Rx port %u does not belong to mlx5", rx_port); return -rte_errno; @@ -893,7 +893,6 @@ mlx5_hairpin_bind(struct rte_eth_dev *dev, uint16_t rx_port) { int ret = 0; uint16_t p, pp; - struct mlx5_priv *priv = dev->data->dev_private; /* * If the Rx port has no hairpin configuration with the current port, @@ -902,7 +901,7 @@ mlx5_hairpin_bind(struct rte_eth_dev *dev, uint16_t rx_port) * information updating. */ if (rx_port == RTE_MAX_ETHPORTS) { - MLX5_ETH_FOREACH_DEV(p, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(p, dev->device) { ret = mlx5_hairpin_bind_single_port(dev, p); if (ret != 0) goto unbind; @@ -912,7 +911,7 @@ mlx5_hairpin_bind(struct rte_eth_dev *dev, uint16_t rx_port) return mlx5_hairpin_bind_single_port(dev, rx_port); } unbind: - MLX5_ETH_FOREACH_DEV(pp, priv->pci_dev) + MLX5_ETH_FOREACH_DEV(pp, dev->device) if (pp < p) mlx5_hairpin_unbind_single_port(dev, pp); return ret; @@ -927,10 +926,9 @@ mlx5_hairpin_unbind(struct rte_eth_dev *dev, uint16_t rx_port) { int ret = 0; uint16_t p; - struct mlx5_priv *priv = dev->data->dev_private; if (rx_port == RTE_MAX_ETHPORTS) - MLX5_ETH_FOREACH_DEV(p, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(p, dev->device) { ret = mlx5_hairpin_unbind_single_port(dev, p); if (ret != 0) return ret; diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 3e5e94444b..11770aeeef 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include @@ -816,7 +817,7 @@ txq_set_params(struct mlx5_txq_ctrl *txq_ctrl) if (config->txqs_inline == MLX5_ARG_UNSET) txqs_inline = #if defined(RTE_ARCH_ARM64) - (priv->pci_dev->id.device_id == + (priv->pci_dev && priv->pci_dev->id.device_id == PCI_DEVICE_ID_MELLANOX_CONNECTX5BF) ? MLX5_INLINE_MAX_TXQS_BLUEFIELD : #endif diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index 5da362a9d5..bf20adaa30 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -391,7 +391,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, * Look for sibling devices in order to reuse their switch domain * if any, otherwise allocate one. */ - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, NULL) { const struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; @@ -1080,6 +1080,7 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, dev_config.dv_flow_en = 1; dev_config.decap_en = 0; dev_config.log_hp_size = MLX5_ARG_UNSET; + list[ns].numa_node = pci_dev->device.numa_node; list[ns].eth_dev = mlx5_dev_spawn(&pci_dev->device, &list[ns], &dev_config); From patchwork Wed Jul 21 14:37:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96163 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 42EF5A0C51; Wed, 21 Jul 2021 16:39:20 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C99D140DF8; Wed, 21 Jul 2021 16:39:07 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2051.outbound.protection.outlook.com [40.107.236.51]) by mails.dpdk.org (Postfix) with ESMTP id 3761A4111B for ; Wed, 21 Jul 2021 16:39:06 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Lk+nlCL4YtGGgpgDLaKy1mxFLW8nVTRorh84S+ImovI2Qms5N9E+NIB10yQ0rdnIKjQ8zqNV/MO5WThp1/UgaVM873EUo8ZCVQgaoe3WG4TQ8JbDfANNVPAqRD9Am+9laDuNo2DGTvMawxy5nYt4yMHDygBpD8OzFu09bleykd/c+GfPX+l9j91L8+nO5qddBvyzvG4atVd1HEcn+OMLBII/uWBgTyUux1ArTSPYK03L5Sdf3fx3BrWhDq6P7BvNc+z2BQh9sOORPS9QQzKVLCeMkDASiaSgatgIkodISQTXJyokDqVK3GD7GfQl61e1kT3tln5qxbK+dF+lkbSTmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2OC8x07EvahJEtqqbioimW3WkdBkaBML+5KfrIlCOOg=; b=mLVcBntpM/FIHWRMWZOLE5cXN6wOorgyXb76oHVJKTfzXH3WwMdDsu5R7SdTSuuXQ9vFsHzXE4OSZvx57B7N7zQukxRdGq47TXNO3e4sbkG2jQ4XzzxM4bGPvyyKXXWe9nef3THYI9tJ+N4d92q2VXAEPIiVOBp18AZS23eL+3qL22rVDsJiNsBGhE+7wXrLB6FCK8fO9qutlrxB+lfxoKPGZq9dmO/+jLJJ7UUPCUL3fbBYKNCJdJ8b5BHJ0/49jEP/ItxZD0ZMmkrcTxe1GOps8mZU28T++Res85mVYclEjvB7ejbXDxhzRAo9C8ahJhpk1JlEuZGW3VoE6ERC0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2OC8x07EvahJEtqqbioimW3WkdBkaBML+5KfrIlCOOg=; b=pcy04p9003/EyKRpQO9wPQ9PbxDxT+wh+MrZzb0CiNdu6SIhYiHPqNPmzt15hizBBZHjMdaCabztlgHPPLm9IadHxbyAaMv3CP4WYv68Ct1CSNPApiw5MJuET1J3ZEuxX3n5qQhC4z5zUMa4l4kPkbPipTZMUbUshiy71dnZjpPLCJnL615Wl2ySoD2SNKNew/Ovf6XwF+kbIa5e+KSuEFwGrfcZ6fJsmpwU2xWUyjZR8tytWkhow1rIdIgDZ8AGjXkgrfn3gbe0PH6cfa9f6Ho3B3RRwxcrpL+Z1CAYewCenUR3vyZu+IHzX273A9osLAcqZUXzeB/C6qBn+ud5tw== Received: from BN0PR04CA0023.namprd04.prod.outlook.com (2603:10b6:408:ee::28) by DM5PR12MB1196.namprd12.prod.outlook.com (2603:10b6:3:74::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.25; Wed, 21 Jul 2021 14:39:05 +0000 Received: from BN8NAM11FT039.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ee:cafe::a) by BN0PR04CA0023.outlook.office365.com (2603:10b6:408:ee::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Wed, 21 Jul 2021 14:39:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT039.mail.protection.outlook.com (10.13.177.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:04 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:02 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Anatoly Burakov Date: Wed, 21 Jul 2021 22:37:34 +0800 Message-ID: <20210721143743.24626-8-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f9673eca-0576-4a94-5879-08d94c554a2b X-MS-TrafficTypeDiagnostic: DM5PR12MB1196: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:30; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wkDiPH/gKednNPIELdGjd/aT6Yw7b6HLHZ6aKsV4xODm+6g0GSkbKxzE56lowlDKthm5SLH5cYahJMgv2tXRGI5o8suv8RHUQhKzhtxGuoSpJrr8kFxip0dHNBCHwxA4n2N8nY/dKbEDUb2U5EyHYB1RaVq+ykVQNZybuIKhh4k7lxX3r5or3aHycunKA+DW0dhg7+vXZzDJSuxK+g6Ko5ccDpBAQVRLBHesTGcs5AoWImhS3t2Ffzd7AyUanIHU28nbwOFNz+WfIv7jqJ5NM9Wiw1COK/Ve+CKEwBPjpa1yZBtQJqy51bCCwjsOzQIGRcvOf0ogigQ+s1WiMTnyPF/lPdbxnsdRzkyrYV1RnNnNJT3+UMyQaTndRYnK2j0OQCjH5KQE2PW4b2cIvFnLSgyn0QwF3pCSaEviEFGc7m00RL/5Iwp2TpAsadNglV5Rg8kciCh6R/NSvztNQdkfRvghgSXGK5KltPulYJMdOaj9N5+VmXyiDcWQdorgJgMkezHTGTi7gAeW/ASnEvWprhu18W5NDdYHH5W99GEk/DWZWk4MzX1PBJWRJ8jMQX3sV6Sx7al3DVwqyfSYONSVx6X7lUJ8x82xHiqySNZ9blJuC4rcUyP3RbdTm5BK+OUTgfORaX9XwMayK3XnWxYgdZAFeuaXpsBFWlyhh1i0RW5XkRZV1UiMYet3fdOaSCKbAXPGY0FNnJPeCoK0NF5sDw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(39860400002)(346002)(396003)(136003)(46966006)(36840700001)(478600001)(2616005)(356005)(8936002)(70206006)(70586007)(83380400001)(1076003)(7636003)(4326008)(26005)(8676002)(86362001)(82740400003)(7696005)(336012)(54906003)(6666004)(2906002)(36906005)(6862004)(316002)(55016002)(30864003)(426003)(82310400003)(36860700001)(37006003)(6286002)(6636002)(186003)(47076005)(16526019)(5660300002)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:04.8252 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f9673eca-0576-4a94-5879-08d94c554a2b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT039.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1196 Subject: [dpdk-dev] [PATCH v4 07/16] net/mlx5: migrate to bus-agnostic common driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 46 ++++++++++++++++++++---------- drivers/net/mlx5/linux/mlx5_os.h | 3 -- drivers/net/mlx5/mlx5.c | 48 +++++++++++++++----------------- drivers/net/mlx5/mlx5.h | 3 +- drivers/net/mlx5/mlx5_mr.c | 38 ++++++++++++------------- drivers/net/mlx5/mlx5_rxtx.h | 9 +++--- 6 files changed, 78 insertions(+), 69 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 4ab30fd244..1b7ee419d1 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -2115,14 +2115,6 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, struct mlx5_bond_info bond_info; int ret = -1; - if (rte_eal_process_type() == RTE_PROC_PRIMARY) - mlx5_pmd_socket_init(); - ret = mlx5_init_once(); - if (ret) { - DRV_LOG(ERR, "unable to init PMD global data: %s", - strerror(rte_errno)); - return -rte_errno; - } errno = 0; ibv_list = mlx5_glue->get_device_list(&ret); if (!ibv_list) { @@ -2569,21 +2561,18 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, } /** - * DPDK callback to register a PCI device. + * Callback to register a PCI device. * * This function spawns Ethernet devices out of a given PCI device. * - * @param[in] pci_drv - * PCI driver structure (mlx5_driver). * @param[in] pci_dev * PCI device information. * * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ -int -mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) +static int +mlx5_os_pci_probe(struct rte_pci_device *pci_dev) { struct rte_eth_devargs eth_da = { .type = RTE_ETH_REPRESENTOR_NONE }; int ret = 0; @@ -2622,6 +2611,35 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, return ret; } +/** + * Net class driver callback to probe a device. + * + * This function probe PCI bus device(s). + * + * @param[in] dev + * Pointer to the generic device. + * + * @return + * 0 on success, the function cannot fail. + */ +int +mlx5_os_net_probe(struct rte_device *dev) +{ + int ret; + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + mlx5_pmd_socket_init(); + ret = mlx5_init_once(); + if (ret) { + DRV_LOG(ERR, "unable to init PMD global data: %s", + strerror(rte_errno)); + return -rte_errno; + } + if (mlx5_dev_is_pci(dev)) + return mlx5_os_pci_probe(RTE_DEV_TO_PCI(dev)); + return 0; +} + static int mlx5_config_doorbell_mapping_env(const struct mlx5_dev_config *config) { diff --git a/drivers/net/mlx5/linux/mlx5_os.h b/drivers/net/mlx5/linux/mlx5_os.h index 4ae7d0ef47..af7cbeb418 100644 --- a/drivers/net/mlx5/linux/mlx5_os.h +++ b/drivers/net/mlx5/linux/mlx5_os.h @@ -19,7 +19,4 @@ enum { #define MLX5_NAMESIZE IF_NAMESIZE -#define PCI_DRV_FLAGS (RTE_PCI_DRV_INTR_LSC | \ - RTE_PCI_DRV_INTR_RMV | \ - RTE_PCI_DRV_PROBE_AGAIN) #endif /* RTE_PMD_MLX5_OS_H_ */ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 8e64bf955b..96e8d189ba 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -12,7 +12,6 @@ #include #include -#include #include #include #include @@ -28,7 +27,6 @@ #include #include #include -#include #include #include "mlx5_defs.h" @@ -43,6 +41,8 @@ #include "mlx5_flow_os.h" #include "rte_pmd_mlx5.h" +#define MLX5_ETH_DRIVER_NAME mlx5_eth + /* Device parameter to enable RX completion queue compression. */ #define MLX5_RXQ_CQE_COMP_EN "rxq_cqe_comp_en" @@ -2345,23 +2345,23 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) } /** - * DPDK callback to remove a PCI device. + * Callback to remove a device. * - * This function removes all Ethernet devices belong to a given PCI device. + * This function removes all Ethernet devices belong to a given device. * - * @param[in] pci_dev - * Pointer to the PCI device. + * @param[in] dev + * Pointer to the generic device. * * @return * 0 on success, the function cannot fail. */ static int -mlx5_pci_remove(struct rte_pci_device *pci_dev) +mlx5_net_remove(struct rte_device *dev) { uint16_t port_id; int ret = 0; - RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) { + RTE_ETH_FOREACH_DEV_OF(port_id, dev) { /* * mlx5_dev_close() is not registered to secondary process, * call the close function explicitly for secondary process. @@ -2452,19 +2452,17 @@ static const struct rte_pci_id mlx5_pci_id_map[] = { } }; -static struct mlx5_pci_driver mlx5_driver = { - .driver_class = MLX5_CLASS_ETH, - .pci_driver = { - .driver = { - .name = MLX5_PCI_DRIVER_NAME, - }, - .id_table = mlx5_pci_id_map, - .probe = mlx5_os_pci_probe, - .remove = mlx5_pci_remove, - .dma_map = mlx5_dma_map, - .dma_unmap = mlx5_dma_unmap, - .drv_flags = PCI_DRV_FLAGS, - }, +static struct mlx5_class_driver mlx5_net_driver = { + .drv_class = MLX5_CLASS_ETH, + .name = RTE_STR(MLX5_ETH_DRIVER_NAME), + .id_table = mlx5_pci_id_map, + .probe = mlx5_os_net_probe, + .remove = mlx5_net_remove, + .dma_map = mlx5_net_dma_map, + .dma_unmap = mlx5_net_dma_unmap, + .probe_again = 1, + .intr_lsc = 1, + .intr_rmv = 1, }; /* Initialize driver log type. */ @@ -2482,9 +2480,9 @@ RTE_INIT(rte_mlx5_pmd_init) mlx5_set_cksum_table(); mlx5_set_swp_types_table(); if (mlx5_glue) - mlx5_pci_driver_register(&mlx5_driver); + mlx5_class_driver_register(&mlx5_net_driver); } -RTE_PMD_EXPORT_NAME(net_mlx5, __COUNTER__); -RTE_PMD_REGISTER_PCI_TABLE(net_mlx5, mlx5_pci_id_map); -RTE_PMD_REGISTER_KMOD_DEP(net_mlx5, "* ib_uverbs & mlx5_core & mlx5_ib"); +RTE_PMD_EXPORT_NAME(MLX5_ETH_DRIVER_NAME, __COUNTER__); +RTE_PMD_REGISTER_PCI_TABLE(MLX5_ETH_DRIVER_NAME, mlx5_pci_id_map); +RTE_PMD_REGISTER_KMOD_DEP(MLX5_ETH_DRIVER_NAME, "* ib_uverbs & mlx5_core & mlx5_ib"); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 749a9e95d4..d88b1433fb 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1751,8 +1751,7 @@ int mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn, const struct mlx5_dev_config *config, struct mlx5_dev_ctx_shared *sh); int mlx5_os_get_pdn(void *pd, uint32_t *pdn); -int mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev); +int mlx5_os_net_probe(struct rte_device *dev); void mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh); void mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh); void mlx5_os_set_reg_mr_cb(mlx5_reg_mr_t *reg_mr_cb, diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index 87fb4371a4..44afda731f 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -7,7 +7,6 @@ #include #include #include -#include #include #include @@ -222,10 +221,10 @@ dev_to_eth_dev(struct rte_device *dev) } /** - * DPDK callback to DMA map external memory to a PCI device. + * Callback to DMA map external memory to a device. * - * @param pdev - * Pointer to the PCI device. + * @param rte_dev + * Pointer to the generic device. * @param addr * Starting virtual address of memory to be mapped. * @param iova @@ -237,18 +236,18 @@ dev_to_eth_dev(struct rte_device *dev) * 0 on success, negative value on error. */ int -mlx5_dma_map(struct rte_pci_device *pdev, void *addr, - uint64_t iova __rte_unused, size_t len) +mlx5_net_dma_map(struct rte_device *rte_dev, void *addr, + uint64_t iova __rte_unused, size_t len) { struct rte_eth_dev *dev; struct mlx5_mr *mr; struct mlx5_priv *priv; struct mlx5_dev_ctx_shared *sh; - dev = dev_to_eth_dev(&pdev->device); + dev = dev_to_eth_dev(rte_dev); if (!dev) { DRV_LOG(WARNING, "unable to find matching ethdev " - "to PCI device %p", (void *)pdev); + "to device %s", rte_dev->name); rte_errno = ENODEV; return -1; } @@ -271,10 +270,10 @@ mlx5_dma_map(struct rte_pci_device *pdev, void *addr, } /** - * DPDK callback to DMA unmap external memory to a PCI device. + * Callback to DMA unmap external memory to a device. * - * @param pdev - * Pointer to the PCI device. + * @param rte_dev + * Pointer to the generic device. * @param addr * Starting virtual address of memory to be unmapped. * @param iova @@ -286,8 +285,8 @@ mlx5_dma_map(struct rte_pci_device *pdev, void *addr, * 0 on success, negative value on error. */ int -mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, - uint64_t iova __rte_unused, size_t len __rte_unused) +mlx5_net_dma_unmap(struct rte_device *rte_dev, void *addr, + uint64_t iova __rte_unused, size_t len __rte_unused) { struct rte_eth_dev *dev; struct mlx5_priv *priv; @@ -295,10 +294,10 @@ mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, struct mlx5_mr *mr; struct mr_cache_entry entry; - dev = dev_to_eth_dev(&pdev->device); + dev = dev_to_eth_dev(rte_dev); if (!dev) { - DRV_LOG(WARNING, "unable to find matching ethdev " - "to PCI device %p", (void *)pdev); + DRV_LOG(WARNING, "unable to find matching ethdev to device %s", + rte_dev->name); rte_errno = ENODEV; return -1; } @@ -308,16 +307,15 @@ mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, mr = mlx5_mr_lookup_list(&sh->share_cache, &entry, (uintptr_t)addr); if (!mr) { rte_rwlock_write_unlock(&sh->share_cache.rwlock); - DRV_LOG(WARNING, "address 0x%" PRIxPTR " wasn't registered " - "to PCI device %p", (uintptr_t)addr, - (void *)pdev); + DRV_LOG(WARNING, "address 0x%" PRIxPTR " wasn't registered to device %s", + (uintptr_t)addr, rte_dev->name); rte_errno = EINVAL; return -1; } LIST_REMOVE(mr, mr); - mlx5_mr_free(mr, sh->share_cache.dereg_mr_cb); DRV_LOG(DEBUG, "port %u remove MR(%p) from list", dev->data->port_id, (void *)mr); + mlx5_mr_free(mr, sh->share_cache.dereg_mr_cb); mlx5_mr_rebuild_cache(&sh->share_cache); /* * No explicit wmb is needed after updating dev_gen due to diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index e168dd46f9..ad1144e218 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -16,7 +16,6 @@ #include #include #include -#include #include #include @@ -48,10 +47,10 @@ int mlx5_queue_state_modify(struct rte_eth_dev *dev, /* mlx5_mr.c */ void mlx5_mr_flush_local_cache(struct mlx5_mr_ctrl *mr_ctrl); -int mlx5_dma_map(struct rte_pci_device *pdev, void *addr, uint64_t iova, - size_t len); -int mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, uint64_t iova, - size_t len); +int mlx5_net_dma_map(struct rte_device *rte_dev, void *addr, uint64_t iova, + size_t len); +int mlx5_net_dma_unmap(struct rte_device *rte_dev, void *addr, uint64_t iova, + size_t len); /** * Get Memory Pool (MP) from mbuf. If mbuf is indirect, the pool from which the From patchwork Wed Jul 21 14:37:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96165 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 778C1A0C51; Wed, 21 Jul 2021 16:39:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6020941124; Wed, 21 Jul 2021 16:39:13 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2060.outbound.protection.outlook.com [40.107.244.60]) by mails.dpdk.org (Postfix) with ESMTP id 6C40241124 for ; Wed, 21 Jul 2021 16:39:11 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Noq1Ck1e/jKV8pTNEuYjMJra3diwE8Ub3gK2sNfQTkLU7/mLYvS//hHKZLx1CJnrz2OJP5XRQ/s7bfOHVHY7ivzoZWS6aWuxRPZ6vuUdBbwQsnJGMIWp4JCm2lDlWe2UWdvRUsJJAFpK7J0wr46ypUdz9kqGRx47bQW53R1hGbS7lsOTwWx98d1U9bGY6d1KQyEzMkSzdQsTkh709EZH6spRkl3gYVAq+nuAtjeBwKE4mTBgeC1ffYpMuThoG0RE3A/syApP7KeVipUsaxn2hkpPzsW8adQwZhJ8GkGx+MwMMRmKo53J7E13Y8yPbwWZ1YTQX97t5BV+M0BwKAzuGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=olypt5m+/StqdKRaOrkfO7iyB04yOODz7LMKw/+rqcE=; b=kAY91KQ1J59BjuIbR+pldYwXPN0Gy1QmRRYgLc7IrT15EaW/dBFzQgP/AET7XnaP3pjQFKW8MK/OerBGVclRy1FxOcaPBOMRMF4/96uiXwXdjMVjMtQ1Ld3j/+V8P3kHEY+OeuxGTf8oE7XeljRFGr7rdLUjGuiL1fK5LSC/DU/lF5Vo3tbTfSiENQrEgd3v7RNvM3EoDP9ZawqOEOuPMT7EeKnRhi1CI0xgDCoVnzk04FZBOslYBdZxx9egq0hK8zCvei3ZikHoeLVItvzKk//PhkViMAioyRsbPxDANhVF8cAcrVqAwsyszBjDoLwJ9hw9xeMljaVXNIM+EDGouA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=olypt5m+/StqdKRaOrkfO7iyB04yOODz7LMKw/+rqcE=; b=fdymKtXdPi3GXUXP9V1WgdcfrWKF8oeKJFpvbu6Mpk5RhdIM3pY+EWKUiXUNq3KymNz6QbxPXxlWjXK49DVfwZsTjB7ECPxzODx4yzvbjYtwJwsqjRFFPOtCLLexn5xOQrJpuw5FebsnYBpMCjQtA6h7vM6pulifO9KDCQK/8kT3xaPbrD+zEf8K0ayZ3sfCQsiKafURORqjVAt64ODPgxyVWLBIZw3JAqPnI2cDxI/BLSPao58y/iLdVN0ssh/qnbWhKv/r9wL/KZ3nF95QNVZ50/hFUXM7I6iL+PFF5ydxAnsnVE87yuapdJApLlfW3fkI076DZp+BTK+/oUREdA== Received: from BN6PR1101CA0002.namprd11.prod.outlook.com (2603:10b6:405:4a::12) by MN2PR12MB3696.namprd12.prod.outlook.com (2603:10b6:208:169::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.25; Wed, 21 Jul 2021 14:39:07 +0000 Received: from BN8NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:405:4a:cafe::86) by BN6PR1101CA0002.outlook.office365.com (2603:10b6:405:4a::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22 via Frontend Transport; Wed, 21 Jul 2021 14:39:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT035.mail.protection.outlook.com (10.13.177.116) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:07 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:04 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Anatoly Burakov Date: Wed, 21 Jul 2021 22:37:35 +0800 Message-ID: <20210721143743.24626-9-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 284c6ddc-1ec5-4ddf-fec4-08d94c554be5 X-MS-TrafficTypeDiagnostic: MN2PR12MB3696: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:199; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bHu91GwiHRhyGhpMsJeZtbdM4RcemEkAJyx0BpGGD+st6P2/5Lv8MRnZyFEZ/VRQNeEQLpiKOHXLfq0R+NKU8NkdK38vnIpE1paAaV9LE1HsHFgnWmkwtc6HXxarbINeUDq+GoOiIWmnvmqMy9h9yHfGU7XssJ7BLYyXSP5QRIkMEESo8/7L1Bdb0kD8P18CJgxDLXaRGlfJnHQgy2cZS9i/BdsQ65GUJcI8l7TYKBFlYuVO734xZl3NeXPPvFYs5RyCeAahEECEDg4WDs5ydk65y+6noXMtURD5pefhoYoTJmAabonuON4aaBvctVCEUTOXKqP3h566e/PNrr7tIJvfuleFfeiVZXaXs1nbO2snUhVzmAveaHqcjnHeIinh8T36IreIvi8/z+9vi67W8o/3pQbAmMyfLsCo91RaD/RaHW7WecR9ekdOMh3BhunYq/9P8vQjjv+8gEsmXnMCMjw43KvgCBkBSgzG/6/9LrjBscylnq9vJXPQfRFyz0MTHURb90vS4iTlTbpsCKPnJVIoeMqpiTGz/75lWBYsHAxKcV6BZqBOcfOKvbuZzMk1hwaKVjF96vBUTFnBjSmSIMBFwaR+XIsbLgkvpvUH7tvcB8rQRAe7gbbc7tLrH/5HtTqyhoosixNHhCiyPbeGq/yqB2qhKL8pHrRpkXzmHVQ5EItRa2TP49dECxT5na65izHTMbR/KBrxCkgbOlsXTA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(55016002)(6636002)(54906003)(19627235002)(508600001)(8936002)(8676002)(426003)(86362001)(36906005)(37006003)(316002)(2906002)(82310400003)(6666004)(70206006)(36860700001)(5660300002)(30864003)(7636003)(36756003)(26005)(83380400001)(336012)(16526019)(186003)(7696005)(1076003)(6286002)(6862004)(2616005)(4326008)(47076005)(70586007)(356005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:07.7342 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 284c6ddc-1ec5-4ddf-fec4-08d94c554be5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3696 Subject: [dpdk-dev] [PATCH v4 08/16] net/mlx5: support SubFunction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch introduces SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 54 +++++---- drivers/net/mlx5/linux/mlx5_ethdev_os.c | 12 +- drivers/net/mlx5/linux/mlx5_os.c | 145 +++++++++++++++++------- drivers/net/mlx5/linux/mlx5_os.h | 2 + drivers/net/mlx5/mlx5.c | 23 +++- drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_mac.c | 2 +- drivers/net/mlx5/mlx5_rxmode.c | 8 +- drivers/net/mlx5/mlx5_trigger.c | 2 +- drivers/net/mlx5/windows/mlx5_os.c | 12 +- 10 files changed, 188 insertions(+), 74 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index f5b727c1ee..a2ecddef13 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -112,6 +112,11 @@ Features - Flow integrity offload API. - Connection tracking. - Sub-Function representors. +- Sub-Function. + +Limitations +----------- + Limitations ----------- @@ -1478,40 +1483,51 @@ the DPDK application. echo switchdev > /sys/class/net//compat/devlink/mode -Sub-Function representor ------------------------- +Sub-Function support +-------------------- Sub-Function is a portion of the PCI device, a SF netdev has its own -dedicated queues(txq, rxq). A SF netdev supports E-Switch representation -offload similar to existing PF and VF representors. A SF shares PCI -level resources with other SFs and/or with its parent PCI function. +dedicated queues(txq, rxq). A SF shares PCI level resources with other SFs +and/or with its parent PCI function. + +0. Requirement:: + + OFED version >= 5.4-0.3.3.0 1. Configure SF feature:: - mlxconfig -d set PF_BAR2_SIZE=<0/1/2/3> PF_BAR2_ENABLE=1 + # Run mlxconfig on both PFs on host and ECPFs on BlueField. + mlxconfig -d set PER_PF_NUM_SF=1 PF_TOTAL_SF=252 PF_SF_BAR_SIZE=12 - Value of PF_BAR2_SIZE: +2. Enable switchdev mode:: - 0: 8 SFs - 1: 16 SFs - 2: 32 SFs - 3: 64 SFs + mlxdevm dev eswitch set pci/ mode switchdev -2. Reset the FW:: +3. Add SF port:: - mlxfwreset -d reset + mlxdevm port add pci/ flavour pcisf pfnum 0 sfnum -3. Enable switchdev mode:: + Get SFID from output: pci// - echo switchdev > /sys/class/net//compat/devlink/mode +4. Modify MAC address:: + + mlxdevm port function set pci// hw_addr + +5. Activate SF port:: + + mlxdevm port function set pci// state active + +6. Devargs to probe SF device:: -4. Create SF:: + auxiliary:mlx5_core.sf.,dv_flow_en=1 - mlnx-sf -d -a create +Sub-Function representor support +-------------------------------- -5. Probe SF representor:: +A SF netdev supports E-Switch representation offload similar to existing PF +and VF representors. Use to probe SF representor. - testpmd> port attach ,representor=sf0,dv_flow_en=1 + testpmd> port attach ,representor=sf,dv_flow_en=1 Performance tuning ------------------ diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c index b05b9fc950..f34133e2c6 100644 --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c @@ -128,6 +128,17 @@ struct ethtool_link_settings { #define ETHTOOL_LINK_MODE_200000baseCR4_Full_BIT 2 /* 66 - 64 */ #endif +/* Get interface index from SubFunction device name. */ +int +mlx5_auxiliary_get_ifindex(const char *sf_name) +{ + char if_name[IF_NAMESIZE] = { 0 }; + + if (mlx5_auxiliary_get_child_name(sf_name, "/net", + if_name, sizeof(if_name)) != 0) + return -rte_errno; + return if_nametoindex(if_name); +} /** * Get interface name from private structure. @@ -1619,4 +1630,3 @@ mlx5_get_mac(struct rte_eth_dev *dev, uint8_t (*mac)[RTE_ETHER_ADDR_LEN]) memcpy(mac, request.ifr_hwaddr.sa_data, RTE_ETHER_ADDR_LEN); return 0; } - diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 1b7ee419d1..f041f9054f 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -2061,6 +2062,27 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, return pf; } +static void +mlx5_os_config_default(struct mlx5_dev_config *config) +{ + memset(config, 0, sizeof(*config)); + config->mps = MLX5_ARG_UNSET; + config->dbnc = MLX5_ARG_UNSET; + config->rx_vec_en = 1; + config->txq_inline_max = MLX5_ARG_UNSET; + config->txq_inline_min = MLX5_ARG_UNSET; + config->txq_inline_mpw = MLX5_ARG_UNSET; + config->txqs_inline = MLX5_ARG_UNSET; + config->vf_nl_en = 1; + config->mr_ext_memseg_en = 1; + config->mprq.max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN; + config->mprq.min_rxqs_num = MLX5_MPRQ_MIN_RXQS; + config->dv_esw_en = 1; + config->dv_flow_en = 1; + config->decap_en = 1; + config->log_hp_size = MLX5_ARG_UNSET; +} + /** * Register a PCI device within bonding. * @@ -2485,23 +2507,8 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, uint32_t restore; /* Default configuration. */ - memset(&dev_config, 0, sizeof(struct mlx5_dev_config)); + mlx5_os_config_default(&dev_config); dev_config.vf = dev_config_vf; - dev_config.mps = MLX5_ARG_UNSET; - dev_config.dbnc = MLX5_ARG_UNSET; - dev_config.rx_vec_en = 1; - dev_config.txq_inline_max = MLX5_ARG_UNSET; - dev_config.txq_inline_min = MLX5_ARG_UNSET; - dev_config.txq_inline_mpw = MLX5_ARG_UNSET; - dev_config.txqs_inline = MLX5_ARG_UNSET; - dev_config.vf_nl_en = 1; - dev_config.mr_ext_memseg_en = 1; - dev_config.mprq.max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN; - dev_config.mprq.min_rxqs_num = MLX5_MPRQ_MIN_RXQS; - dev_config.dv_esw_en = 1; - dev_config.dv_flow_en = 1; - dev_config.decap_en = 1; - dev_config.log_hp_size = MLX5_ARG_UNSET; dev_config.allow_duplicate_pattern = 1; list[i].numa_node = pci_dev->device.numa_node; list[i].eth_dev = mlx5_dev_spawn(&pci_dev->device, @@ -2560,6 +2567,35 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, return ret; } +static int +mlx5_os_parse_eth_devargs(struct rte_device *dev, + struct rte_eth_devargs *eth_da) +{ + int ret = 0; + + if (dev->devargs == NULL) + return 0; + memset(eth_da, 0, sizeof(*eth_da)); + /* Parse representor information first from class argument. */ + if (dev->devargs->cls_str) + ret = rte_eth_devargs_parse(dev->devargs->cls_str, eth_da); + if (ret != 0) { + DRV_LOG(ERR, "failed to parse device arguments: %s", + dev->devargs->cls_str); + return -rte_errno; + } + if (eth_da->type == RTE_ETH_REPRESENTOR_NONE) { + /* Parse legacy device argument */ + ret = rte_eth_devargs_parse(dev->devargs->args, eth_da); + if (ret) { + DRV_LOG(ERR, "failed to parse device arguments: %s", + dev->devargs->args); + return -rte_errno; + } + } + return 0; +} + /** * Callback to register a PCI device. * @@ -2574,31 +2610,13 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, static int mlx5_os_pci_probe(struct rte_pci_device *pci_dev) { - struct rte_eth_devargs eth_da = { .type = RTE_ETH_REPRESENTOR_NONE }; + struct rte_eth_devargs eth_da = { .nb_ports = 0 }; int ret = 0; uint16_t p; - if (pci_dev->device.devargs) { - /* Parse representor information from device argument. */ - if (pci_dev->device.devargs->cls_str) - ret = rte_eth_devargs_parse - (pci_dev->device.devargs->cls_str, ð_da); - if (ret) { - DRV_LOG(ERR, "failed to parse device arguments: %s", - pci_dev->device.devargs->cls_str); - return -rte_errno; - } - if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) { - /* Support legacy device argument */ - ret = rte_eth_devargs_parse - (pci_dev->device.devargs->args, ð_da); - if (ret) { - DRV_LOG(ERR, "failed to parse device arguments: %s", - pci_dev->device.devargs->args); - return -rte_errno; - } - } - } + ret = mlx5_os_parse_eth_devargs(&pci_dev->device, ð_da); + if (ret != 0) + return ret; if (eth_da.nb_ports > 0) { /* Iterate all port if devargs pf is range: "pf[0-1]vf[...]". */ @@ -2611,10 +2629,56 @@ mlx5_os_pci_probe(struct rte_pci_device *pci_dev) return ret; } +/* Probe a single SF device on auxiliary bus, no representor support. */ +static int +mlx5_os_auxiliary_probe(struct rte_device *dev) +{ + struct rte_eth_devargs eth_da = { .nb_ports = 0 }; + struct mlx5_dev_config config; + struct mlx5_dev_spawn_data spawn = { .pf_bond = -1 }; + struct rte_auxiliary_device *adev = RTE_DEV_TO_AUXILIARY(dev); + struct rte_eth_dev *eth_dev; + int ret = 0; + + /* Parse ethdev devargs. */ + ret = mlx5_os_parse_eth_devargs(dev, ð_da); + if (ret != 0) + return ret; + /* Set default config data. */ + mlx5_os_config_default(&config); + config.sf = 1; + /* Init spawn data. */ + spawn.max_port = 1; + spawn.phys_port = 1; + spawn.phys_dev = mlx5_os_get_ibv_dev(dev); + if (spawn.phys_dev == NULL) + return -rte_errno; + ret = mlx5_auxiliary_get_ifindex(dev->name); + if (ret < 0) { + DRV_LOG(ERR, "failed to get ethdev ifindex: %s", dev->name); + return ret; + } + spawn.ifindex = ret; + spawn.numa_node = dev->numa_node; + /* Spawn device. */ + eth_dev = mlx5_dev_spawn(dev, &spawn, &config, ð_da); + if (eth_dev == NULL) + return -rte_errno; + /* Post create. */ + eth_dev->intr_handle = &adev->intr_handle; + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC; + eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_RMV; + eth_dev->data->numa_node = dev->numa_node; + } + rte_eth_dev_probing_finish(eth_dev); + return 0; +} + /** * Net class driver callback to probe a device. * - * This function probe PCI bus device(s). + * This function probe PCI bus device(s) or a single SF on auxiliary bus. * * @param[in] dev * Pointer to the generic device. @@ -2637,7 +2701,8 @@ mlx5_os_net_probe(struct rte_device *dev) } if (mlx5_dev_is_pci(dev)) return mlx5_os_pci_probe(RTE_DEV_TO_PCI(dev)); - return 0; + else + return mlx5_os_auxiliary_probe(dev); } static int diff --git a/drivers/net/mlx5/linux/mlx5_os.h b/drivers/net/mlx5/linux/mlx5_os.h index af7cbeb418..2991d37df2 100644 --- a/drivers/net/mlx5/linux/mlx5_os.h +++ b/drivers/net/mlx5/linux/mlx5_os.h @@ -19,4 +19,6 @@ enum { #define MLX5_NAMESIZE IF_NAMESIZE +int mlx5_auxiliary_get_ifindex(const char *sf_name); + #endif /* RTE_PMD_MLX5_OS_H_ */ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 96e8d189ba..818e37fd48 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -399,6 +399,24 @@ mlx5_is_hpf(struct rte_eth_dev *dev) MLX5_REPRESENTOR_REPR(-1) == repr; } +/** + * Decide whether representor ID is a SF port representor. + * + * @param dev + * Pointer to Ethernet device structure. + * + * @return + * Non-zero if HPF, otherwise 0. + */ +bool +mlx5_is_sf_repr(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + int type = MLX5_REPRESENTOR_TYPE(priv->representor_id); + + return priv->representor != 0 && type == RTE_ETH_REPRESENTOR_SF; +} + /** * Initialize the ASO aging management structure. * @@ -2335,7 +2353,10 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) (dev->device == odev || (dev->device->driver && dev->device->driver->name && - !strcmp(dev->device->driver->name, MLX5_PCI_DRIVER_NAME)))) + ((strcmp(dev->device->driver->name, + MLX5_PCI_DRIVER_NAME) == 0) || + (strcmp(dev->device->driver->name, + MLX5_AUXILIARY_DRIVER_NAME) == 0))))) break; port_id++; } diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index d88b1433fb..c3736e3d56 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -243,6 +243,7 @@ struct mlx5_dev_config { unsigned int hw_fcs_strip:1; /* FCS stripping is supported. */ unsigned int hw_padding:1; /* End alignment padding is supported. */ unsigned int vf:1; /* This is a VF. */ + unsigned int sf:1; /* This is a SF. */ unsigned int tunnel_en:1; /* Whether tunnel stateless offloads are supported. */ unsigned int mpls_en:1; /* MPLS over GRE/UDP is enabled. */ @@ -1466,6 +1467,7 @@ int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev, uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev); int mlx5_dev_close(struct rte_eth_dev *dev); bool mlx5_is_hpf(struct rte_eth_dev *dev); +bool mlx5_is_sf_repr(struct rte_eth_dev *dev); void mlx5_age_event_prepare(struct mlx5_dev_ctx_shared *sh); /* Macro to iterate over all valid ports for mlx5 driver. */ diff --git a/drivers/net/mlx5/mlx5_mac.c b/drivers/net/mlx5/mlx5_mac.c index 19981d26d8..a791fedc91 100644 --- a/drivers/net/mlx5/mlx5_mac.c +++ b/drivers/net/mlx5/mlx5_mac.c @@ -159,7 +159,7 @@ mlx5_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr) * Configuring the VF instead of its representor, * need to skip the special case of HPF on Bluefield. */ - if (priv->representor && !mlx5_is_hpf(dev)) { + if (priv->representor && !mlx5_is_hpf(dev) && !mlx5_is_sf_repr(dev)) { DRV_LOG(DEBUG, "VF represented by port %u setting primary MAC address", dev->data->port_id); if (priv->pf_bond >= 0) { diff --git a/drivers/net/mlx5/mlx5_rxmode.c b/drivers/net/mlx5/mlx5_rxmode.c index 25fb47c9ed..7f19b235c2 100644 --- a/drivers/net/mlx5/mlx5_rxmode.c +++ b/drivers/net/mlx5/mlx5_rxmode.c @@ -36,7 +36,7 @@ mlx5_promiscuous_enable(struct rte_eth_dev *dev) dev->data->port_id); return 0; } - if (priv->config.vf) { + if (priv->config.vf || priv->config.sf) { ret = mlx5_os_set_promisc(dev, 1); if (ret) return ret; @@ -69,7 +69,7 @@ mlx5_promiscuous_disable(struct rte_eth_dev *dev) int ret; dev->data->promiscuous = 0; - if (priv->config.vf) { + if (priv->config.vf || priv->config.sf) { ret = mlx5_os_set_promisc(dev, 0); if (ret) return ret; @@ -109,7 +109,7 @@ mlx5_allmulticast_enable(struct rte_eth_dev *dev) dev->data->port_id); return 0; } - if (priv->config.vf) { + if (priv->config.vf || priv->config.sf) { ret = mlx5_os_set_allmulti(dev, 1); if (ret) goto error; @@ -142,7 +142,7 @@ mlx5_allmulticast_disable(struct rte_eth_dev *dev) int ret; dev->data->all_multicast = 0; - if (priv->config.vf) { + if (priv->config.vf || priv->config.sf) { ret = mlx5_os_set_allmulti(dev, 0); if (ret) goto error; diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index 6d2351f5a8..a9d5d58fd9 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -1259,7 +1259,7 @@ mlx5_traffic_enable(struct rte_eth_dev *dev) } mlx5_txq_release(dev, i); } - if (priv->config.dv_esw_en && !priv->config.vf) { + if (priv->config.dv_esw_en && !priv->config.vf && !priv->config.sf) { if (mlx5_flow_create_esw_table_zero_flow(dev)) priv->fdb_def_rule = 1; else diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index bf20adaa30..ee09acc96b 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -922,20 +922,18 @@ mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf, /** * DPDK callback to register a PCI device. * - * This function spawns Ethernet devices out of a given PCI device. + * This function spawns Ethernet devices out of a given device. * - * @param[in] pci_drv - * PCI driver structure (mlx5_driver). - * @param[in] pci_dev - * PCI device information. + * @param[in] dev + * Pointer to the generic device. * * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) +mlx5_os_net_probe(struct rte_device *dev) { + struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev); struct devx_device_bdf *devx_bdf_devs, *orig_devx_bdf_devs; /* * Number of found IB Devices matching with requested PCI BDF. From patchwork Wed Jul 21 14:37:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96164 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA4B3A0C51; Wed, 21 Jul 2021 16:39:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4BD4241109; Wed, 21 Jul 2021 16:39:12 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2045.outbound.protection.outlook.com [40.107.244.45]) by mails.dpdk.org (Postfix) with ESMTP id 051B341124 for ; Wed, 21 Jul 2021 16:39:11 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PRl+1VIDz87dXCLCMZUOAIpa5K2Eps5k5N59AqduNgyYeWxnWoeTLLNOOV5/8VpmIX2Gwt5y4b6+uz99IffEY848WYPV38uL7c7aoX+Efuxy54N7+YM6iUCI65iU6EYnc+6CEqtvIxplgdUziLlTE9RBxduIc3SteGdTIijH0H7PnO+55cvEa2fRLneD3QrQhvxFwORUw45eFsPCabM9geeD5Fj/0ntvqQFwoDLYqL/uZLjGc8pFumf5dVms7xEBoK5js9sKQwC4rltCGnYRSCMXmAE4oJbectyjkrwExw8FmINCrx706MuR4UgDB++zb2BUup4lZhUNVhutKrp6Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xLWFttL1YYSCjPgy2gRDI/qscTd03+dU2tU5A33ZAXw=; b=j3zIPYPtl5KjZvebdMEDh13ayVaXfkp13zMIWL5a46ezl7D08cw+4DWG/9stl+EhWP1NO/hEabYovtBwRSm/gOOwTxlx6KWbI/SW80wNS3h8NXCqGYcGKXlV+tEOggmBMuwVPZ4RN+XaBdHW1JOnd30Lhv4rSH07ABi6DTG4f7qOpsAUMLjRo3huNBDtXMzfLVv7/MHR288nYiuD9+FDYjCw5WX94bAsanK7J9p3z/G0TXOHpw8wqQMwzLowl6dGWg66TUo4c/fphJwYg/78F/cvjXqkhA0xQZhO4V3vLk9KeBfsQHr3059RPOKeNF5vHRZdQUczXz3tge4aXfsf1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xLWFttL1YYSCjPgy2gRDI/qscTd03+dU2tU5A33ZAXw=; b=sFglfmXMHJQXc7IX6h/bnFCc52HoKvy+EYHmnceXFNN2cAx/gDzLHle9O4apStDM9JqQJPpUMEzR8bv6DE12Tr9tkPgWRMJE4pAPuM+fQcxt8XQ/2ArxXewrlcVlCP4r3D6Hf8Pn+o8TE4HSYmZYJaf1TbLZ45p1372BWcuPpMYrINUsrHAE4lvyuZdqiudoNFSoiJESkGNph9SyoXhS5KnYSVxB6xt1B42SeoDTwH5A+kHrrCB0Ed2Fbmo1oCXl4mexccG01EL1PFpw6SRVnqDhLAqi1ZL+Tu60E5ColaK/9zhUIT2AsJ+SmecVmbDjlKVECW7w9Tck/fr+G39sOg== Received: from BN6PR1101CA0014.namprd11.prod.outlook.com (2603:10b6:405:4a::24) by BN8PR12MB3474.namprd12.prod.outlook.com (2603:10b6:408:46::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.29; Wed, 21 Jul 2021 14:39:09 +0000 Received: from BN8NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:405:4a:cafe::6) by BN6PR1101CA0014.outlook.office365.com (2603:10b6:405:4a::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT035.mail.protection.outlook.com (10.13.177.116) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:09 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:06 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler Date: Wed, 21 Jul 2021 22:37:36 +0800 Message-ID: <20210721143743.24626-10-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2bd001fc-c88c-4b16-53bd-08d94c554cfa X-MS-TrafficTypeDiagnostic: BN8PR12MB3474: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:346; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ag1+iEGXq5csNij6RsZp9YICpgkap7SZ0vsy86wm10snqaMgJ6ZHnzEbh2KlNZoLMMzSo89nWU6oLavW/fTGpvxlInpE7YtjiGDZGj/Z52wO3VM3dA8yAdfsYUWkBX9aTuSiN+hApck8H8mWikzu+on6u75e5BDOFczLh05hj8s7CAxxdccvhlIfpQC/zJibzcJwU0k9c7CANA1yRJKJa04/UrAlod2KEaWY9zLqFOCpcSY4N3ZbjgBmXUnA+lPoUZ0r8orc+cacljIucMBGPQSUWJTlrnO+T/TopEhzXaPLbCuUmmAwSDAXJSlIDvcKa6b3pfNsJTvr4+41psyn/g2mMyPqNApQ2ZzfyOlAfPE0ET63F4h1yRlbryx3iB8nhNHG4UI3WwA2nIjZTo5tqUKrZhiH3YcVfiwc16ZOV9Rg4vYHSAOh6LOpw0iJrs5lp89IPQenckBTg9d9ONv1X2WAexBA4ZkTvxKTq8P91HtLONNMkij3YZ5esbGoAM1LDfQiy9S/RU7RgFYYZqMNxk+Kg3QXj8LCCJKx3p6wSHvmgmdyFqoYTOL9obUAcqmQrCY5WCGSwTRGOvuGM6CO/Ky847UMTu6QFmy4sb3xCkeC+qFESB+PAmRfCsjd0/mX4YKTuXXYcnsaB+dF5sYO5F0zRHcJOyBroLrVo4q/GSrRNvRp0GjAwPQeCMjzajQ0b4fbiIMibngeyukgf2E5uQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(346002)(396003)(136003)(39860400002)(46966006)(36840700001)(426003)(336012)(55016002)(2906002)(4744005)(6286002)(2616005)(70586007)(6636002)(6862004)(16526019)(36756003)(5660300002)(47076005)(70206006)(1076003)(36906005)(4326008)(316002)(356005)(478600001)(82740400003)(36860700001)(6666004)(54906003)(37006003)(8936002)(7696005)(82310400003)(107886003)(86362001)(26005)(186003)(8676002)(7636003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:09.5571 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2bd001fc-c88c-4b16-53bd-08d94c554cfa X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB3474 Subject: [dpdk-dev] [PATCH v4 09/16] net/mlx5: check max Verbs port number X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Verbs API doesn't support Device port number larger than 255 by design. Adds check and fails probing with proper error log. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index f041f9054f..db44169b84 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1294,6 +1294,12 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, config->dv_flow_en = 0; } #endif + if (spawn->max_port > UINT8_MAX) { + /* Verbs can't support ports larger than 255 by design. */ + DRV_LOG(ERR, "can't support IB ports > UINT8_MAX"); + err = EINVAL; + goto error; + } config->ind_table_max_size = sh->device_attr.max_rwq_indirection_table_size; /* From patchwork Wed Jul 21 14:37:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96166 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C0E9CA0C51; Wed, 21 Jul 2021 16:39:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C756C410E0; Wed, 21 Jul 2021 16:39:30 +0200 (CEST) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2088.outbound.protection.outlook.com [40.107.102.88]) by mails.dpdk.org (Postfix) with ESMTP id 4009A410DA for ; Wed, 21 Jul 2021 16:39:29 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KsuACilu7SBvjev/OuXRFlIAbOHYHVr1BEJElBMpf6lSye3A1MN0UiKxk0fNhxDHJ/Tc2Gl0mp+3gD/+kmZs+AtFlN9D0IW97Kjpsi8YmKgRcrO3Du/g5tZYw/Os9g9SvnzWi91pJZ+3IoUya/a9dldjeZiHF/fGg8+U+lt98EdjZDIQxK9h+rO76XnTSFyZ+rCdFxqXnKZCw1D/tdB6rUAlCwPN/MK69zX8/wafT0CdlDyvmiRQmWGQ5IaMVS4BHvzUm+1D7Yl8WkwkCVYs7DaHp9ycY+geVRLdmUebVFtFdWRjD/ikwurFbGNBg7Ns5HXXgIKwMasmT0jNEj8inA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rl2uYNU3PJwFDLMEVwCQ501cJ41fMuc9/av32MXb7/Y=; b=HDWzLcslHsr9+uv9PZ+uwV6IjkZgPdIPJuHl4bRJMqPoFzhQULSSTckcz07SgZuwWTSCCd/McQXYMOZu+jF7os1EvslQw8ASgR+QEvX1fpCURvGjwfRTopWUBnhaGARJ2We1X+dXluAoj4ZBk/5/WNbbou3wUwlch1zryIyES4xTwkyp7XD1eJf6XrDqzVPtlktCCGWkwZ0SHblQv7RQfzg5kVGCSqubwrli4oCJDH546vxOzRVLHwT9m2OTg8broUb9J9uruXgNGdef1QArBiMa3L8xTkCSci1cOtPo/yKM2HbL7lfT4X5FcXeal7CpN/TvcFSlFgqpnkS57918wA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rl2uYNU3PJwFDLMEVwCQ501cJ41fMuc9/av32MXb7/Y=; b=SR+Ba6mbisonfa38+oPgqnsR7R8Ju/xhdtWIiYc3j4ULfBiVEyqLi6puFweUGxXYbh4pw26LlDVKeyDc+FImyPSURgXf7bgeA7xtphcHHtQh4h0OR0QsyUyrTSxcd0Ye6jYZKjzt2m2kXXGYAET0AhH2JTyWqyrLLwp4RyrlUDEK7W5bywTCeMAv+XovQDu8QrgCxvFGPTr8QIKQOTou1nfeLz4k/QSRIJSuRTdhmUYIK258Uo+KYOkoibZtoMYJkftS2e6bO82bpdeFs6ve5rj/rxQN7Ur9sOJjC5OKBYXdWTTUZyQKhmqtnAr8gGGuMooL8ejqE4ohygeTsT1IUw== Received: from BN0PR03CA0013.namprd03.prod.outlook.com (2603:10b6:408:e6::18) by DM8PR12MB5493.namprd12.prod.outlook.com (2603:10b6:8:3d::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.23; Wed, 21 Jul 2021 14:39:28 +0000 Received: from BN8NAM11FT021.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e6:cafe::9f) by BN0PR03CA0013.outlook.office365.com (2603:10b6:408:e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22 via Frontend Transport; Wed, 21 Jul 2021 14:39:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT021.mail.protection.outlook.com (10.13.177.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:27 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:25 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Ori Kam Date: Wed, 21 Jul 2021 22:37:37 +0800 Message-ID: <20210721143743.24626-11-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3282342d-1d2e-42f3-80b3-08d94c5557ea X-MS-TrafficTypeDiagnostic: DM8PR12MB5493: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2582; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rcJ3GANTB17yI4OMVePEoGv/jnBKytEZyUr2wRZiy0vqcilXh6Xd3r8PHnX593jiSnbvbUfM9Rxw/WhSPCqSwMhHel6iOpDPX/j6o4pTNeN5NxyteiR+zK9nMV5qckVE1IArmA3zLNtEj8jpjED8E1cd2rkCIexjsush15UOfTLYr8WoSMRShTo2Do8dEB1V1/oWqTzTPMX7ASdSBIEU/5cbu3Jm8vY+rxzy5X8E7we9S7RI83nciFzqOjmBgib97KbImJQr5CjQ2iDBkFp5ZcYByJmZ2UhEPHkyjrHbz0B7NPIb/7nyMItbWoDNF4qLeMrFYgNhHq925nd/jV/jPX79v4Qbg942HuRsAybppgUWRh6mPtOBC7/75XlZ4NrD0zsK/JgSJIILF3TZt98MOH2TZhAsPImraVDzDsqinYopv3Vmpjht7xN7mwWQfwOAC17w53RMIqBUF2Iyto5a1BkBc0zmF2Qtq0Z8Mai3IvfEcc7cyT69dVN7+l2UnUNnSikCxfTK37fXNDIKEBrI1rgFyMAfeHW7hl451QSDCqaRhbid8m7P0PRGdmPeA+LCzVXUI0LPpayGK25lVsX2QSXpY9X/bewZ40mgNDrAJlvx8jAr84xtpVCrJtzyjlXS7qMKGJxGpO69vw+pMLsoA7c3agsIQy8y0S9zYjSnidnQZXqmERDWelE9xpL5In6pDMrl7kn3ANSkJrT69FtyyA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(346002)(396003)(39860400002)(376002)(46966006)(36840700001)(1076003)(478600001)(6636002)(82310400003)(336012)(4326008)(7696005)(36756003)(2616005)(70206006)(8676002)(8936002)(316002)(36906005)(37006003)(26005)(86362001)(107886003)(82740400003)(36860700001)(6286002)(54906003)(83380400001)(47076005)(6862004)(55016002)(2906002)(70586007)(16526019)(7636003)(356005)(426003)(5660300002)(6666004)(186003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:27.8904 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3282342d-1d2e-42f3-80b3-08d94c5557ea X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT021.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5493 Subject: [dpdk-dev] [PATCH v4 10/16] regex/mlx5: migrate to common driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support auxiliary bus, upgrades driver to use mlx5 common driver structure. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/regex/mlx5/mlx5_regex.c | 49 ++++++++++++--------------------- drivers/regex/mlx5/mlx5_regex.h | 1 - 2 files changed, 18 insertions(+), 32 deletions(-) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index dcb2ced88e..9d93eaa934 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -9,8 +9,8 @@ #include #include #include +#include -#include #include #include #include @@ -76,15 +76,13 @@ mlx5_regex_engines_status(struct ibv_context *ctx, int num_engines) } static void -mlx5_regex_get_name(char *name, struct rte_pci_device *pci_dev __rte_unused) +mlx5_regex_get_name(char *name, struct rte_device *dev) { - sprintf(name, "mlx5_regex_%02x:%02x.%02x", pci_dev->addr.bus, - pci_dev->addr.devid, pci_dev->addr.function); + sprintf(name, "mlx5_regex_%s", dev->name); } static int -mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) +mlx5_regex_dev_probe(struct rte_device *rte_dev) { struct ibv_device *ibv; struct mlx5_regex_priv *priv = NULL; @@ -94,16 +92,10 @@ mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, int ret; uint32_t val; - ibv = mlx5_os_get_ibv_device(&pci_dev->addr); - if (!ibv) { - DRV_LOG(ERR, "No matching IB device for PCI slot " - PCI_PRI_FMT ".", pci_dev->addr.domain, - pci_dev->addr.bus, pci_dev->addr.devid, - pci_dev->addr.function); + ibv = mlx5_os_get_ibv_dev(rte_dev); + if (ibv == NULL) return -rte_errno; - } - DRV_LOG(INFO, "PCI information matches for device \"%s\".", - ibv->name); + DRV_LOG(INFO, "Probe device \"%s\".", ibv->name); ctx = mlx5_glue->dv_open_device(ibv); if (!ctx) { DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); @@ -146,7 +138,7 @@ mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, priv->is_bf2 = 1; /* Default RXP programming mode to Shared. */ priv->prog_mode = MLX5_RXP_SHARED_PROG_MODE; - mlx5_regex_get_name(name, pci_dev); + mlx5_regex_get_name(name, rte_dev); priv->regexdev = rte_regexdev_register(name); if (priv->regexdev == NULL) { DRV_LOG(ERR, "Failed to register RegEx device."); @@ -180,7 +172,7 @@ mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, priv->regexdev->enqueue = mlx5_regexdev_enqueue_gga; #endif priv->regexdev->dequeue = mlx5_regexdev_dequeue; - priv->regexdev->device = (struct rte_device *)pci_dev; + priv->regexdev->device = rte_dev; priv->regexdev->data->dev_private = priv; priv->regexdev->state = RTE_REGEXDEV_READY; priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr; @@ -213,13 +205,13 @@ mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, } static int -mlx5_regex_pci_remove(struct rte_pci_device *pci_dev) +mlx5_regex_dev_remove(struct rte_device *rte_dev) { char name[RTE_REGEXDEV_NAME_MAX_LEN]; struct rte_regexdev *dev; struct mlx5_regex_priv *priv = NULL; - mlx5_regex_get_name(name, pci_dev); + mlx5_regex_get_name(name, rte_dev); dev = rte_regexdev_get_device_by_name(name); if (!dev) return 0; @@ -254,24 +246,19 @@ static const struct rte_pci_id mlx5_regex_pci_id_map[] = { } }; -static struct mlx5_pci_driver mlx5_regex_driver = { - .driver_class = MLX5_CLASS_REGEX, - .pci_driver = { - .driver = { - .name = RTE_STR(MLX5_REGEX_DRIVER_NAME), - }, - .id_table = mlx5_regex_pci_id_map, - .probe = mlx5_regex_pci_probe, - .remove = mlx5_regex_pci_remove, - .drv_flags = 0, - }, +static struct mlx5_class_driver mlx5_regex_driver = { + .drv_class = MLX5_CLASS_REGEX, + .name = RTE_STR(MLX5_REGEX_DRIVER_NAME), + .id_table = mlx5_regex_pci_id_map, + .probe = mlx5_regex_dev_probe, + .remove = mlx5_regex_dev_remove, }; RTE_INIT(rte_mlx5_regex_init) { mlx5_common_init(); if (mlx5_glue) - mlx5_pci_driver_register(&mlx5_regex_driver); + mlx5_class_driver_register(&mlx5_regex_driver); } RTE_LOG_REGISTER_DEFAULT(mlx5_regex_logtype, NOTICE) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 51a2101e53..45200bf937 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -59,7 +59,6 @@ struct mlx5_regex_db { struct mlx5_regex_priv { TAILQ_ENTRY(mlx5_regex_priv) next; struct ibv_context *ctx; /* Device context. */ - struct rte_pci_device *pci_dev; struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ uint16_t nb_queues; /* Number of queues. */ struct mlx5_regex_qp *qps; /* Pointer to the qp array. */ From patchwork Wed Jul 21 14:37:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96167 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 26268A0C51; Wed, 21 Jul 2021 16:39:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EDD04410F8; Wed, 21 Jul 2021 16:39:33 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2087.outbound.protection.outlook.com [40.107.94.87]) by mails.dpdk.org (Postfix) with ESMTP id D622E410F8 for ; Wed, 21 Jul 2021 16:39:31 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BJvdmjQnikhClmC4teHz8xrzfeXOGJBi/p4ubaiieoHiTEY8XVUq0nOanFGkPdFPGRa0n0RJApxH37eibOyB6Grqm5t2hYR6P0Jnjgq9A0NG6LQlAF+xLGGF5+YWsxt9oUimo86pOUQJHSfkMe769RLrvII3M0s5s43UGvJ31jfOZu8DOAPv+GTu+A9TlxcWMoXVZa66sYrKZFtso7g36KR2ymiEOYtXIMMAy2apj5DN61nwWr0q5MiqUFwCZie1JZtBoN85Y6HvmJMJo1b0hUIz4fNPv980TGbkrQuwexOnou65el2rBRal80nTDiHFWi/rb9KDxkx7WCJcoBcHug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CxgrXPyVjQHoqRXbRavUHSv/qxhUJvtrlW0oAgcoKEk=; b=YhyUjJMEm12xxar/0oEe2tVHlZfYKxGgaP5HE8dsocj81tNfT4ZLJP4kk4by/+lPL4G9tvwA4zujKL4lQOwlROC6Nl8FCOzC8cawHnxzYPJMLt16x0Ch6VnlmYyDrA1NXOYCuC5xP0TtTjIuzYaOrqFIPSwYKaY+Igm1b4q+ZXVy81nM/VNvQnWgSw4Vy6dfpVH+0ETdPGaAw6S4aWVB35xZj95K4ba+SILoIknCbrCNYfgbbOixJlw56a76PDItHKSERIIhycQjEuC7oiCNQyrVl/gG5lje3th0jgVw+2311blfvhhclcrU6DVN3bfEJ1rMMXWYXlh1qiH0jzWugA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CxgrXPyVjQHoqRXbRavUHSv/qxhUJvtrlW0oAgcoKEk=; b=b4lsYQKFdpuixXb8FKmJa2vJbl2JK1i8TmmdRVLFLvfLvUlaOb8ZEPKNuHGZ6lsZdDTBBICq9s/Qkz9XYLVk3ixdHz+PHKTEpFJdpAP45v0jMOv9+vjkQ1oEq2Be67bojhXPcHl0qIpqnq/UMYIY67wslIuCNrJ7YEePTS9IB2AcUzOLKYqAV7kCzr0zyWXBPNh/KRSJNd25yot14VT7XNsSqGHBRU/wsz4RIoUTsyCUj1DsDvdNS2ZqtvPp6qxr9NLA4z+spUVNaaEBuUNrj5dQP/w0/5EkF67EWF1fekyV/z53CML++JCj6DfZd0PMdX/f2thpPE/6SwdOs0KEKQ== Received: from BN9PR03CA0237.namprd03.prod.outlook.com (2603:10b6:408:f8::32) by DM6PR12MB3019.namprd12.prod.outlook.com (2603:10b6:5:3d::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.25; Wed, 21 Jul 2021 14:39:30 +0000 Received: from BN8NAM11FT060.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f8:cafe::93) by BN9PR03CA0237.outlook.office365.com (2603:10b6:408:f8::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Wed, 21 Jul 2021 14:39:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT060.mail.protection.outlook.com (10.13.177.211) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:30 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:27 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad Date: Wed, 21 Jul 2021 22:37:38 +0800 Message-ID: <20210721143743.24626-12-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2969fe0f-cdca-4f02-743e-08d94c55595b X-MS-TrafficTypeDiagnostic: DM6PR12MB3019: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1079; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: G383KU60L4kn8BkNspvMV5/Pxs75+DG3xFTMwWWgqbsh9ojAPRx1QNjId1pfiN95Pqq0YLVnn1G9PA2KM2wpaiUj866cgkgBgO18WZ8hYu5Cja5tLOEXg+plgbKsPBydKmbkb9dP4kZCUpyGCPCiXbHbR3XWEpUW+WmwQ2DNtkwtn0I77RgSDb9CI5ma0hL37Yo8uP7zOh74/ICDxkHUEhKXWokb5jcPzIXqzKy1noGCYuKAvNi5jm3YApDQ9OegBBdzboOtFJcIkF7HVfrM3Q3BEHSdXMAQZjwyqqVsfPss/06uSOXVZFuYxpbk4XOc3lrDXWrsOMespBou/vAx+EIBO90lhvPB5QVQAtJLunYAAPQOLg7cXY7pQnLZ5aUJhrMrjXV+EsfWGPd3fPWH3t6M17B132/FT6qlxUjDjhuMbYsmWQgqqdkwJOSfNzOdq1v4+oHSXpDiIfNHP5S1UMPJKl0r+XCruI3jX91OVv2SHUgxYhqpPTwIJ8dFw9Km1LhcW9qlqWPegaT8PjpU1itcIguNsYzLT2gSLoSIDmo+IULecRVkoQJuht18O5vn0qRm3v23VyRVPKukuaSDHkCcnRbskUizvk0uegCtH6q3APQQVgX6oeq3PhJ4/tIcUtli/64CsNU5DS2lknfdjSfZOhq5KVsn+uXuXwrHyUN1u4+iH5RN1qkUrTo0mjSDVH73DZOw7cGaK20pqdBpcw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(39860400002)(46966006)(36840700001)(36860700001)(186003)(336012)(16526019)(36756003)(82310400003)(7636003)(6862004)(7696005)(2616005)(26005)(6636002)(8676002)(107886003)(4326008)(8936002)(83380400001)(478600001)(55016002)(356005)(47076005)(6666004)(82740400003)(1076003)(2906002)(86362001)(426003)(316002)(70586007)(70206006)(5660300002)(37006003)(6286002)(36906005)(54906003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:30.3133 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2969fe0f-cdca-4f02-743e-08d94c55595b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT060.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3019 Subject: [dpdk-dev] [PATCH v4 11/16] vdpa/mlx5: define driver name as macro X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon Uses macro for pmd driver name. Signed-off-by: Thomas Monjalon Acked-by: Viacheslav Ovsiienko --- drivers/vdpa/mlx5/mlx5_vdpa.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 8b5bfd8c3d..5ab7c525c2 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -24,6 +24,7 @@ #include "mlx5_vdpa_utils.h" #include "mlx5_vdpa.h" +#define MLX5_VDPA_DRIVER_NAME vdpa_mlx5 #define MLX5_VDPA_DEFAULT_FEATURES ((1ULL << VHOST_USER_F_PROTOCOL_FEATURES) | \ (1ULL << VIRTIO_F_ANY_LAYOUT) | \ @@ -834,7 +835,7 @@ static struct mlx5_pci_driver mlx5_vdpa_driver = { .driver_class = MLX5_CLASS_VDPA, .pci_driver = { .driver = { - .name = "mlx5_vdpa", + .name = RTE_STR(MLX5_VDPA_DRIVER_NAME), }, .id_table = mlx5_vdpa_pci_id_map, .probe = mlx5_vdpa_pci_probe, @@ -855,6 +856,6 @@ RTE_INIT(rte_mlx5_vdpa_init) mlx5_pci_driver_register(&mlx5_vdpa_driver); } -RTE_PMD_EXPORT_NAME(net_mlx5_vdpa, __COUNTER__); -RTE_PMD_REGISTER_PCI_TABLE(net_mlx5_vdpa, mlx5_vdpa_pci_id_map); -RTE_PMD_REGISTER_KMOD_DEP(net_mlx5_vdpa, "* ib_uverbs & mlx5_core & mlx5_ib"); +RTE_PMD_EXPORT_NAME(MLX5_VDPA_DRIVER_NAME, __COUNTER__); +RTE_PMD_REGISTER_PCI_TABLE(MLX5_VDPA_DRIVER_NAME, mlx5_vdpa_pci_id_map); +RTE_PMD_REGISTER_KMOD_DEP(MLX5_VDPA_DRIVER_NAME, "* ib_uverbs & mlx5_core & mlx5_ib"); From patchwork Wed Jul 21 14:37:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96168 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 647FFA0C52; Wed, 21 Jul 2021 16:39:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0CA474111A; Wed, 21 Jul 2021 16:39:35 +0200 (CEST) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam07on2059.outbound.protection.outlook.com [40.107.95.59]) by mails.dpdk.org (Postfix) with ESMTP id 6575E410DA for ; Wed, 21 Jul 2021 16:39:33 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kSVmWNbiHcjOANiheD7ZC4RhgDr5p+GQQ2OT2SUjm1nYQAbtOY/U23wO+5zLOI3lCOpGZ7QKxhjfdHuk8G0404FWG+gybr1a6A9uAz5U36huD+Gy/eqYZLlhie4nCz1ikm5Rrhk9QSCoa4IimOWsk/T1Y8igvu9BEJP4CIXb54ANxq5H8xV0QXmtacfrG4IBKyGr6OZ92S61RbBpLxmB9pH6ieTbpQp/vaUZk0xAGIf+QuCG6IwSGYGr8iOypDnRXvKO/Sog3KhrZuUv8rL7C0I+WhGxSDdDVITOAPDeYK7xB8jBE6s5aczbs6GAcKkFpgWt+m/5WUS6mXfMDqvLxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=89xJK82I5/X4RoVJwR6cwQNxjWBCwc+cFd+vzdRWyzg=; b=Uz6nRIK1CFZpotXjN4U9lF1Kd72yCHoCQ5p3vsrdfWKhLDspnNJy7PrzzbNtwO885bSZa9bsGRCHhkFptvtwiCeoxiTqaUo4FG9qR6Ii8igVN0ovCGjSAdZFqNqj7r4axi5k3vLrg3XzJCQuk4Q3VCM2fExR3hSu410vNJdY0cnf/n4lkOKMjHHeWz6xhVW7GxgnnjYWbC49m1A9VyzYIGqkn4mATfGYJuxi8lsCsfB0N9wSx1aADO8O3HGt6pIFMoEi4bii5mmSxkucla5wgkATdll3UDgPSlz/tbh/nfA/nEIjQ57qsF/M14QLyZVqSULyyaVjnph4SAz9G1wPSg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=89xJK82I5/X4RoVJwR6cwQNxjWBCwc+cFd+vzdRWyzg=; b=O6JaIXaY5apP8xmwhjV3STC0iIRunRWyYJTI2P52chzLTttWxOZJMOMpRsqqPglaBLnQeADngcRKp9oXrIhP+vwUwskDh/BhcqPHPIQp5qePTZ/zN7RJD33cGZefL4N7KddkVvAA6W8fBOALhDfeV9z7zZmF+AjxGErdgtL2CCudPHkuru5tUyUOnBqjf3uDLj/AnDiETngUCXnAXkWqz2n+Y6wIdX4yfvLJ0NoQu9QJMeMrp1af1+toRg1UQTKEKMUoMdEi1oQMSWiSa+tA4QL3hpQiTYnTjOBOElAWWSET+wZDMeziGy269Kdp4eBCp5DhnHcKpyH9YwdRTJCCzA== Received: from BN9PR03CA0228.namprd03.prod.outlook.com (2603:10b6:408:f8::23) by DM5PR12MB1388.namprd12.prod.outlook.com (2603:10b6:3:78::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.26; Wed, 21 Jul 2021 14:39:31 +0000 Received: from BN8NAM11FT060.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f8:cafe::45) by BN9PR03CA0228.outlook.office365.com (2603:10b6:408:f8::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22 via Frontend Transport; Wed, 21 Jul 2021 14:39:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT060.mail.protection.outlook.com (10.13.177.211) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:31 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:29 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad Date: Wed, 21 Jul 2021 22:37:39 +0800 Message-ID: <20210721143743.24626-13-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f3ca8113-94e8-427f-6ef5-08d94c555a2f X-MS-TrafficTypeDiagnostic: DM5PR12MB1388: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1850; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: D1eksvmjwNPEG1mk/6C5fdKNcmQdtBLnUhP8gw6cfYmpn+ugksWYVXIkyGWUcDcCyMA+i8aktHBSED2eJKDmqQIGS/RsDZenibe2XPuxKIwQQZEnskZBuokaIII4XDxlqtXeAQ1PJewhFD4LgmFIln9MgiJIAz/WhwTcjW03/brhr6XO25puzsuibGpMlPTSin5N+3s7bVxf9KRTo+YFkBmKDbniENwbxWg5RjY7PMKAW+lRzPpqQFTBsYvVrP8YK141YSaBHv2qBc7UzE0eBPSKCeO70nSqYLUOSO0BzXkhJ/D+1UUjzRPlbwPEvYDqN2FBCLURkqAhFXeo0jXrTZXizRlsOkDbTMf6RmkyOeKDFEMfErV9a/RqeUbPJevpGTIUY5gil0Dwi4kO3tQGoR9Ls792Kw6VHP5wjBjRhwiak40+28jdv8cIa0IEXXEyOIxLoxwPlwBb1qCWAWUvZiB4jkwujnEgmWDaciYfwacss5b8ICbJJICrxTleOkEC+IQOLuWn41uf7yB0NrroJ7GeCxDnmZ240mSty25Q3DWKW55PedmeprFKWY3jGbjEOd61BcR8En/FTsMxxn7qfdawjnpBeG75F3J2aJyei3vbUq/GMOHfbjdwy6t5lscCKHjKaFLyDZFbBukOfARdvt2Weg1/0s+WwzjQ0YFr6vVMbmVZSqY6sv29JrJekr6qqWM86R74dkFZ+xX3ZWGViQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(346002)(376002)(396003)(39860400002)(36840700001)(46966006)(26005)(70586007)(316002)(6286002)(70206006)(478600001)(7636003)(54906003)(356005)(36906005)(16526019)(55016002)(186003)(1076003)(107886003)(7696005)(6862004)(8676002)(83380400001)(86362001)(37006003)(6666004)(426003)(2906002)(82310400003)(36860700001)(82740400003)(4326008)(336012)(36756003)(47076005)(2616005)(8936002)(6636002)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:31.6386 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f3ca8113-94e8-427f-6ef5-08d94c555a2f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT060.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1388 Subject: [dpdk-dev] [PATCH v4 12/16] vdpa/mlx5: remove PCI specifics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon Removes PCI specific driver, replaces with common class driver. Signed-off-by: Thomas Monjalon Acked-by: Viacheslav Ovsiienko --- drivers/vdpa/mlx5/mlx5_vdpa.c | 119 ++++++++++------------------------ drivers/vdpa/mlx5/mlx5_vdpa.h | 1 - 2 files changed, 34 insertions(+), 86 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 5ab7c525c2..9c9a552ba0 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -11,12 +11,11 @@ #include #include #include -#include #include +#include #include #include -#include #include #include #include @@ -552,34 +551,13 @@ mlx5_vdpa_sys_roce_disable(const char *addr) } static int -mlx5_vdpa_roce_disable(struct rte_pci_addr *addr, struct ibv_device **ibv) +mlx5_vdpa_roce_disable(struct rte_device *dev) { - char addr_name[64] = {0}; - - rte_pci_device_name(addr, addr_name, sizeof(addr_name)); /* Firstly try to disable ROCE by Netlink and fallback to sysfs. */ - if (mlx5_vdpa_nl_roce_disable(addr_name) == 0 || - mlx5_vdpa_sys_roce_disable(addr_name) == 0) { - /* - * Succeed to disable ROCE, wait for the IB device to appear - * again after reload. - */ - int r; - struct ibv_device *ibv_new; - - for (r = MLX5_VDPA_MAX_RETRIES; r; r--) { - ibv_new = mlx5_os_get_ibv_device(addr); - if (ibv_new) { - *ibv = ibv_new; - return 0; - } - usleep(MLX5_VDPA_USEC); - } - DRV_LOG(ERR, "Cannot much device %s after ROCE disable, " - "retries exceed %d", addr_name, MLX5_VDPA_MAX_RETRIES); - rte_errno = EAGAIN; - } - return -rte_errno; + if (mlx5_vdpa_nl_roce_disable(dev->name) != 0 && + mlx5_vdpa_sys_roce_disable(dev->name) != 0) + return -rte_errno; + return 0; } static int @@ -647,44 +625,33 @@ mlx5_vdpa_config_get(struct rte_devargs *devargs, struct mlx5_vdpa_priv *priv) DRV_LOG(DEBUG, "no traffic max is %u.", priv->no_traffic_max); } -/** - * DPDK callback to register a mlx5 PCI device. - * - * This function spawns vdpa device out of a given PCI device. - * - * @param[in] pci_drv - * PCI driver structure (mlx5_vpda_driver). - * @param[in] pci_dev - * PCI device information. - * - * @return - * 0 on success, 1 to skip this driver, a negative errno value otherwise - * and rte_errno is set. - */ static int -mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev __rte_unused) +mlx5_vdpa_dev_probe(struct rte_device *dev) { struct ibv_device *ibv; struct mlx5_vdpa_priv *priv = NULL; struct ibv_context *ctx = NULL; struct mlx5_hca_attr attr; + int retry; int ret; - ibv = mlx5_os_get_ibv_device(&pci_dev->addr); - if (!ibv) { - DRV_LOG(ERR, "No matching IB device for PCI slot " - PCI_PRI_FMT ".", pci_dev->addr.domain, - pci_dev->addr.bus, pci_dev->addr.devid, - pci_dev->addr.function); + if (mlx5_vdpa_roce_disable(dev) != 0) { + DRV_LOG(WARNING, "Failed to disable ROCE for \"%s\".", + dev->name); return -rte_errno; - } else { - DRV_LOG(INFO, "PCI information matches for device \"%s\".", - ibv->name); } - if (mlx5_vdpa_roce_disable(&pci_dev->addr, &ibv) != 0) { - DRV_LOG(WARNING, "Failed to disable ROCE for \"%s\".", - ibv->name); + /* Wait for the IB device to appear again after reload. */ + for (retry = MLX5_VDPA_MAX_RETRIES; retry > 0; --retry) { + ibv = mlx5_os_get_ibv_dev(dev); + if (ibv != NULL) + break; + usleep(MLX5_VDPA_USEC); + } + if (ibv == NULL) { + DRV_LOG(ERR, "Cannot get IB device after disabling RoCE for " + "\"%s\", retries exceed %d.", + dev->name, MLX5_VDPA_MAX_RETRIES); + rte_errno = EAGAIN; return -rte_errno; } ctx = mlx5_glue->dv_open_device(ibv); @@ -722,20 +689,18 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, if (attr.num_lag_ports == 0) priv->num_lag_ports = 1; priv->ctx = ctx; - priv->pci_dev = pci_dev; priv->var = mlx5_glue->dv_alloc_var(ctx, 0); if (!priv->var) { DRV_LOG(ERR, "Failed to allocate VAR %u.", errno); goto error; } - priv->vdev = rte_vdpa_register_device(&pci_dev->device, - &mlx5_vdpa_ops); + priv->vdev = rte_vdpa_register_device(dev, &mlx5_vdpa_ops); if (priv->vdev == NULL) { DRV_LOG(ERR, "Failed to register vDPA device."); rte_errno = rte_errno ? rte_errno : EINVAL; goto error; } - mlx5_vdpa_config_get(pci_dev->device.devargs, priv); + mlx5_vdpa_config_get(dev->devargs, priv); SLIST_INIT(&priv->mr_list); pthread_mutex_init(&priv->vq_config_lock, NULL); pthread_mutex_lock(&priv_list_lock); @@ -754,26 +719,15 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, return -rte_errno; } -/** - * DPDK callback to remove a PCI device. - * - * This function removes all vDPA devices belong to a given PCI device. - * - * @param[in] pci_dev - * Pointer to the PCI device. - * - * @return - * 0 on success, the function cannot fail. - */ static int -mlx5_vdpa_pci_remove(struct rte_pci_device *pci_dev) +mlx5_vdpa_dev_remove(struct rte_device *dev) { struct mlx5_vdpa_priv *priv = NULL; int found = 0; pthread_mutex_lock(&priv_list_lock); TAILQ_FOREACH(priv, &priv_list, next) { - if (!rte_pci_addr_cmp(&priv->pci_dev->addr, &pci_dev->addr)) { + if (priv->vdev->device == dev) { found = 1; break; } @@ -831,17 +785,12 @@ static const struct rte_pci_id mlx5_vdpa_pci_id_map[] = { } }; -static struct mlx5_pci_driver mlx5_vdpa_driver = { - .driver_class = MLX5_CLASS_VDPA, - .pci_driver = { - .driver = { - .name = RTE_STR(MLX5_VDPA_DRIVER_NAME), - }, - .id_table = mlx5_vdpa_pci_id_map, - .probe = mlx5_vdpa_pci_probe, - .remove = mlx5_vdpa_pci_remove, - .drv_flags = 0, - }, +static struct mlx5_class_driver mlx5_vdpa_driver = { + .drv_class = MLX5_CLASS_VDPA, + .name = RTE_STR(MLX5_VDPA_DRIVER_NAME), + .id_table = mlx5_vdpa_pci_id_map, + .probe = mlx5_vdpa_dev_probe, + .remove = mlx5_vdpa_dev_remove, }; RTE_LOG_REGISTER_DEFAULT(mlx5_vdpa_logtype, NOTICE) @@ -853,7 +802,7 @@ RTE_INIT(rte_mlx5_vdpa_init) { mlx5_common_init(); if (mlx5_glue) - mlx5_pci_driver_register(&mlx5_vdpa_driver); + mlx5_class_driver_register(&mlx5_vdpa_driver); } RTE_PMD_EXPORT_NAME(MLX5_VDPA_DRIVER_NAME, __COUNTER__); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h index 722c72b65e..2a04e36607 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/mlx5_vdpa.h @@ -133,7 +133,6 @@ struct mlx5_vdpa_priv { struct rte_vdpa_device *vdev; /* vDPA device. */ int vid; /* vhost device id. */ struct ibv_context *ctx; /* Device context. */ - struct rte_pci_device *pci_dev; struct mlx5_hca_vdpa_attr caps; uint32_t pdn; /* Protection Domain number. */ struct ibv_pd *pd; From patchwork Wed Jul 21 14:37:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96169 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4BA92A0C52; Wed, 21 Jul 2021 16:40:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 81E0E41153; Wed, 21 Jul 2021 16:39:36 +0200 (CEST) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2077.outbound.protection.outlook.com [40.107.100.77]) by mails.dpdk.org (Postfix) with ESMTP id 26A5F41120 for ; Wed, 21 Jul 2021 16:39:35 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oH8DEoZnEOCqmqA6mTGOupIUV2VNobknOUGMPt/YHsOafuklZ9A/rxCeaqhhudurQv+cdR2gEHdBGzyH8SC/XEFv1zEyb9jreYRZEfBiffJ3/z1jyjqpBtP8VDzJH/px8az6KmzLZGfC18SjUZKJHALE4n09llUbXAoDasMvd7ChLZj4iDrsZpKQBmK6RSdFp6zu7ZHjlSF9HVRj5uMT9Cvt042HMsghGkZVdjixUvTde8J2FR++1LWVezdNGtTy7JqM6VUwI2c5MQjoAIIZ7IDbqLUDcOylRXiwJSNsHaRZ5WU5eAXRk4pNrI9muHOh+GiKmhZgLlrusmXSPm772A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1cbfXDhn2HIAmQnVJZOhmDE92rUwWIoEL1zGkoAe0M0=; b=HEpBTQHYQ8En1BZC7mD9tGWHnlqXRHYFydDguUQEz72Z+jgPuhy9RQ1gVTtDxmg4U8MM1Og1aymK5dW8kj1llt0AkmPgqO2sWr3WshRkJdDTMPCPNgqnx4AxoX8tCqDpMqeVyiTJlF44TQ0LlWR5fes/01rJQZoJJTfb+f9i22bw0bLTjbfVN+b9NhroZfEoH+Q0OocLI4Iz8bxuLCqD+FUY9fVgS0Hi+FbkiWiwnUEZNTC7yWSBVIPYJ/sYWiUWv57eQG+Mj8fKnfR6YE6iMZ7UR5JQ0DUKVHLs9F6fyQGM147N111heom+9hbnjqU+UZDc0rRl8U2h2tfvLXSfsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1cbfXDhn2HIAmQnVJZOhmDE92rUwWIoEL1zGkoAe0M0=; b=kQOAP0KPffCFGP7FSEeyu4TKlEwFTkTumvIv93S+AfAPl+uIK9Iha9W6rzdv9BR5VNPHaSVs6kMZ4p1R0UQnh3+CEci/eBl7nD3gLWgorOHaBNUUkGkPnE6OSxp2wES3eC5eCvlY7UyRhXtCCjpckFbWGH/7+jhKLLPNj2Lrt3CTpxMESaP0jDXklif6Y4gSFQTH8bkQiH/LCz5a0rYqoj8GhEAco1PMSdt3vTc6qJv3FEDx5JKEApz79aaJGABJb03xPd00y+7evtxdeWYrxzV9D0Io0aq/ve5wKxwyNqtn+fWvznegiUyDuUYz6AAVA5nyTRmi08WHnoMajvaCZg== Received: from BN9PR03CA0217.namprd03.prod.outlook.com (2603:10b6:408:f8::12) by DM5PR1201MB0090.namprd12.prod.outlook.com (2603:10b6:4:53::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22; Wed, 21 Jul 2021 14:39:33 +0000 Received: from BN8NAM11FT060.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f8:cafe::c1) by BN9PR03CA0217.outlook.office365.com (2603:10b6:408:f8::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Wed, 21 Jul 2021 14:39:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT060.mail.protection.outlook.com (10.13.177.211) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:33 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:31 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad Date: Wed, 21 Jul 2021 22:37:40 +0800 Message-ID: <20210721143743.24626-14-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 40960335-de23-4202-78ed-08d94c555b65 X-MS-TrafficTypeDiagnostic: DM5PR1201MB0090: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:565; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Z1IgaKrYRdGn2ZhHlh2ie+NUUyzdmzc0HJ43DRADVoUIXu2PrwpuHJW9ut4rDPiwx/uqxMUQ2ywTg004PA/hN2Rym+twi1HSwqNQ2xkfysTkIGwowDpDdQbL1iPeGqdTbcAF4E5bCbcICIUUH+IImFdO7kIkABqnD4RhjWy5noWyJ6HJ4rAwewAX+j1Br/Z2YCiRtMb6JZJMAE8eXfp4gojSApAtASWjKr5C0rUQ7TiejpRfrqsxAGggMGDvIiRI3Uan1Ze2CB8qBs9HL4PsM7X4KwbFHbkFaQBEJObVHeHw/lR1DcAijMmeq3xFHlYStHlMZN1uH7papCYgOIEBtckj52M7qiZlAI3bMb8ZAQMsXL9yG6+iYf9B5ZuPbU+nFpzK0inVV6yRHlCBnxB+jO8kqDgY121DdTwonCoZYZ7FLL491AUm/6CKaGtpj7A0JsJQNdkUA7bIuiQ3q+58EmGTMxqS3t1x3+u1YZW+gENoulwNrz19OUVjLydjnfB74wNlDJktbdz6wW3dPXiDSrNkIsFu3Hxop1JmvrOfYDbo+gEqGJ3QKrG/llwBPRJe7OlVAJP/C3scyPy7EEWaRwdvcIXaNLfzXQJa7dX5ak+rEWVH/iQs5/3q7fXEdrkoJegdz2ZvJqgnZS/w89+yFhM+Qzr9NHsem69Np0FGTgM4LSoTV2UBI+/58DZ4SEeAMOu1NrCZs2EFMDWj/R+Bog== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(2616005)(6666004)(86362001)(16526019)(336012)(426003)(70206006)(186003)(54906003)(70586007)(508600001)(8936002)(8676002)(7636003)(356005)(36756003)(6286002)(36860700001)(36906005)(55016002)(26005)(83380400001)(2906002)(316002)(7696005)(47076005)(107886003)(6862004)(4326008)(5660300002)(6636002)(1076003)(82310400003)(37006003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:33.7424 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 40960335-de23-4202-78ed-08d94c555b65 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT060.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB0090 Subject: [dpdk-dev] [PATCH v4 13/16] vdpa/mlx5: support SubFunction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon Supports SubFunction on auxiliary bus. SF probe devargs: auxiliary:mlx5_core.sf.,class=vdpa Signed-off-by: Thomas Monjalon Acked-by: Viacheslav Ovsiienko --- doc/guides/vdpadevs/mlx5.rst | 10 ++++++++++ drivers/vdpa/mlx5/mlx5_vdpa.c | 8 ++++++-- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/doc/guides/vdpadevs/mlx5.rst b/doc/guides/vdpadevs/mlx5.rst index 9b2f9f12c7..e81dbd0004 100644 --- a/doc/guides/vdpadevs/mlx5.rst +++ b/doc/guides/vdpadevs/mlx5.rst @@ -162,6 +162,16 @@ Driver options - 0, HW default. +Devargs example +^^^^^^^^^^^^^^^ + +- PCI devargs: + + -a 0000:03:00.2,class=vdpa + +- Auxiliary devargs: + + -a auxiliary:mlx5_core.sf.2,class=vdpa Error handling ^^^^^^^^^^^^^^ diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 9c9a552ba0..6d17d7a6f3 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -553,9 +553,13 @@ mlx5_vdpa_sys_roce_disable(const char *addr) static int mlx5_vdpa_roce_disable(struct rte_device *dev) { + char pci_addr[PCI_PRI_STR_SIZE] = { 0 }; + + if (mlx5_dev_to_pci_str(dev, pci_addr, sizeof(pci_addr)) < 0) + return -rte_errno; /* Firstly try to disable ROCE by Netlink and fallback to sysfs. */ - if (mlx5_vdpa_nl_roce_disable(dev->name) != 0 && - mlx5_vdpa_sys_roce_disable(dev->name) != 0) + if (mlx5_vdpa_nl_roce_disable(pci_addr) != 0 && + mlx5_vdpa_sys_roce_disable(pci_addr) != 0) return -rte_errno; return 0; } From patchwork Wed Jul 21 14:37:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96170 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6CCD1A0C51; Wed, 21 Jul 2021 16:40:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A12744113E; Wed, 21 Jul 2021 16:39:38 +0200 (CEST) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2049.outbound.protection.outlook.com [40.107.102.49]) by mails.dpdk.org (Postfix) with ESMTP id 9E24941164 for ; Wed, 21 Jul 2021 16:39:37 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Nmf60psGRqyzZ/DKPYNbsYXSG3MLLU9mGwNdLW04ZfrWQmUYW9YGUv4E1FCCAqffmDztYyYJNPFH6dfP1Rv193YG8WRtB7wVC4XTL6b3s2XH2GmQk7dFiim1KNSV8gpA4ZB1liUSpbJwYHPPctnrKpXyHMeORguBxiY6CFQlLzABMzt4MfOODWfADeD+bcoNI/JAsmqVxKPZAs6WD+G1ZFuJQO1Fw8CQJHBJr4hGOZl+T2oYsde1iei+MAMzpDFrTQNoSqJ9h4m7Q9dQlzyv2y8S0L47rWkSVvNTOHa3xeUfocE8vXesFvut00KfhqwoFRcxoXd3TMMXXoU9kNgtlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BOeHXbRB8KYMmw13fPNgKl/aJDNifRRIrsXweF6geO4=; b=it/slazCk2/yi9Skt1JNO4SRAt6Hw+xiI/4hGvq6PDgZm7QlCSpu0+/NV5NDStJJsHypGhceGGaSIH+p41Y9Zl4drdvZi5PGBQi1WmADdUSZI6l5GkeQwGFMrUeQBTw4WeUPe5ckYHq/peNNRRYKh3fl4EbdJbFZPt7MPXYh0ZAYDrj1GJAjRSjVMtI8aBIFzODkDaLIbRMh92E7PTV41O2TabyVUONOG4d1ElC9uoYxWzaUYi2IGB/buir7UHuBUOkxvvKPlk7KJtE3Mln+UDQMAxXDEGUgh8jFclRZIo8sc3pjUfhClsQRyIqh8RVigEhguZGTPOrT4Z7p8pO2MA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BOeHXbRB8KYMmw13fPNgKl/aJDNifRRIrsXweF6geO4=; b=JpRB9AhRTS7rjWSpSzS3l5/qeLGn+czrUb25yLSri9mafzaRCTLvR/91Mprp2FbzoaYuUjQlil+hzuFFdXVm19Hm2SGk3cMHuEEQDQg8DHzvmHp2XKnyz+smj2gXyjhMZyRRN2iQpG833oB9ShuMBo6QpXnzAkAoXeKN1yKLyWS2mTOm8Zf+ejSxQl7FqEbnsSXqVg3UshB2Mb1gZQdDfCfyrV+FURzUtljrSKN5vKo9YDR8eZIiMN8fCw08MuMPvXlTYSHknYBjKgRgpX67Vmogg1rBfXqSZjFR+KVaKXXtz5f5GUgGSOvSTO7+iZehz/zqDif9MvLn8LtEqhn1hA== Received: from BN9PR03CA0228.namprd03.prod.outlook.com (2603:10b6:408:f8::23) by BN6PR12MB1828.namprd12.prod.outlook.com (2603:10b6:404:108::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.25; Wed, 21 Jul 2021 14:39:35 +0000 Received: from BN8NAM11FT060.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f8:cafe::20) by BN9PR03CA0228.outlook.office365.com (2603:10b6:408:f8::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22 via Frontend Transport; Wed, 21 Jul 2021 14:39:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT060.mail.protection.outlook.com (10.13.177.211) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:35 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:33 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Fiona Trahe , Ashish Gupta Date: Wed, 21 Jul 2021 22:37:41 +0800 Message-ID: <20210721143743.24626-15-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: acb0e445-7d06-45d3-8d99-08d94c555c7c X-MS-TrafficTypeDiagnostic: BN6PR12MB1828: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4303; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +GxJYBDLB6ZWfzPl2qPg2TTvNIID4uNxXQgiYg8w+68iffQL9ZSftaLaSj9eBLehQlAqcgxagpVqXrOu7LurypFBzH7KJkkrQMvut7nknqGMexV577d7scO1moqePlG2wWdzIJ5pYDEsOPtgLyllA8Df0a2rxEEU5W2fxUZAlV8MoXKRzuZaBFCRw5OSAZ+bmFDkeJdZ2dN14ThzRGMVpNuIN0UnBjvNFl0BDkLJ3zya16vU1k7/B+rETqQNKbIzmKrs4UG4Y60wVejSFybtcN9DZJpPDJpREjgrbKxJ15FvTPYXpxTv4FCvl/3N5LGaCI6/lq1vUWq1anwjaJY70j6F/DfZteYoLLb3Ld7z0KODpgjIU3TY30XhfhKu5NF+sAimqZ1lE/X6KZ8/d13sC9mwfBaNh61W1yCqFnOGrFhb7v/EqUCz/HMtTgeG50wae0+jnLR/EhvPzb+Gv3ndnZ9Uh5p2qIXLmK0m/aM/9zIULsC2RX0Doe593Aui8TZitrLhTca1Ly5QA8G2tVMVnCS2slwstUSzA/LWKW18sC6JXG+uAn7mk3GbYcQQkeExQ3viw68jpLvnF149VnawGNoiAlfypDZUsPWVV3fwSFRvlykpe/yNT7T2vR/LAbk7m47osSHASC4Ntsb1DoTqce0atNm8/HI4vgIOta6SoQOPfgImIufWnMn0Z6yC7yXjjV+jyA8dPZURimS/oCdWFQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(8676002)(6636002)(316002)(54906003)(36906005)(37006003)(6666004)(8936002)(86362001)(82310400003)(426003)(4326008)(6286002)(6862004)(2616005)(2906002)(356005)(336012)(47076005)(36756003)(1076003)(83380400001)(7636003)(26005)(5660300002)(70206006)(70586007)(186003)(508600001)(16526019)(7696005)(36860700001)(55016002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:35.5584 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: acb0e445-7d06-45d3-8d99-08d94c555c7c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT060.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1828 Subject: [dpdk-dev] [PATCH v4 14/16] compress/mlx5: migrate to common driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support auxiliary bus, upgrades driver to use mlx5 common driver structure. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/compress/mlx5/mlx5_compress.c | 71 ++++++--------------------- 1 file changed, 15 insertions(+), 56 deletions(-) diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c index f5f51c0ebe..9775c81789 100644 --- a/drivers/compress/mlx5/mlx5_compress.c +++ b/drivers/compress/mlx5/mlx5_compress.c @@ -5,7 +5,7 @@ #include #include #include -#include +#include #include #include #include @@ -13,7 +13,6 @@ #include #include -#include #include #include #include @@ -37,7 +36,6 @@ struct mlx5_compress_xform { struct mlx5_compress_priv { TAILQ_ENTRY(mlx5_compress_priv) next; struct ibv_context *ctx; /* Device context. */ - struct rte_pci_device *pci_dev; struct rte_compressdev *cdev; void *uar; uint32_t pdn; /* Protection Domain number. */ @@ -780,23 +778,8 @@ mlx5_compress_mr_mem_event_cb(enum rte_mem_event event_type, const void *addr, } } -/** - * DPDK callback to register a PCI device. - * - * This function spawns compress device out of a given PCI device. - * - * @param[in] pci_drv - * PCI driver structure (mlx5_compress_driver). - * @param[in] pci_dev - * PCI device information. - * - * @return - * 0 on success, 1 to skip this driver, a negative errno value otherwise - * and rte_errno is set. - */ static int -mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, - struct rte_pci_device *pci_dev) +mlx5_compress_dev_probe(struct rte_device *dev) { struct ibv_device *ibv; struct rte_compressdev *cdev; @@ -805,24 +788,17 @@ mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, struct mlx5_hca_attr att = { 0 }; struct rte_compressdev_pmd_init_params init_params = { .name = "", - .socket_id = pci_dev->device.numa_node, + .socket_id = dev->numa_node, }; - RTE_SET_USED(pci_drv); if (rte_eal_process_type() != RTE_PROC_PRIMARY) { DRV_LOG(ERR, "Non-primary process type is not supported."); rte_errno = ENOTSUP; return -rte_errno; } - ibv = mlx5_os_get_ibv_device(&pci_dev->addr); - if (ibv == NULL) { - DRV_LOG(ERR, "No matching IB device for PCI slot " - PCI_PRI_FMT ".", pci_dev->addr.domain, - pci_dev->addr.bus, pci_dev->addr.devid, - pci_dev->addr.function); + ibv = mlx5_os_get_ibv_dev(dev); + if (ibv == NULL) return -rte_errno; - } - DRV_LOG(INFO, "PCI information matches for device \"%s\".", ibv->name); ctx = mlx5_glue->dv_open_device(ibv); if (ctx == NULL) { DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); @@ -838,7 +814,7 @@ mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, rte_errno = ENOTSUP; return -ENOTSUP; } - cdev = rte_compressdev_pmd_create(ibv->name, &pci_dev->device, + cdev = rte_compressdev_pmd_create(ibv->name, dev, sizeof(*priv), &init_params); if (cdev == NULL) { DRV_LOG(ERR, "Failed to create device \"%s\".", ibv->name); @@ -853,7 +829,6 @@ mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, cdev->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED; priv = cdev->data->dev_private; priv->ctx = ctx; - priv->pci_dev = pci_dev; priv->cdev = cdev; priv->min_block_size = att.compress_min_block_size; priv->sq_ts_format = att.sq_ts_format; @@ -884,25 +859,14 @@ mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, return 0; } -/** - * DPDK callback to remove a PCI device. - * - * This function removes all compress devices belong to a given PCI device. - * - * @param[in] pci_dev - * Pointer to the PCI device. - * - * @return - * 0 on success, the function cannot fail. - */ static int -mlx5_compress_pci_remove(struct rte_pci_device *pdev) +mlx5_compress_dev_remove(struct rte_device *dev) { struct mlx5_compress_priv *priv = NULL; pthread_mutex_lock(&priv_list_lock); TAILQ_FOREACH(priv, &mlx5_compress_priv_list, next) - if (rte_pci_addr_cmp(&priv->pci_dev->addr, &pdev->addr) != 0) + if (priv->cdev->device == dev) break; if (priv) TAILQ_REMOVE(&mlx5_compress_priv_list, priv, next); @@ -929,24 +893,19 @@ static const struct rte_pci_id mlx5_compress_pci_id_map[] = { } }; -static struct mlx5_pci_driver mlx5_compress_driver = { - .driver_class = MLX5_CLASS_COMPRESS, - .pci_driver = { - .driver = { - .name = RTE_STR(MLX5_COMPRESS_DRIVER_NAME), - }, - .id_table = mlx5_compress_pci_id_map, - .probe = mlx5_compress_pci_probe, - .remove = mlx5_compress_pci_remove, - .drv_flags = 0, - }, +static struct mlx5_class_driver mlx5_compress_driver = { + .drv_class = MLX5_CLASS_COMPRESS, + .name = RTE_STR(MLX5_COMPRESS_DRIVER_NAME), + .id_table = mlx5_compress_pci_id_map, + .probe = mlx5_compress_dev_probe, + .remove = mlx5_compress_dev_remove, }; RTE_INIT(rte_mlx5_compress_init) { mlx5_common_init(); if (mlx5_glue != NULL) - mlx5_pci_driver_register(&mlx5_compress_driver); + mlx5_class_driver_register(&mlx5_compress_driver); } RTE_LOG_REGISTER_DEFAULT(mlx5_compress_logtype, NOTICE) From patchwork Wed Jul 21 14:37:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96171 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 82249A0C51; Wed, 21 Jul 2021 16:40:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 954B6410F6; Wed, 21 Jul 2021 16:39:57 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2049.outbound.protection.outlook.com [40.107.237.49]) by mails.dpdk.org (Postfix) with ESMTP id 4AFE540DF8 for ; Wed, 21 Jul 2021 16:39:56 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HKyfPgZq+sx2Z4KL31TOMDM8dYdyLkLdCmsh6oU7s0lcS8T7NJn4p3N1BpEb6/BQ1VZc6sozHHMb1hfE3k6HCTO6btxze6F1QNJruAWuRY8ZEwGAvgQ05qr8oOE9m7iqlM0MrvdSfyJfazSl6DKT1kql0T9xjXyv1LoMOBONuKOlxEjqtevIJlprlJTSIi+ndxaO3Aa7eANnWaS2XiURcE81vXxVUY85vIBJCjeTTYN6yZPgbxXFrAhHO8dvKuuA8Lwonx/ALSRXFvv0/6YP/t0EK9eP8JQNRXfTSpqQdv8LBG1oI0L+6ELkyIb/2ODJLAMszj0mgUD/vCu/QgYNKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V4ILCgNy2Guy8F2M2xD6oU9UDJUEw57m8p9RuwHeahg=; b=dLO0Jvy4oHkVO1s1zcAkloQKX8XZlLF04kwh9w/bgPyLOI7JSwgp2zm91WDyBNa1qT3E4Y8XBEOVLo0UJwjDXKn6R0U0CKN3Igk6nWkbd61aTBLX/HMCb3ZKedoIXlqJ2Ude14rTRnNzTkEhLuCbgwcreG6CCEJKEVxqZAXIpNJe/hu5VFRS8s7U/x9IcHemJyhxS6glYwmYOA3xzjY7ImAABD5g1sxIqo5QMEdA6BzjF3qziOUuEkTUp42IClrDDiZGH9tZaS8rwuVIp96rj5vY1iUysjav1pPB4iR9TgM+o3jtjOJhNNMkYkaRUBWI4ABykpelOd7LpO2rss7Brw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V4ILCgNy2Guy8F2M2xD6oU9UDJUEw57m8p9RuwHeahg=; b=WnoNpyxA5YRe5H1cqy6xStHzYMSyescCEPrxVehU4qCV4YZ2iDTUfjFs7Jfz+XR5Kz0VOyfXkY6HHOjb3jXbTicZ0ZSRJrhTZdMiOGr5SI7dh538S9N/A1TSIgbrfUQxyK98eV2lHp+i0c5m/oUUjejXFiGQLF2EEUYcz1prcjWg3HJDqnFRdMfn9ElkUGaOESzz0SE10hn0BuO5P1bgZ9QTfiRwP1Uj8v3uBp4KOjCwA0j6xLZEQyYDtoDO83lnVd7dEMrtxhuXyk5Z/fzTQAkQ/SyATEP3kSJTHDsoSCBnezpfN3BIescKYTAzCOH6BA/9XpKZwyB9XDoU/QUymA== Received: from BN6PR16CA0048.namprd16.prod.outlook.com (2603:10b6:405:14::34) by MN2PR12MB4077.namprd12.prod.outlook.com (2603:10b6:208:1da::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.24; Wed, 21 Jul 2021 14:39:55 +0000 Received: from BN8NAM11FT056.eop-nam11.prod.protection.outlook.com (2603:10b6:405:14:cafe::41) by BN6PR16CA0048.outlook.office365.com (2603:10b6:405:14::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT056.mail.protection.outlook.com (10.13.177.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:54 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:52 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad Date: Wed, 21 Jul 2021 22:37:42 +0800 Message-ID: <20210721143743.24626-16-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b22d11bc-1b70-430e-a18d-08d94c556807 X-MS-TrafficTypeDiagnostic: MN2PR12MB4077: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:883; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3M5XoBO16DtOS9Moe6oawNQe0hKRSl4bE0ZyGxhmmW3iSPItSNabT3cyw80wXWWiNfGOlWfNzJU+d8lNEK8nAS6tTwi3L8L9LOYP8qSZCYPOTfmawDW/P6vFA68U5RQHESfR8ve2iiLCWTWYRfaRTJvR/mLe87YQzzUqGK8KMM0yASHJsphKlmSRCb/nb/MreeLeKbVTLwxKWWb/gaYcwGoPCsvCErxrnu2nGb5nvBvDjiGgYGMKTi9eV6EJE3eoMUT8rRSZOBDw7xZCTva70LuVTOpuiDcn5BIqjDLzIx2n/PaRQeLM2V1La/6WvdZwY6LAlOyhY/ro2Mhgw+MxhLEJSlyqvDDErRv3OURhbM+P+6XQjv//Iu8bsAx4vmNlHUNMVGsrvY10PFAYWwHzWrU0L2+kdwwNXMO+ofOIWFTwGq9MJJJrqnkr4YUbrUgnZAvG3ivUybhdzPdEHhqkeMkwqi7EbW0qbVR4jesOx/1QkBtIpe4Z62bGUuUXpUuBuvA8nO+2P3VrE415YV5vHdbpR6fRAtnMRCfG/y/ErMi0m+0x/LHBj2SjQizM3G9GI/pSdigtBk7i3SJ4STafF/TLFZkgSweO0czaBGR+Kdd0hip1uwtlnWGemd6DDdzkb6JIZa6pxDfUNhe+BklihfdR4ZCEJXxqm7MMwzoPdJdLCYugpUZ+RON/DxocZ5LDWm//hhY9Djo9p7RaCGAv8A== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(396003)(39860400002)(136003)(376002)(46966006)(36840700001)(6636002)(1076003)(7636003)(186003)(16526019)(4326008)(7696005)(107886003)(478600001)(2906002)(8676002)(5660300002)(55016002)(26005)(6862004)(82310400003)(36860700001)(426003)(70586007)(86362001)(336012)(6666004)(8936002)(36756003)(6286002)(37006003)(54906003)(356005)(2616005)(82740400003)(316002)(36906005)(47076005)(70206006)(83380400001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:54.9318 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b22d11bc-1b70-430e-a18d-08d94c556807 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT056.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4077 Subject: [dpdk-dev] [PATCH v4 15/16] crypto/mlx5: migrate to common driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support auxiliary bus, upgrades driver to use mlx5 common driver structure. Signed-off-by: Xueming Li --- drivers/crypto/mlx5/mlx5_crypto.c | 61 ++++++++----------------------- drivers/crypto/mlx5/mlx5_crypto.h | 1 - 2 files changed, 16 insertions(+), 46 deletions(-) diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index fc05bb7d46..ea734f4d5c 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -6,12 +6,11 @@ #include #include #include -#include +#include #include #include #include -#include #include #include @@ -977,23 +976,8 @@ mlx5_crypto_mr_mem_event_cb(enum rte_mem_event event_type, const void *addr, } } -/** - * DPDK callback to register a PCI device. - * - * This function spawns crypto device out of a given PCI device. - * - * @param[in] pci_drv - * PCI driver structure (mlx5_crypto_driver). - * @param[in] pci_dev - * PCI device information. - * - * @return - * 0 on success, 1 to skip this driver, a negative errno value otherwise - * and rte_errno is set. - */ static int -mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, - struct rte_pci_device *pci_dev) +mlx5_crypto_dev_probe(struct rte_device *dev) { struct ibv_device *ibv; struct rte_cryptodev *crypto_dev; @@ -1005,28 +989,21 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, struct rte_cryptodev_pmd_init_params init_params = { .name = "", .private_data_size = sizeof(struct mlx5_crypto_priv), - .socket_id = pci_dev->device.numa_node, + .socket_id = dev->numa_node, .max_nb_queue_pairs = RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS, }; uint16_t rdmw_wqe_size; int ret; - RTE_SET_USED(pci_drv); if (rte_eal_process_type() != RTE_PROC_PRIMARY) { DRV_LOG(ERR, "Non-primary process type is not supported."); rte_errno = ENOTSUP; return -rte_errno; } - ibv = mlx5_os_get_ibv_device(&pci_dev->addr); - if (ibv == NULL) { - DRV_LOG(ERR, "No matching IB device for PCI slot " - PCI_PRI_FMT ".", pci_dev->addr.domain, - pci_dev->addr.bus, pci_dev->addr.devid, - pci_dev->addr.function); + ibv = mlx5_os_get_ibv_dev(dev); + if (ibv == NULL) return -rte_errno; - } - DRV_LOG(INFO, "PCI information matches for device \"%s\".", ibv->name); ctx = mlx5_glue->dv_open_device(ibv); if (ctx == NULL) { DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); @@ -1041,7 +1018,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, rte_errno = ENOTSUP; return -ENOTSUP; } - ret = mlx5_crypto_parse_devargs(pci_dev->device.devargs, &devarg_prms); + ret = mlx5_crypto_parse_devargs(dev->devargs, &devarg_prms); if (ret) { DRV_LOG(ERR, "Failed to parse devargs."); return -rte_errno; @@ -1052,7 +1029,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, DRV_LOG(ERR, "Failed to configure login."); return -rte_errno; } - crypto_dev = rte_cryptodev_pmd_create(ibv->name, &pci_dev->device, + crypto_dev = rte_cryptodev_pmd_create(ibv->name, dev, &init_params); if (crypto_dev == NULL) { DRV_LOG(ERR, "Failed to create device \"%s\".", ibv->name); @@ -1069,7 +1046,6 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, priv = crypto_dev->data->dev_private; priv->ctx = ctx; priv->login_obj = login; - priv->pci_dev = pci_dev; priv->crypto_dev = crypto_dev; if (mlx5_crypto_hw_global_prepare(priv) != 0) { rte_cryptodev_pmd_destroy(priv->crypto_dev); @@ -1112,13 +1088,13 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, } static int -mlx5_crypto_pci_remove(struct rte_pci_device *pdev) +mlx5_crypto_dev_remove(struct rte_device *dev) { struct mlx5_crypto_priv *priv = NULL; pthread_mutex_lock(&priv_list_lock); TAILQ_FOREACH(priv, &mlx5_crypto_priv_list, next) - if (rte_pci_addr_cmp(&priv->pci_dev->addr, &pdev->addr) != 0) + if (priv->crypto_dev->device == dev) break; if (priv) TAILQ_REMOVE(&mlx5_crypto_priv_list, priv, next); @@ -1146,24 +1122,19 @@ static const struct rte_pci_id mlx5_crypto_pci_id_map[] = { } }; -static struct mlx5_pci_driver mlx5_crypto_driver = { - .driver_class = MLX5_CLASS_CRYPTO, - .pci_driver = { - .driver = { - .name = RTE_STR(MLX5_CRYPTO_DRIVER_NAME), - }, - .id_table = mlx5_crypto_pci_id_map, - .probe = mlx5_crypto_pci_probe, - .remove = mlx5_crypto_pci_remove, - .drv_flags = 0, - }, +static struct mlx5_class_driver mlx5_crypto_driver = { + .drv_class = MLX5_CLASS_CRYPTO, + .name = RTE_STR(MLX5_CRYPTO_DRIVER_NAME), + .id_table = mlx5_crypto_pci_id_map, + .probe = mlx5_crypto_dev_probe, + .remove = mlx5_crypto_dev_remove, }; RTE_INIT(rte_mlx5_crypto_init) { mlx5_common_init(); if (mlx5_glue != NULL) - mlx5_pci_driver_register(&mlx5_crypto_driver); + mlx5_class_driver_register(&mlx5_crypto_driver); } RTE_PMD_REGISTER_CRYPTO_DRIVER(mlx5_cryptodev_driver, mlx5_drv, diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index e751c9c202..d49b0001f0 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -20,7 +20,6 @@ struct mlx5_crypto_priv { TAILQ_ENTRY(mlx5_crypto_priv) next; struct ibv_context *ctx; /* Device context. */ - struct rte_pci_device *pci_dev; struct rte_cryptodev *crypto_dev; void *uar; /* User Access Region. */ volatile uint64_t *uar_addr; From patchwork Wed Jul 21 14:37:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 96172 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C0087A0C51; Wed, 21 Jul 2021 16:40:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C43E14111B; Wed, 21 Jul 2021 16:40:01 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2054.outbound.protection.outlook.com [40.107.94.54]) by mails.dpdk.org (Postfix) with ESMTP id AF7D84114B for ; Wed, 21 Jul 2021 16:40:00 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SJJW3PBXtlO7Fh143oQgGgmWIf3+w6Br8dorlAhI4ZOJlO1XZ2KkEk+xT44+JeJXerQGJk/pVvYp4PLXm/jblS7GVTqqtKVIJsGYQCfIn/LA1h2kv4e+q8oLEY9NNstkzwOSwa5+PHAj2ntbCaELzKemxlHh+FiE1/sgTJoUd5vLJpKO05hs3yKX+Bi+aQv028xF6DjuwWTNHJgYxGIE3o7O/e+BfNStW69J/uDCgw5leb47WlDhRUzEvevcAqxqNLQrG5NTYYfNBGICsBa7l0rq/wB5J/4sfLMYpqjAWXJoMjdMUwBFOcqU8UZlusFCflxhp7QLsVqVFZ2LvtcVWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hB57NZTEI8r8aCZSq4AWy2HtvTV8NxC3SwrJw/i/ugo=; b=I9hID2dMUkEC4jWDWQCKBzqzac/Ao2y/rLv6UrwmTBs+/J3NRdsGisDtylCu1j87EqXdh2RQ4vsQbHdnP1549tiF2+y/NqPr4XWPp8xDjjHBJFLYNIMC4ljpQFzLusc4WfaSfJtFRtkeytZoeg5TDWBu0bQ2gf+4W6NNFIFyqklbAgQG6/QREHVoGWjj0ZYQL7klejTtdommRX7NwTTdifsL55+3WEI4RiVZ6JthiOLXvWDYDe70cRjKJlfcrnI6M/w34s6SPqgXQuuxGhz1C38xyP1eq8Yr5fTo2oiuTne3EW9tjE7XmkSjhTqztLNnK767uQZQfxHrI+Ad6a/ekw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=ashroe.eu smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hB57NZTEI8r8aCZSq4AWy2HtvTV8NxC3SwrJw/i/ugo=; b=jV9pbqSaOxe2ziLkT8CtPozGu5Z/jON/ALopoAyf76lR5oNDJdDEUw4TkuH1Jxc6xNZvukh4vRlpgKYI6Ei4gwY/+t0Qlpc8iwBsp5cHFwCLkHBee5DjRw5jQORMtf8ScIPGNxnh75hnT+dLcXmQvdD8mdJIYVhP7levikDVvUGciuV4I1vAxpynbIVPWQiq6PbvNf8mr1Xg75vIZ79ot/Wg7T++ggU9HGpyBc6ujf2Xs/W6DULZ4xTSUzDGY2lT8EvHwuqSFZk9Hj+Z3KdGZTAveeZat7InWRj4nMQ17YJnNGIN1XUU0seQdEz8omvU6okQG5G8GPCdFNlB0i5Ihg== Received: from BN9PR03CA0935.namprd03.prod.outlook.com (2603:10b6:408:108::10) by CY4PR12MB1510.namprd12.prod.outlook.com (2603:10b6:910:9::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.24; Wed, 21 Jul 2021 14:39:58 +0000 Received: from BN8NAM11FT010.eop-nam11.prod.protection.outlook.com (2603:10b6:408:108:cafe::c2) by BN9PR03CA0935.outlook.office365.com (2603:10b6:408:108::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22 via Frontend Transport; Wed, 21 Jul 2021 14:39:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; ashroe.eu; dkim=none (message not signed) header.d=none;ashroe.eu; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT010.mail.protection.outlook.com (10.13.177.53) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Wed, 21 Jul 2021 14:39:57 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 21 Jul 2021 14:39:54 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Ray Kinsella Date: Wed, 21 Jul 2021 22:37:43 +0800 Message-ID: <20210721143743.24626-17-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210721143743.24626-1-xuemingl@nvidia.com> References: <20210616040935.311733-1-xuemingl@nvidia.com> <20210721143743.24626-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a5849322-3e15-420b-1c65-08d94c5569aa X-MS-TrafficTypeDiagnostic: CY4PR12MB1510: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:586; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: r3Ogv6Hi3SwCxrPAZhzSb5NLtw/8QCA4o22gS2ZlCkM8DyQnown4xVE2hW3rYBhrYu9KlzSxQmrJd19u7D+3kw0oXL568iJ21HHWl8WOSobLo4+JY3HX9FY0E9C2UWQV4pNWzgoUm42aUuRMZl7Rq50C5PIbZWJ2Vby/+tXJuL4ALgMJ80yWudTHUzM6VbAEYZbLFYzkOzhB4h8pxGHJAeNomVKgn0nS9Gt0z2EbRimV4NojmYgqAYolP9z28w5+506u88NtUuwbtv2h1SodUAALCD3fV6Miff7BB6mJejKSYvG2Pa+mCRkEMvEZEGYuFKIGYe4nyuW4MpOKGZRK6ghyWhGWSrmALtoUipXVpy7eizRZKTDeKqB0l0brJxel8Vm3hnMcuMFwzwYuG+K4Gja+LpxPhCYS/3hf6LLvSaFfp/QC7opIAnSAiRi3/g5swxBXy0rh7vFYyD/lXtTyqAbRuB0jPiGij1+QfNB0FdsxqfpkFr3/TDTotLRNfEbQJGb/ovnVzI5ufpICQL2/VX7tB5AqHJgDnl7Rupp16asztwjc4dDYrw/aVCJswqFuL0Es/VlCB1GPyQsfrO9rHxTF6bd2cHev1jZCPmvMfZZuwBPW9eL8gMpzTmWkwrLic+LHMzynyEeIiWg8wQ9eocHJDvl4TNrIKI2YBEhsoOKjoR2WvesPAbQo6swC5et5Aj4yHqJ6yCcx8Za0B0Tp1g== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(346002)(136003)(376002)(39860400002)(46966006)(36840700001)(54906003)(37006003)(316002)(4326008)(6862004)(6666004)(36906005)(70586007)(6636002)(82310400003)(36756003)(336012)(7636003)(86362001)(2616005)(2906002)(356005)(8936002)(6286002)(426003)(70206006)(8676002)(55016002)(82740400003)(47076005)(1076003)(30864003)(186003)(26005)(16526019)(36860700001)(478600001)(83380400001)(7696005)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2021 14:39:57.6044 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a5849322-3e15-420b-1c65-08d94c5569aa X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT010.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1510 Subject: [dpdk-dev] [PATCH v4 16/16] common/mlx5: clean up legacy PCI bus driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Clean up legacy PCI bus driver since all mlx5 PMDs moved to new common PCI bus driver. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/mlx5_common_os.h | 1 - drivers/common/mlx5/mlx5_common.c | 1 - drivers/common/mlx5/mlx5_common.h | 1 + drivers/common/mlx5/mlx5_common_pci.c | 449 +-------------------- drivers/common/mlx5/mlx5_common_pci.h | 78 ---- drivers/common/mlx5/mlx5_common_private.h | 1 + drivers/common/mlx5/version.map | 3 - 7 files changed, 4 insertions(+), 530 deletions(-) delete mode 100644 drivers/common/mlx5/mlx5_common_pci.h diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h index 86d0cb09b0..2b03bf811e 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.h +++ b/drivers/common/mlx5/linux/mlx5_common_os.h @@ -289,7 +289,6 @@ mlx5_os_free(void *addr) free(addr); } -__rte_internal struct ibv_device * mlx5_os_get_ibv_device(const struct rte_pci_addr *addr); diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 8fe36f7077..459cf4bcc4 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -14,7 +14,6 @@ #include "mlx5_common.h" #include "mlx5_common_os.h" #include "mlx5_common_log.h" -#include "mlx5_common_pci.h" #include "mlx5_common_private.h" uint8_t haswell_broadwell_cpu; diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 66c83047a1..25737bfffb 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index a7db6e2c11..8b38091d87 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -8,447 +8,17 @@ #include #include #include +#include +#include #include "mlx5_common_log.h" -#include "mlx5_common_pci.h" #include "mlx5_common_private.h" static struct rte_pci_driver mlx5_common_pci_driver; -/********** Legacy PCI bus driver, to be removed ********/ - -struct mlx5_pci_device { - struct rte_pci_device *pci_dev; - TAILQ_ENTRY(mlx5_pci_device) next; - uint32_t classes_loaded; -}; - -/* Head of list of drivers. */ -static TAILQ_HEAD(mlx5_pci_bus_drv_head, mlx5_pci_driver) drv_list = - TAILQ_HEAD_INITIALIZER(drv_list); - -/* Head of mlx5 pci devices. */ -static TAILQ_HEAD(mlx5_pci_devices_head, mlx5_pci_device) devices_list = - TAILQ_HEAD_INITIALIZER(devices_list); - -static const struct { - const char *name; - unsigned int driver_class; -} mlx5_classes[] = { - { .name = "vdpa", .driver_class = MLX5_CLASS_VDPA }, - { .name = "eth", .driver_class = MLX5_CLASS_ETH }, - /* Keep name "net" for backward compatibility. */ - { .name = "net", .driver_class = MLX5_CLASS_ETH }, - { .name = "regex", .driver_class = MLX5_CLASS_REGEX }, - { .name = "compress", .driver_class = MLX5_CLASS_COMPRESS }, - { .name = "crypto", .driver_class = MLX5_CLASS_CRYPTO }, -}; - -static const unsigned int mlx5_class_combinations[] = { - MLX5_CLASS_ETH, - MLX5_CLASS_VDPA, - MLX5_CLASS_REGEX, - MLX5_CLASS_COMPRESS, - MLX5_CLASS_CRYPTO, - MLX5_CLASS_ETH | MLX5_CLASS_REGEX, - MLX5_CLASS_VDPA | MLX5_CLASS_REGEX, - MLX5_CLASS_ETH | MLX5_CLASS_COMPRESS, - MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS, - MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, - MLX5_CLASS_ETH | MLX5_CLASS_CRYPTO, - MLX5_CLASS_ETH | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, - MLX5_CLASS_VDPA | MLX5_CLASS_CRYPTO, - MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, - MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, - MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, - MLX5_CLASS_ETH | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, - MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, - MLX5_CLASS_ETH | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, - MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, - MLX5_CLASS_ETH | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | - MLX5_CLASS_CRYPTO, - MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | - MLX5_CLASS_CRYPTO, - /* New class combination should be added here. */ -}; - -static int -class_name_to_value(const char *class_name) -{ - unsigned int i; - - for (i = 0; i < RTE_DIM(mlx5_classes); i++) { - if (strcmp(class_name, mlx5_classes[i].name) == 0) - return mlx5_classes[i].driver_class; - } - return -EINVAL; -} - -static struct mlx5_pci_driver * -driver_get(uint32_t class) -{ - struct mlx5_pci_driver *driver; - - TAILQ_FOREACH(driver, &drv_list, next) { - if (driver->driver_class == class) - return driver; - } - return NULL; -} - -static int -bus_cmdline_options_handler(__rte_unused const char *key, - const char *class_names, void *opaque) -{ - int *ret = opaque; - char *nstr_org; - int class_val; - char *found; - char *nstr; - char *refstr = NULL; - - *ret = 0; - nstr = strdup(class_names); - if (!nstr) { - *ret = -ENOMEM; - return *ret; - } - nstr_org = nstr; - found = strtok_r(nstr, ":", &refstr); - if (!found) - goto err; - do { - /* Extract each individual class name. Multiple - * class key,value is supplied as class=net:vdpa:foo:bar. - */ - class_val = class_name_to_value(found); - /* Check if its a valid class. */ - if (class_val < 0) { - *ret = -EINVAL; - goto err; - } - *ret |= class_val; - found = strtok_r(NULL, ":", &refstr); - } while (found); -err: - free(nstr_org); - if (*ret < 0) - DRV_LOG(ERR, "Invalid mlx5 class options %s." - " Maybe typo in device class argument setting?", - class_names); - return *ret; -} - -static int -parse_class_options(const struct rte_devargs *devargs) -{ - const char *key = RTE_DEVARGS_KEY_CLASS; - struct rte_kvargs *kvlist; - int ret = 0; - - if (devargs == NULL) - return 0; - kvlist = rte_kvargs_parse(devargs->args, NULL); - if (kvlist == NULL) - return 0; - if (rte_kvargs_count(kvlist, key)) - rte_kvargs_process(kvlist, key, bus_cmdline_options_handler, - &ret); - rte_kvargs_free(kvlist); - return ret; -} - -static bool -mlx5_bus_match(const struct mlx5_pci_driver *drv, - const struct rte_pci_device *pci_dev) -{ - const struct rte_pci_id *id_table; - - for (id_table = drv->pci_driver.id_table; id_table->vendor_id != 0; - id_table++) { - /* Check if device's ids match the class driver's ids. */ - if (id_table->vendor_id != pci_dev->id.vendor_id && - id_table->vendor_id != RTE_PCI_ANY_ID) - continue; - if (id_table->device_id != pci_dev->id.device_id && - id_table->device_id != RTE_PCI_ANY_ID) - continue; - if (id_table->subsystem_vendor_id != - pci_dev->id.subsystem_vendor_id && - id_table->subsystem_vendor_id != RTE_PCI_ANY_ID) - continue; - if (id_table->subsystem_device_id != - pci_dev->id.subsystem_device_id && - id_table->subsystem_device_id != RTE_PCI_ANY_ID) - continue; - if (id_table->class_id != pci_dev->id.class_id && - id_table->class_id != RTE_CLASS_ANY_ID) - continue; - return true; - } - return false; -} - -static int -is_valid_class_combination(uint32_t user_classes) -{ - unsigned int i; - - /* Verify if user specified valid supported combination. */ - for (i = 0; i < RTE_DIM(mlx5_class_combinations); i++) { - if (mlx5_class_combinations[i] == user_classes) - return 0; - } - /* Not found any valid class combination. */ - return -EINVAL; -} - -static struct mlx5_pci_device * -pci_to_mlx5_device(const struct rte_pci_device *pci_dev) -{ - struct mlx5_pci_device *dev; - - TAILQ_FOREACH(dev, &devices_list, next) { - if (dev->pci_dev == pci_dev) - return dev; - } - return NULL; -} - -static bool -device_class_enabled(const struct mlx5_pci_device *device, uint32_t class) -{ - return (device->classes_loaded & class) ? true : false; -} - -static void -dev_release(struct mlx5_pci_device *dev) -{ - TAILQ_REMOVE(&devices_list, dev, next); - rte_free(dev); -} - -static int -drivers_remove(struct mlx5_pci_device *dev, uint32_t enabled_classes) -{ - struct mlx5_pci_driver *driver; - int local_ret = -ENODEV; - unsigned int i = 0; - int ret = 0; - - enabled_classes &= dev->classes_loaded; - while (enabled_classes) { - driver = driver_get(RTE_BIT64(i)); - if (driver) { - local_ret = driver->pci_driver.remove(dev->pci_dev); - if (!local_ret) - dev->classes_loaded &= ~RTE_BIT64(i); - else if (ret == 0) - ret = local_ret; - } - enabled_classes &= ~RTE_BIT64(i); - i++; - } - if (local_ret) - ret = local_ret; - return ret; -} - -static int -drivers_probe(struct mlx5_pci_device *dev, struct rte_pci_driver *pci_drv, - struct rte_pci_device *pci_dev, uint32_t user_classes) -{ - struct mlx5_pci_driver *driver; - uint32_t enabled_classes = 0; - bool already_loaded; - int ret; - - TAILQ_FOREACH(driver, &drv_list, next) { - if ((driver->driver_class & user_classes) == 0) - continue; - if (!mlx5_bus_match(driver, pci_dev)) - continue; - already_loaded = dev->classes_loaded & driver->driver_class; - if (already_loaded && - !(driver->pci_driver.drv_flags & RTE_PCI_DRV_PROBE_AGAIN)) { - DRV_LOG(ERR, "Device %s is already probed", - pci_dev->device.name); - ret = -EEXIST; - goto probe_err; - } - ret = driver->pci_driver.probe(pci_drv, pci_dev); - if (ret < 0) { - DRV_LOG(ERR, "Failed to load driver %s", - driver->pci_driver.driver.name); - goto probe_err; - } - enabled_classes |= driver->driver_class; - } - dev->classes_loaded |= enabled_classes; - return 0; -probe_err: - /* Only unload drivers which are enabled which were enabled - * in this probe instance. - */ - drivers_remove(dev, enabled_classes); - return ret; -} - -/** - * DPDK callback to register to probe multiple drivers for a PCI device. - * - * @param[in] pci_drv - * PCI driver structure. - * @param[in] dev - * PCI device information. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) -{ - struct mlx5_pci_device *dev; - uint32_t user_classes = 0; - bool new_device = false; - int ret; - - ret = parse_class_options(pci_dev->device.devargs); - if (ret < 0) - return ret; - user_classes = ret; - if (user_classes) { - /* Validate combination here. */ - ret = is_valid_class_combination(user_classes); - if (ret) { - DRV_LOG(ERR, "Unsupported mlx5 classes supplied."); - return ret; - } - } else { - /* Default to net class. */ - user_classes = MLX5_CLASS_ETH; - } - dev = pci_to_mlx5_device(pci_dev); - if (!dev) { - dev = rte_zmalloc("mlx5_pci_device", sizeof(*dev), 0); - if (!dev) - return -ENOMEM; - dev->pci_dev = pci_dev; - TAILQ_INSERT_HEAD(&devices_list, dev, next); - new_device = true; - } - ret = drivers_probe(dev, pci_drv, pci_dev, user_classes); - if (ret) - goto class_err; - return 0; -class_err: - if (new_device) - dev_release(dev); - return ret; -} - -/** - * DPDK callback to remove one or more drivers for a PCI device. - * - * This function removes all drivers probed for a given PCI device. - * - * @param[in] pci_dev - * Pointer to the PCI device. - * - * @return - * 0 on success, the function cannot fail. - */ -static int -mlx5_pci_remove(struct rte_pci_device *pci_dev) -{ - struct mlx5_pci_device *dev; - int ret; - - dev = pci_to_mlx5_device(pci_dev); - if (!dev) - return -ENODEV; - /* Matching device found, cleanup and unload drivers. */ - ret = drivers_remove(dev, dev->classes_loaded); - if (!ret) - dev_release(dev); - return ret; -} - -static int -mlx5_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) -{ - struct mlx5_pci_driver *driver = NULL; - struct mlx5_pci_driver *temp; - struct mlx5_pci_device *dev; - int ret = -EINVAL; - - dev = pci_to_mlx5_device(pci_dev); - if (!dev) - return -ENODEV; - TAILQ_FOREACH(driver, &drv_list, next) { - if (device_class_enabled(dev, driver->driver_class) && - driver->pci_driver.dma_map) { - ret = driver->pci_driver.dma_map(pci_dev, addr, - iova, len); - if (ret) - goto map_err; - } - } - return ret; -map_err: - TAILQ_FOREACH(temp, &drv_list, next) { - if (temp == driver) - break; - if (device_class_enabled(dev, temp->driver_class) && - temp->pci_driver.dma_map && temp->pci_driver.dma_unmap) - temp->pci_driver.dma_unmap(pci_dev, addr, iova, len); - } - return ret; -} - -static int -mlx5_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) -{ - struct mlx5_pci_driver *driver; - struct mlx5_pci_device *dev; - int local_ret = -EINVAL; - int ret; - - dev = pci_to_mlx5_device(pci_dev); - if (!dev) - return -ENODEV; - ret = 0; - /* There is no unmap error recovery in current implementation. */ - TAILQ_FOREACH_REVERSE(driver, &drv_list, mlx5_pci_bus_drv_head, next) { - if (device_class_enabled(dev, driver->driver_class) && - driver->pci_driver.dma_unmap) { - local_ret = driver->pci_driver.dma_unmap(pci_dev, addr, - iova, len); - if (local_ret && (ret == 0)) - ret = local_ret; - } - } - if (local_ret) - ret = local_ret; - return ret; -} - /* PCI ID table is build dynamically based on registered mlx5 drivers. */ static struct rte_pci_id *mlx5_pci_id_table; -static struct rte_pci_driver mlx5_pci_driver = { - .driver = { - .name = MLX5_PCI_DRIVER_NAME, - }, - .probe = mlx5_pci_probe, - .remove = mlx5_pci_remove, - .dma_map = mlx5_pci_dma_map, - .dma_unmap = mlx5_pci_dma_unmap, -}; - static int pci_id_table_size_get(const struct rte_pci_id *id_table) { @@ -525,7 +95,6 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) } /* Terminate table with empty entry. */ updated_table[i].vendor_id = 0; - mlx5_pci_driver.id_table = updated_table; mlx5_common_pci_driver.id_table = updated_table; mlx5_pci_id_table = updated_table; if (old_table) @@ -533,20 +102,6 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) return 0; } -void -mlx5_pci_driver_register(struct mlx5_pci_driver *driver) -{ - int ret; - - ret = pci_ids_table_update(driver->pci_driver.id_table); - if (ret) - return; - mlx5_pci_driver.drv_flags |= driver->pci_driver.drv_flags; - TAILQ_INSERT_TAIL(&drv_list, driver, next); -} - -/********** New common PCI bus driver ********/ - bool mlx5_dev_is_pci(const struct rte_device *dev) { diff --git a/drivers/common/mlx5/mlx5_common_pci.h b/drivers/common/mlx5/mlx5_common_pci.h deleted file mode 100644 index cb8d2f5f87..0000000000 --- a/drivers/common/mlx5/mlx5_common_pci.h +++ /dev/null @@ -1,78 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright 2020 Mellanox Technologies, Ltd - */ - -#ifndef _MLX5_COMMON_PCI_H_ -#define _MLX5_COMMON_PCI_H_ - -/** - * @file - * - * RTE Mellanox PCI Driver Interface - * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex,compress - * and crypto devices. This layer enables creating such multiple class of - * devices on a single PCI device by allowing to bind multiple class specific - * device driver to attach to mlx5_pci driver. - * - * -------- -------- --------- ------------ ---------- - * | mlx5 | | mlx5 | | mlx5 | | mlx5 | | mlx5 | - * | net | | vdpa | | regex | | compress | | crypto | - * | pmd | | pmd | | pmd | | pmd | | pmd | - * -------- -------- --------- ------------ ---------- - * \ \ | / / - * \ \ | / / - * \ \_--------------_/ / - * \_______________| mlx5 |_______________/ - * | pci common | - * -------------- - * | - * ----------- - * | mlx5 | - * | pci dev | - * ----------- - * - * - mlx5 pci driver binds to mlx5 PCI devices defined by PCI - * ID table of all related mlx5 PCI devices. - * - mlx5 class driver such as net, vdpa, regex PMD defines its - * specific PCI ID table and mlx5 bus driver probes matching - * class drivers. - * - mlx5 pci bus driver is cental place that validates supported - * class combinations. - */ - -#ifdef __cplusplus -extern "C" { -#endif /* __cplusplus */ - -#include -#include - -#include - -void mlx5_common_pci_init(void); - -/** - * A structure describing a mlx5 pci driver. - */ -struct mlx5_pci_driver { - struct rte_pci_driver pci_driver; /**< Inherit core pci driver. */ - uint32_t driver_class; /**< Class of this driver, enum mlx5_class */ - TAILQ_ENTRY(mlx5_pci_driver) next; -}; - -/** - * Register a mlx5_pci device driver. - * - * @param driver - * A pointer to a mlx5_pci_driver structure describing the driver - * to be registered. - */ -__rte_internal -void -mlx5_pci_driver_register(struct mlx5_pci_driver *driver); - -#ifdef __cplusplus -} -#endif /* __cplusplus */ - -#endif /* _MLX5_COMMON_PCI_H_ */ diff --git a/drivers/common/mlx5/mlx5_common_private.h b/drivers/common/mlx5/mlx5_common_private.h index 1096fa85e7..c929840408 100644 --- a/drivers/common/mlx5/mlx5_common_private.h +++ b/drivers/common/mlx5/mlx5_common_private.h @@ -31,6 +31,7 @@ int mlx5_common_dev_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, /* Common PCI bus driver: */ +void mlx5_common_pci_init(void); void mlx5_common_driver_on_register_pci(struct mlx5_class_driver *driver); bool mlx5_dev_pci_match(const struct mlx5_class_driver *drv, const struct rte_device *dev); diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index a47c86e354..da91d167b5 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -138,13 +138,10 @@ INTERNAL { mlx5_nl_vlan_vmwa_create; # WINDOWS_NO_EXPORT mlx5_nl_vlan_vmwa_delete; # WINDOWS_NO_EXPORT - mlx5_pci_driver_register; - mlx5_os_alloc_pd; mlx5_os_dealloc_pd; mlx5_os_dereg_mr; mlx5_os_get_ibv_dev; # WINDOWS_NO_EXPORT - mlx5_os_get_ibv_device; # WINDOWS_NO_EXPORT mlx5_os_reg_mr; mlx5_os_umem_dereg; mlx5_os_umem_reg;