From patchwork Thu Jul 15 16:41:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95907 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 73DF9A0C41; Thu, 15 Jul 2021 18:42:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 61FDD41264; Thu, 15 Jul 2021 18:42:07 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2085.outbound.protection.outlook.com [40.107.223.85]) by mails.dpdk.org (Postfix) with ESMTP id E5BA44125B for ; Thu, 15 Jul 2021 18:42:04 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=l8q6IUiHKGbn1OUL2LwU//RUwCqnv9/uRR6DYUo8u6POqM8wzUh/IjSXQtouE4eW305iD/VANW3fxfYzOh8XoOZwM+yoE57NTsAXwUCwFaUj6CzrDcNiwP7KQ/PjoP7npk8FEpP9V+UTewFlQSyhQ9kLINYscEXk1eNs3M8LXymRqFUH8PjpG3l8DFBKCGAhiFHF9uLX4mFsJgBnzV3UPiMUb2yRULF777jcmPbW3IaCbU6sueeBflTn91pRcz6pIhQbqDOYK28JiAgOtdrtV0Q9xqFqnlpvus3tOWBpnjbprH25BEUaqfNgk4rK59IPyz+QXUGZFaexMST7k9TkHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+Nr6HFysRWinOgWMhz7lmeSYtCS+mxafW8cCrRlE8Qs=; b=cTuU4cYaZXvXZCklKVLNnNhYtMjnUi0eYgqYQ6rE3zQds6msV1N8/TjUbFNvgRlxNUp68gVgYYNVsSus1k4Qf03b5kTxX3P76sUGc/xWajXf9fapu5oDYKUIHyMH/2Caw5XW7q7wm5ghazCzMFf4G4PwmQgNTqkLQhqI1ZbNsZr+1X8zytajuRU0w6h3nAOt3NlWwe9OwWxa9AoNfopT9VVncM61tEKoFv0hnuGurLOVB84afjXvLMoqUZLUor2wyGGeD5I6SAaZ6gezzE327ht0ooFKhZv0LSp7k3y6FJ3DF/nNR3PEOv+c7W4pcEHZKoK2hlY2HYJSOfk4eDY+Pg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+Nr6HFysRWinOgWMhz7lmeSYtCS+mxafW8cCrRlE8Qs=; b=r9/drh+MkTiIDvfIpwDCK6PT7MWae9fbxXHbmEqoAOQs3yt5GAmFmiXRLLbog3fWLNF/IePcOIxJ0O4nqQQINT/VHfg1EwTcD3W7JYGK4QTKq+C4jkS5RuIkILkabuUe7LN5+JmbfUEdjIf2FJgweorJ5BRYh4DAWssJ1erZrOUZ3y+/Eyq2oCMQF9fg3+0MATWZAoaYjbvhG/CD50gN8YW9dZhqTD5kuF+LYWnJdcTHH6LkLX7m8g1otI8Ilp4sL/P150TORYXwSPeBjnmF1eyV7BS4nvz9DiokgBETkPIa3lcqzm5vY9XG3hYMiJaB8DBM9TwX+MNFp3OjqK6EVA== Received: from CO2PR04CA0095.namprd04.prod.outlook.com (2603:10b6:104:6::21) by BYAPR12MB3109.namprd12.prod.outlook.com (2603:10b6:a03:db::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.23; Thu, 15 Jul 2021 16:42:02 +0000 Received: from CO1NAM11FT024.eop-nam11.prod.protection.outlook.com (2603:10b6:104:6:cafe::a9) by CO2PR04CA0095.outlook.office365.com (2603:10b6:104:6::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.23 via Frontend Transport; Thu, 15 Jul 2021 16:42:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT024.mail.protection.outlook.com (10.13.174.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4331.21 via Frontend Transport; Thu, 15 Jul 2021 16:42:02 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 15 Jul 2021 16:42:00 +0000 From: Shiri Kuzin To: CC: , , , Date: Thu, 15 Jul 2021 19:41:11 +0300 Message-ID: <20210715164126.54073-2-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210715164126.54073-1-shirik@nvidia.com> References: <20210715150817.51485-1-shirik@nvidia.com> <20210715164126.54073-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c368688c-c126-4237-d6f5-08d947af791b X-MS-TrafficTypeDiagnostic: BYAPR12MB3109: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:121; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1YosM6RQI3K72xw/saHS4geTfbgW+inus+EmlySnwvHk7APH/5vUpAkF1zL2i1nn9PHmBdDF4P9fsy1CoETPOlwV8w5W93RgvWLW9NSEUfIKURLcvXKpExoeL1XVmhDJGxrfuCJpgwHxyZSy+8fdy/UNblSZfnbMkvFn2bPEU4O/vsu0Gn/uiGXApsadBemNBLhF3elgrSKgVU39yuStQAvG1C536vGP6QU1pgbX7QTdl9a5Y5gOpzL6FSk5xKOnUUlGg8bVmF+DGjBerc7hbpLOiAu2YdrrZpA7+yvbOX1QBDci34dGcqzw71dQju183SszhNDVMKj+I/KcIJwpuxG5Lfrjehd14XMjc4OD7Gd+1ORW5d70BXTJyxzandD2zyewnuQdvHAPCBUQw4SdjaE4j0RVk7V+/yF/zzwDAGLIdzorhV0x20jtYMQYFFs2wYjKQNuqI2nYDUwFLeunFpW2/DY/Pc+Xpmfll0pTRtk91iw1BEoAMDdl7TCyOvZshOxeESUYFWjMVWJMYxo9C/rY5oJ7wkTqnDOqilYNkVg6CQo0dUYZmJhxTbY1PdqCZn069PRJw5E30eFgMe4tyjcQZSVrtcLqz8e3COlesFY6piTmRchUHr5eRnT2RwIZmBK2LMPHKGsDqzjj6z4KD0rnimxdkH/oSl1fT/EYwkv5vr9ZT0i+GU52cpg1aAL0aCU2jieupjVyxRHuOi8PCpuZ3dQbUxmKDeP85W0v0Hw= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(376002)(39860400002)(46966006)(36840700001)(5660300002)(86362001)(478600001)(4326008)(36756003)(70586007)(2616005)(83380400001)(7696005)(70206006)(30864003)(36860700001)(426003)(2906002)(16526019)(36906005)(26005)(186003)(8676002)(82310400003)(6916009)(316002)(34020700004)(8936002)(336012)(7636003)(54906003)(47076005)(55016002)(82740400003)(1076003)(6286002)(6666004)(356005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2021 16:42:02.5283 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c368688c-c126-4237-d6f5-08d947af791b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT024.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3109 Subject: [dpdk-dev] [PATCH v8 01/16] drivers: introduce mlx5 crypto PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a new PMD for Mellanox devices- crypto PMD. The crypto PMD will be supported starting Nvidia ConnectX6 and BlueField2. The crypto PMD will add the support of encryption and decryption using the AES-XTS symmetric algorithm. The crypto PMD requires rdma-core and uses mlx5 DevX. This patch adds the PCI probing, basic functions, build files and log utility. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- MAINTAINERS | 4 + doc/guides/cryptodevs/features/mlx5.ini | 27 +++ doc/guides/cryptodevs/index.rst | 1 + doc/guides/cryptodevs/mlx5.rst | 63 ++++++ doc/guides/rel_notes/release_21_08.rst | 5 + drivers/common/mlx5/mlx5_common.h | 1 + drivers/common/mlx5/mlx5_common_pci.c | 14 ++ drivers/common/mlx5/mlx5_common_pci.h | 21 +- drivers/crypto/meson.build | 1 + drivers/crypto/mlx5/meson.build | 26 +++ drivers/crypto/mlx5/mlx5_crypto.c | 275 ++++++++++++++++++++++++ drivers/crypto/mlx5/mlx5_crypto_utils.h | 19 ++ drivers/crypto/mlx5/version.map | 3 + 13 files changed, 450 insertions(+), 10 deletions(-) create mode 100644 doc/guides/cryptodevs/features/mlx5.ini create mode 100644 doc/guides/cryptodevs/mlx5.rst create mode 100644 drivers/crypto/mlx5/meson.build create mode 100644 drivers/crypto/mlx5/mlx5_crypto.c create mode 100644 drivers/crypto/mlx5/mlx5_crypto_utils.h create mode 100644 drivers/crypto/mlx5/version.map diff --git a/MAINTAINERS b/MAINTAINERS index af2a91d7c4..ba2cb4ef42 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1102,6 +1102,10 @@ F: drivers/crypto/octeontx2/ F: doc/guides/cryptodevs/octeontx2.rst F: doc/guides/cryptodevs/features/octeontx2.ini +Mellanox mlx5 +M: Matan Azrad +F: drivers/crypto/mlx5/ + Null Crypto M: Declan Doherty F: drivers/crypto/null/ diff --git a/doc/guides/cryptodevs/features/mlx5.ini b/doc/guides/cryptodevs/features/mlx5.ini new file mode 100644 index 0000000000..ceadd967b6 --- /dev/null +++ b/doc/guides/cryptodevs/features/mlx5.ini @@ -0,0 +1,27 @@ +; +; Features of a mlx5 crypto driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +HW Accelerated = Y + +; +; Supported crypto algorithms of a mlx5 crypto driver. +; +[Cipher] + +; +; Supported authentication algorithms of a mlx5 crypto driver. +; +[Auth] + +; +; Supported AEAD algorithms of a mlx5 crypto driver. +; +[AEAD] + +; +; Supported Asymmetric algorithms of a mlx5 crypto driver. +; +[Asymmetric] diff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst index 067e7d784e..0f981c77b5 100644 --- a/doc/guides/cryptodevs/index.rst +++ b/doc/guides/cryptodevs/index.rst @@ -23,6 +23,7 @@ Crypto Device Drivers octeontx octeontx2 openssl + mlx5 mvsam nitrox null diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst new file mode 100644 index 0000000000..05a0a449e2 --- /dev/null +++ b/doc/guides/cryptodevs/mlx5.rst @@ -0,0 +1,63 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright (c) 2021 NVIDIA Corporation & Affiliates + +.. include:: + +MLX5 Crypto Driver +================== + +The MLX5 crypto driver library +(**librte_crypto_mlx5**) provides support for **Mellanox ConnectX-6** +family adapters. + +Overview +-------- + +The device can provide disk encryption services, allowing data encryption +and decryption towards a disk. Having all encryption/decryption +operations done in a single device can reduce cost and overheads of the related +FIPS certification, as ConnectX-6 is FIPS 140-2 level-2 ready. +The encryption cipher is AES-XTS of 256/512 bit key size. + +MKEY is a memory region object in the hardware, that holds address translation information and +attributes per memory area. Its ID must be tied to addresses provided to the hardware. +The encryption operations are performed with MKEY read/write transactions, when +the MKEY is configured to perform crypto operations. + +The encryption does not require text to be aligned to the AES block size (128b). + +The PMD uses libibverbs and libmlx5 to access the device firmware or to +access the hardware components directly. +There are different levels of objects and bypassing abilities. +To get the best performances: + +- Verbs is a complete high-level generic API. +- Direct Verbs is a device-specific API. +- DevX allows to access firmware objects. + +Enabling librte_crypto_mlx5 causes DPDK applications to be linked against +libibverbs. + + +Driver options +-------------- + +- ``class`` parameter [string] + + Select the class of the driver that should probe the device. + `crypto` for the mlx5 crypto driver. + + +Supported NICs +-------------- + +* Mellanox\ |reg| ConnectX\ |reg|-6 200G MCX654106A-HCAT (2x200G) + +Prerequisites +------------- + +- Mellanox OFED version: **5.3** + see :doc:`../../nics/mlx5` guide for more Mellanox OFED details. + +- Compilation can be done also with rdma-core v15+. + see :doc:`../../nics/mlx5` guide for more rdma-core details. diff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst index 7d289e07e3..2bf4ce7a73 100644 --- a/doc/guides/rel_notes/release_21_08.rst +++ b/doc/guides/rel_notes/release_21_08.rst @@ -125,6 +125,11 @@ New Features The experimental PMD power management API now supports managing multiple Ethernet Rx queues per lcore. +* **Added support for Nvidia crypto device driver.** + + * Added mlx5 crypto driver to support AES-XTS cipher operations. + The first device to support it is ConnectX-6. + Removed Items ------------- diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 7fb7d40b38..962179a5a5 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -216,6 +216,7 @@ enum mlx5_class { MLX5_CLASS_VDPA = RTE_BIT64(1), MLX5_CLASS_REGEX = RTE_BIT64(2), MLX5_CLASS_COMPRESS = RTE_BIT64(3), + MLX5_CLASS_CRYPTO = RTE_BIT64(4), }; #define MLX5_DBR_SIZE RTE_CACHE_LINE_SIZE diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 34747c4e07..5547e62d6b 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -31,6 +31,7 @@ static const struct { { .name = "net", .driver_class = MLX5_CLASS_NET }, { .name = "regex", .driver_class = MLX5_CLASS_REGEX }, { .name = "compress", .driver_class = MLX5_CLASS_COMPRESS }, + { .name = "crypto", .driver_class = MLX5_CLASS_CRYPTO }, }; static const unsigned int mlx5_class_combinations[] = { @@ -38,13 +39,26 @@ static const unsigned int mlx5_class_combinations[] = { MLX5_CLASS_VDPA, MLX5_CLASS_REGEX, MLX5_CLASS_COMPRESS, + MLX5_CLASS_CRYPTO, MLX5_CLASS_NET | MLX5_CLASS_REGEX, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX, MLX5_CLASS_NET | MLX5_CLASS_COMPRESS, MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS, MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, + MLX5_CLASS_NET | MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_CRYPTO, + MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, + MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_NET | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, + MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | + MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | + MLX5_CLASS_CRYPTO, /* New class combination should be added here. */ }; diff --git a/drivers/common/mlx5/mlx5_common_pci.h b/drivers/common/mlx5/mlx5_common_pci.h index de89bb98bc..cb8d2f5f87 100644 --- a/drivers/common/mlx5/mlx5_common_pci.h +++ b/drivers/common/mlx5/mlx5_common_pci.h @@ -9,17 +9,18 @@ * @file * * RTE Mellanox PCI Driver Interface - * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex and - * compress devices. This layer enables creating such multiple class of devices - * on a single PCI device by allowing to bind multiple class specific device - * driver to attach to mlx5_pci driver. + * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex,compress + * and crypto devices. This layer enables creating such multiple class of + * devices on a single PCI device by allowing to bind multiple class specific + * device driver to attach to mlx5_pci driver. * - * ----------- ------------ ------------- ---------------- - * | mlx5 | | mlx5 | | mlx5 | | mlx5 | - * | net pmd | | vdpa pmd | | regex pmd | | compress pmd | - * ----------- ------------ ------------- ---------------- - * \ \ / / - * \ \ / / + * -------- -------- --------- ------------ ---------- + * | mlx5 | | mlx5 | | mlx5 | | mlx5 | | mlx5 | + * | net | | vdpa | | regex | | compress | | crypto | + * | pmd | | pmd | | pmd | | pmd | | pmd | + * -------- -------- --------- ------------ ---------- + * \ \ | / / + * \ \ | / / * \ \_--------------_/ / * \_______________| mlx5 |_______________/ * | pci common | diff --git a/drivers/crypto/meson.build b/drivers/crypto/meson.build index cb865aa0d5..ea239f4c56 100644 --- a/drivers/crypto/meson.build +++ b/drivers/crypto/meson.build @@ -16,6 +16,7 @@ drivers = [ 'dpaa_sec', 'dpaa2_sec', 'kasumi', + 'mlx5', 'mvsam', 'nitrox', 'null', diff --git a/drivers/crypto/mlx5/meson.build b/drivers/crypto/mlx5/meson.build new file mode 100644 index 0000000000..6fd70bc477 --- /dev/null +++ b/drivers/crypto/mlx5/meson.build @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2021 NVIDIA Corporation & Affiliates + +if not is_linux + build = false + reason = 'only supported on Linux' + subdir_done() +endif + +fmt_name = 'mlx5_crypto' +deps += ['common_mlx5', 'eal', 'cryptodev'] +sources = files( + 'mlx5_crypto.c', +) +cflags_options = [ + '-std=c11', + '-Wno-strict-prototypes', + '-D_BSD_SOURCE', + '-D_DEFAULT_SOURCE', + '-D_XOPEN_SOURCE=600', +] +foreach option:cflags_options + if cc.has_argument(option) + cflags += option + endif +endforeach diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c new file mode 100644 index 0000000000..fbe3c21aae --- /dev/null +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -0,0 +1,275 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "mlx5_crypto_utils.h" + +#define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto +#define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5 + +#define MLX5_CRYPTO_FEATURE_FLAGS \ + RTE_CRYPTODEV_FF_HW_ACCELERATED + +struct mlx5_crypto_priv { + TAILQ_ENTRY(mlx5_crypto_priv) next; + struct ibv_context *ctx; /* Device context. */ + struct rte_pci_device *pci_dev; + struct rte_cryptodev *crypto_dev; + void *uar; /* User Access Region. */ + uint32_t pdn; /* Protection Domain number. */ + struct ibv_pd *pd; +}; + +TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list = + TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list); +static pthread_mutex_t priv_list_lock = PTHREAD_MUTEX_INITIALIZER; + +int mlx5_crypto_logtype; + +uint8_t mlx5_crypto_driver_id; + +static const char mlx5_crypto_drv_name[] = RTE_STR(MLX5_CRYPTO_DRIVER_NAME); + +static const struct rte_driver mlx5_drv = { + .name = mlx5_crypto_drv_name, + .alias = mlx5_crypto_drv_name +}; + +static struct cryptodev_driver mlx5_cryptodev_driver; + +static struct rte_cryptodev_ops mlx5_crypto_ops = { + .dev_configure = NULL, + .dev_start = NULL, + .dev_stop = NULL, + .dev_close = NULL, + .dev_infos_get = NULL, + .stats_get = NULL, + .stats_reset = NULL, + .queue_pair_setup = NULL, + .queue_pair_release = NULL, + .sym_session_get_size = NULL, + .sym_session_configure = NULL, + .sym_session_clear = NULL, + .sym_get_raw_dp_ctx_size = NULL, + .sym_configure_raw_dp_ctx = NULL, +}; + +static void +mlx5_crypto_hw_global_release(struct mlx5_crypto_priv *priv) +{ + if (priv->pd != NULL) { + claim_zero(mlx5_glue->dealloc_pd(priv->pd)); + priv->pd = NULL; + } + if (priv->uar != NULL) { + mlx5_glue->devx_free_uar(priv->uar); + priv->uar = NULL; + } +} + +static int +mlx5_crypto_pd_create(struct mlx5_crypto_priv *priv) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT + struct mlx5dv_obj obj; + struct mlx5dv_pd pd_info; + int ret; + + priv->pd = mlx5_glue->alloc_pd(priv->ctx); + if (priv->pd == NULL) { + DRV_LOG(ERR, "Failed to allocate PD."); + return errno ? -errno : -ENOMEM; + } + obj.pd.in = priv->pd; + obj.pd.out = &pd_info; + ret = mlx5_glue->dv_init_obj(&obj, MLX5DV_OBJ_PD); + if (ret != 0) { + DRV_LOG(ERR, "Fail to get PD object info."); + mlx5_glue->dealloc_pd(priv->pd); + priv->pd = NULL; + return -errno; + } + priv->pdn = pd_info.pdn; + return 0; +#else + (void)priv; + DRV_LOG(ERR, "Cannot get pdn - no DV support."); + return -ENOTSUP; +#endif /* HAVE_IBV_FLOW_DV_SUPPORT */ +} + +static int +mlx5_crypto_hw_global_prepare(struct mlx5_crypto_priv *priv) +{ + if (mlx5_crypto_pd_create(priv) != 0) + return -1; + priv->uar = mlx5_devx_alloc_uar(priv->ctx, -1); + if (priv->uar == NULL || mlx5_os_get_devx_uar_reg_addr(priv->uar) == + NULL) { + rte_errno = errno; + claim_zero(mlx5_glue->dealloc_pd(priv->pd)); + DRV_LOG(ERR, "Failed to allocate UAR."); + return -1; + } + return 0; +} + +/** + * DPDK callback to register a PCI device. + * + * This function spawns crypto device out of a given PCI device. + * + * @param[in] pci_drv + * PCI driver structure (mlx5_crypto_driver). + * @param[in] pci_dev + * PCI device information. + * + * @return + * 0 on success, 1 to skip this driver, a negative errno value otherwise + * and rte_errno is set. + */ +static int +mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct ibv_device *ibv; + struct rte_cryptodev *crypto_dev; + struct ibv_context *ctx; + struct mlx5_crypto_priv *priv; + struct mlx5_hca_attr attr = { 0 }; + struct rte_cryptodev_pmd_init_params init_params = { + .name = "", + .private_data_size = sizeof(struct mlx5_crypto_priv), + .socket_id = pci_dev->device.numa_node, + .max_nb_queue_pairs = + RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS, + }; + RTE_SET_USED(pci_drv); + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + DRV_LOG(ERR, "Non-primary process type is not supported."); + rte_errno = ENOTSUP; + return -rte_errno; + } + ibv = mlx5_os_get_ibv_device(&pci_dev->addr); + if (ibv == NULL) { + DRV_LOG(ERR, "No matching IB device for PCI slot " + PCI_PRI_FMT ".", pci_dev->addr.domain, + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + return -rte_errno; + } + DRV_LOG(INFO, "PCI information matches for device \"%s\".", ibv->name); + ctx = mlx5_glue->dv_open_device(ibv); + if (ctx == NULL) { + DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); + rte_errno = ENODEV; + return -rte_errno; + } + if (mlx5_devx_cmd_query_hca_attr(ctx, &attr) != 0 || + attr.crypto == 0 || attr.aes_xts == 0) { + DRV_LOG(ERR, "Not enough capabilities to support crypto " + "operations, maybe old FW/OFED version?"); + claim_zero(mlx5_glue->close_device(ctx)); + rte_errno = ENOTSUP; + return -ENOTSUP; + } + crypto_dev = rte_cryptodev_pmd_create(ibv->name, &pci_dev->device, + &init_params); + if (crypto_dev == NULL) { + DRV_LOG(ERR, "Failed to create device \"%s\".", ibv->name); + claim_zero(mlx5_glue->close_device(ctx)); + return -ENODEV; + } + DRV_LOG(INFO, + "Crypto device %s was created successfully.", ibv->name); + crypto_dev->dev_ops = &mlx5_crypto_ops; + crypto_dev->dequeue_burst = NULL; + crypto_dev->enqueue_burst = NULL; + crypto_dev->feature_flags = MLX5_CRYPTO_FEATURE_FLAGS; + crypto_dev->driver_id = mlx5_crypto_driver_id; + priv = crypto_dev->data->dev_private; + priv->ctx = ctx; + priv->pci_dev = pci_dev; + priv->crypto_dev = crypto_dev; + if (mlx5_crypto_hw_global_prepare(priv) != 0) { + rte_cryptodev_pmd_destroy(priv->crypto_dev); + claim_zero(mlx5_glue->close_device(priv->ctx)); + return -1; + } + pthread_mutex_lock(&priv_list_lock); + TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next); + pthread_mutex_unlock(&priv_list_lock); + return 0; +} + +static int +mlx5_crypto_pci_remove(struct rte_pci_device *pdev) +{ + struct mlx5_crypto_priv *priv = NULL; + + pthread_mutex_lock(&priv_list_lock); + TAILQ_FOREACH(priv, &mlx5_crypto_priv_list, next) + if (rte_pci_addr_cmp(&priv->pci_dev->addr, &pdev->addr) != 0) + break; + if (priv) + TAILQ_REMOVE(&mlx5_crypto_priv_list, priv, next); + pthread_mutex_unlock(&priv_list_lock); + if (priv) { + mlx5_crypto_hw_global_release(priv); + rte_cryptodev_pmd_destroy(priv->crypto_dev); + claim_zero(mlx5_glue->close_device(priv->ctx)); + } + return 0; +} + +static const struct rte_pci_id mlx5_crypto_pci_id_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6) + }, + { + .vendor_id = 0 + } + }; + +static struct mlx5_pci_driver mlx5_crypto_driver = { + .driver_class = MLX5_CLASS_CRYPTO, + .pci_driver = { + .driver = { + .name = RTE_STR(MLX5_CRYPTO_DRIVER_NAME), + }, + .id_table = mlx5_crypto_pci_id_map, + .probe = mlx5_crypto_pci_probe, + .remove = mlx5_crypto_pci_remove, + .drv_flags = 0, + }, +}; + +RTE_INIT(rte_mlx5_crypto_init) +{ + mlx5_common_init(); + if (mlx5_glue != NULL) + mlx5_pci_driver_register(&mlx5_crypto_driver); +} + +RTE_PMD_REGISTER_CRYPTO_DRIVER(mlx5_cryptodev_driver, mlx5_drv, + mlx5_crypto_driver_id); + +RTE_LOG_REGISTER_DEFAULT(mlx5_crypto_logtype, NOTICE) +RTE_PMD_EXPORT_NAME(MLX5_CRYPTO_DRIVER_NAME, __COUNTER__); +RTE_PMD_REGISTER_PCI_TABLE(MLX5_CRYPTO_DRIVER_NAME, mlx5_crypto_pci_id_map); +RTE_PMD_REGISTER_KMOD_DEP(MLX5_CRYPTO_DRIVER_NAME, "* ib_uverbs & mlx5_core & mlx5_ib"); diff --git a/drivers/crypto/mlx5/mlx5_crypto_utils.h b/drivers/crypto/mlx5/mlx5_crypto_utils.h new file mode 100644 index 0000000000..b6c60ca782 --- /dev/null +++ b/drivers/crypto/mlx5/mlx5_crypto_utils.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#ifndef RTE_PMD_MLX5_CRYPTO_UTILS_H_ +#define RTE_PMD_MLX5_CRYPTO_UTILS_H_ + +#include + +extern int mlx5_crypto_logtype; + +#define MLX5_CRYPTO_LOG_PREFIX "mlx5_crypto" +/* Generic printf()-like logging macro with automatic line feed. */ +#define DRV_LOG(level, ...) \ + PMD_DRV_LOG_(level, mlx5_crypto_logtype, MLX5_CRYPTO_LOG_PREFIX, \ + __VA_ARGS__ PMD_DRV_LOG_STRIP PMD_DRV_LOG_OPAREN, \ + PMD_DRV_LOG_CPAREN) + +#endif /* RTE_PMD_MLX5_CRYPTO_UTILS_H_ */ diff --git a/drivers/crypto/mlx5/version.map b/drivers/crypto/mlx5/version.map new file mode 100644 index 0000000000..4a76d1d52d --- /dev/null +++ b/drivers/crypto/mlx5/version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +};