From patchwork Thu Jul 15 16:41:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 95916 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B5C08A0C41; Thu, 15 Jul 2021 18:43:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B56F441331; Thu, 15 Jul 2021 18:42:26 +0200 (CEST) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam07on2078.outbound.protection.outlook.com [40.107.212.78]) by mails.dpdk.org (Postfix) with ESMTP id C83D741316 for ; Thu, 15 Jul 2021 18:42:20 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cWjVE9bTJJ3CbXLlFnn7Cbx8zCYyvewWu+VAbwVYT9kSm9s8K0ATOmtLm0J5RVvam8j4bZlK03E/jT+7ySQNxRCDB4YpJeh3C8WtgpK/Yyp7QIIDSxWXII7q4BBR1ENFCHPv1vy/gnl3oeIX6O1UJrLU2nt5wAiHZYDvOWFSHFHFSePzpwP32KMnfur86XX9g3BXB2qqGfzikT+TcyWBBMxXgKe9xa9dmJjD4vX3aNH3UpgL+ILZJmTP98c7t1NvFUVpvbouQ2LHLPLYkE2cl8NZHn6ZD2SYfeexSIp90G18izFZg49MsEWNdNaFeZC5q6J/rhVH4NL9noQMZg2xwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U16QMk4+K7oH8a2eITgqff0aHsCyPUYnry1A6cimFjg=; b=ONcA4ri7m0bdrlDCFEyJkPnqbe/WhwGwtIHgZH+5kOpLb6kNhE86wUD0xS+Wh65HWptv9KefaaDXfRe3OM68kqL8MO5AkNE93A1P6+cwb7M6YQr3CutLLZ/H0h3RXw4iHpkFJQqXZktqNxfk9czgpSUbZosu6pFmqD9UmRwkfr2m4zW8B+w+HTj3pcsSXq76lvbsQP+BH4VOVssZmu7F1BSbvqEt6hcz78jwqjo+Wj+xhGFHhSilSWCb4c5fz8IVIWJk+mdQ74JMGBm8FjhZu400HRJKhLeeWXUcU36rxL4IIsIQu9912a/CH38XPBGUGvPgbS9NtCrgf7uyNqr7qw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U16QMk4+K7oH8a2eITgqff0aHsCyPUYnry1A6cimFjg=; b=B4XbHXVMbl7aoHjIeoBHpqs94DrE3mJ7QO8hXdBbpjpPl04mjWoJFbKJn1nGOHIaStLEFY3ItmTZjhyvtQMbMNI3aGTj4YiHogL26fr0cJ2CpuziqlbQn0Y7qfXFsdY0QoUrpM28Em0TuBpruzL2ji9zdIAgpdPJt74W762nsid9Lr91VRNeVTNtkCQZT7H/AF1031HuGLVC1rYXjsNZ6VNGWUe8B1gv2NkLoyIhsIKRC1OUfDanQmJDDBzSOcZDbtXzO4D1GE0yUmVUPGZRXMjUYp6XYmKmjDPDsA3BOivjdcYjqRj1ISWqr4v0B4Wd0Wxqb33dzE+RSiIWQ7RTMg== Received: from MW4P221CA0010.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::15) by BN6PR12MB1329.namprd12.prod.outlook.com (2603:10b6:404:19::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.23; Thu, 15 Jul 2021 16:42:19 +0000 Received: from CO1NAM11FT015.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8b:cafe::22) by MW4P221CA0010.outlook.office365.com (2603:10b6:303:8b::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.23 via Frontend Transport; Thu, 15 Jul 2021 16:42:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT015.mail.protection.outlook.com (10.13.175.130) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4331.21 via Frontend Transport; Thu, 15 Jul 2021 16:42:19 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 15 Jul 2021 16:42:16 +0000 From: Shiri Kuzin To: CC: , , , Date: Thu, 15 Jul 2021 19:41:20 +0300 Message-ID: <20210715164126.54073-11-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210715164126.54073-1-shirik@nvidia.com> References: <20210715150817.51485-1-shirik@nvidia.com> <20210715164126.54073-1-shirik@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 56695800-407f-4894-0f4e-08d947af8302 X-MS-TrafficTypeDiagnostic: BN6PR12MB1329: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:193; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5ID9mArZ5wG8TkD59zQ15VcggxK6tx5p/Y2CHofQnWpFpUmHr7GOwgLCWtu7qC7H3iW+PvhYEcuT8McnwYX/Uv5F48jze7f5EpIedikktHs9wD/KOhEqMkLtqcgxODV8rTPWnBgeXRKu6WMUPoQbgd67KUyR22iIBHs0CYyLjTYgTx2nc5d7sB5ABzUseY88JPkT32ZVy3DtIrweLkTHKlUjpapI2EmQHxSjh9S30PPbCEfv1QxoRLRz6/0edJ26AuKtO2zWMbTo9V7VfxhLYKptcOHn7TK8C+D93iKtkvybhsq6vWoEiivjcj49/QY/jKMqmUe3ucxMq1oApTWjqkTPYoBA9FPABaEYuVZwzBoTB2YFNDp8nZVBVPTj1vKmas/v4alRoUXMfc5IKLGCjTeg8huYbQxF1khLpaV72EOT/7XKToJUMqTtAtOwIAJ/E+yzXDfzzWJgLm8uFlvgXjL6HS8oYiLbRfPuhbG/0WK83Q+gC4mV0+pacha5xrGYRF+5SgFxI01UoYml3S2g9wp2WWLh4tRoYEz9kNYJ1VrYzTQcB2KEhEnWRLUZsgpA4/qzTD1Vsy3+mEXRPD7h8W3FGgUiQgGbB/DapVGZkjRhMD+y+0OpRuOB1/l1Gn6S06dkgrlyP0epkKETWHVajENQd4JjDyEQqjJF1c+Q6GTimwyrO+FJmKjr0TgAF0TmSEMHr1mzpwVQ5+35Z+en+40MhPrJgU4TVMoAblEvBy4= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(136003)(346002)(396003)(39860400002)(36840700001)(46966006)(70586007)(186003)(1076003)(83380400001)(47076005)(6666004)(8676002)(6286002)(8936002)(7696005)(70206006)(5660300002)(16526019)(336012)(2906002)(36756003)(36906005)(4326008)(36860700001)(34020700004)(316002)(6916009)(82310400003)(426003)(86362001)(26005)(2616005)(478600001)(7636003)(54906003)(82740400003)(55016002)(356005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2021 16:42:19.1195 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 56695800-407f-4894-0f4e-08d947af8302 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT015.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1329 Subject: [dpdk-dev] [PATCH v8 10/16] crypto/mlx5: add maximum segments devarg X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Suanming Mou The mlx5 HW crypto operations are done by attaching crypto property to a memory region. Once done, every access to the memory via the crypto-enabled memory region will result with in-line encryption or decryption of the data. As a result, the design choice is to provide two types of WQEs. One is UMR WQE which sets the crypto property and the other is rdma write WQE which sends DMA command to copy data from local MR to remote MR. The size of the WQEs will be defined by a new devarg called max_segs_num. This devarg also defines the maximum segments in mbuf chain that will be supported for crypto operations. Signed-off-by: Suanming Mou Signed-off-by: Matan Azrad --- doc/guides/cryptodevs/mlx5.rst | 4 ++++ drivers/crypto/mlx5/mlx5_crypto.c | 33 +++++++++++++++++++++++++++---- drivers/crypto/mlx5/mlx5_crypto.h | 6 ++++++ 3 files changed, 39 insertions(+), 4 deletions(-) diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst index c3632484a5..dd4705b744 100644 --- a/doc/guides/cryptodevs/mlx5.rst +++ b/doc/guides/cryptodevs/mlx5.rst @@ -120,6 +120,10 @@ Driver options The plaintext of the keytag appanded to the AES-XTS keys, default value is 0. +- ``max_segs_num`` parameter [int] + + Maximum number of mbuf chain segments(src or dest), default value is 8. + Supported NICs -------------- diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 0f786ac4ca..fc251f3f4c 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -21,6 +21,7 @@ #define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto #define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5 #define MLX5_CRYPTO_MAX_QPS 1024 +#define MLX5_CRYPTO_MAX_SEGS 56 #define MLX5_CRYPTO_FEATURE_FLAGS \ (RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | \ @@ -496,14 +497,24 @@ mlx5_crypto_args_check_handler(const char *key, const char *val, void *opaque) DRV_LOG(WARNING, "%s: \"%s\" is an invalid integer.", key, val); return -errno; } - if (strcmp(key, "import_kek_id") == 0) + if (strcmp(key, "max_segs_num") == 0) { + if (!tmp || tmp > MLX5_CRYPTO_MAX_SEGS) { + DRV_LOG(WARNING, "Invalid max_segs_num: %d, should" + " be less than %d.", + (uint32_t)tmp, MLX5_CRYPTO_MAX_SEGS); + rte_errno = EINVAL; + return -rte_errno; + } + devarg_prms->max_segs_num = (uint32_t)tmp; + } else if (strcmp(key, "import_kek_id") == 0) { attr->session_import_kek_ptr = (uint32_t)tmp; - else if (strcmp(key, "credential_id") == 0) + } else if (strcmp(key, "credential_id") == 0) { attr->credential_pointer = (uint32_t)tmp; - else if (strcmp(key, "keytag") == 0) + } else if (strcmp(key, "keytag") == 0) { devarg_prms->keytag = tmp; - else + } else { DRV_LOG(WARNING, "Invalid key %s.", key); + } return 0; } @@ -518,6 +529,7 @@ mlx5_crypto_parse_devargs(struct rte_devargs *devargs, attr->credential_pointer = 0; attr->session_import_kek_ptr = 0; devarg_prms->keytag = 0; + devarg_prms->max_segs_num = 8; if (devargs == NULL) { DRV_LOG(ERR, "No login devargs in order to enable crypto operations in the device."); @@ -614,6 +626,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, .max_nb_queue_pairs = RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS, }; + uint16_t rdmw_wqe_size; int ret; RTE_SET_USED(pci_drv); @@ -692,6 +705,18 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr; priv->mr_scache.dereg_mr_cb = mlx5_common_verbs_dereg_mr; priv->keytag = rte_cpu_to_be_64(devarg_prms.keytag); + priv->max_segs_num = devarg_prms.max_segs_num; + priv->umr_wqe_size = sizeof(struct mlx5_wqe_umr_bsf_seg) + + sizeof(struct mlx5_umr_wqe) + + RTE_ALIGN(priv->max_segs_num, 4) * + sizeof(struct mlx5_wqe_dseg); + rdmw_wqe_size = sizeof(struct mlx5_rdma_write_wqe) + + sizeof(struct mlx5_wqe_dseg) * + (priv->max_segs_num <= 2 ? 2 : 2 + + RTE_ALIGN(priv->max_segs_num - 2, 4)); + priv->wqe_set_size = priv->umr_wqe_size + rdmw_wqe_size; + priv->umr_wqe_stride = priv->umr_wqe_size / MLX5_SEND_WQE_BB; + priv->max_rdmar_ds = rdmw_wqe_size / sizeof(struct mlx5_wqe_dseg); /* Register callback function for global shared MR cache management. */ if (TAILQ_EMPTY(&mlx5_crypto_priv_list)) rte_mem_event_callback_register("MLX5_MEM_EVENT_CB", diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index d374d3c4e9..7a386fe76a 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -25,12 +25,17 @@ struct mlx5_crypto_priv { struct rte_cryptodev *crypto_dev; void *uar; /* User Access Region. */ uint32_t pdn; /* Protection Domain number. */ + uint32_t max_segs_num; /* Maximum supported data segs. */ struct ibv_pd *pd; struct mlx5_hlist *dek_hlist; /* Dek hash list. */ struct rte_cryptodev_config dev_config; struct mlx5_mr_share_cache mr_scache; /* Global shared MR cache. */ struct mlx5_devx_obj *login_obj; uint64_t keytag; + uint16_t wqe_set_size; + uint16_t umr_wqe_size; + uint16_t umr_wqe_stride; + uint16_t max_rdmar_ds; }; struct mlx5_crypto_qp { @@ -54,6 +59,7 @@ struct mlx5_crypto_devarg_params { bool login_devarg; struct mlx5_devx_crypto_login_attr login_attr; uint64_t keytag; + uint32_t max_segs_num; }; int