From patchwork Tue Dec 20 14:32:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121075 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5AFD6A0545; Tue, 20 Dec 2022 15:32:49 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1D8E742D15; Tue, 20 Dec 2022 15:32:44 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A72DB42D15 for ; Tue, 20 Dec 2022 15:32:42 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEObIV019239 for ; Tue, 20 Dec 2022 06:32:42 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Lc70aCnu8cDpLsJK/jYEJQ2CGJxOcyjfoCNcW27jjtM=; b=EyW0fxKlntuE6fMIWgRPLNlvR/NXKllB//4RwonL1y7MhwmvSxa/dUhU5rVEyCG9oT1o hm6FJLH1+CxWNfEZW0Hk++t8YLtnmpVT8ywMp7fJGEhjjiD78WiZ8CqxjE/+ITWZ8Lon ER3SCmdJqlWGv4ueUzW5rCvTCiMkA4O8n44CqrFW08SsQtrS1L2uo7FNn4lt0PN+V5I0 IgRZFDXyaEaXRPeH/72RitWNgKkmVFlAbfSngxiqPFa5c+OtR6Vg05KUR/YWaZGALmyo 4l7AP+UzDZjJ1Xgyb+/Afr4W0gpbmFn/N+rNrktvSPFNf9ewNqqgOeq3RebotbuCw3yG 2A== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb52-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:32:41 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:32:39 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:32:39 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 23E983F704C; Tue, 20 Dec 2022 06:32:36 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , "Aakash Sasidharan" , Subject: [PATCH 01/17] common/cnxk: perform LF fini ops only when allocated Date: Tue, 20 Dec 2022 20:02:16 +0530 Message-ID: <20221220143232.2519650-2-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: mABC-pHsXzbQ0QNbTwVZuL9Vr4E_cN_3 X-Proofpoint-ORIG-GUID: mABC-pHsXzbQ0QNbTwVZuL9Vr4E_cN_3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph If LFs are not attached, LF detach and free ops should not be attempted. Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_cpt.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/common/cnxk/roc_cpt.c b/drivers/common/cnxk/roc_cpt.c index fb97ec89b2..bf216b5545 100644 --- a/drivers/common/cnxk/roc_cpt.c +++ b/drivers/common/cnxk/roc_cpt.c @@ -861,6 +861,9 @@ roc_cpt_dev_clear(struct roc_cpt *roc_cpt) if (cpt == NULL) return; + if (roc_cpt->nb_lf == 0) + return; + for (i = 0; i < roc_cpt->nb_lf; i++) cpt->lf_msix_off[i] = 0; From patchwork Tue Dec 20 14:32:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121076 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AAC7DA0545; Tue, 20 Dec 2022 15:32:55 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F2B1C42D1A; Tue, 20 Dec 2022 15:32:46 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 49B3542D1A for ; Tue, 20 Dec 2022 15:32:45 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEOQ0t018991 for ; Tue, 20 Dec 2022 06:32:44 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=sjAYvoU/vMhUCh3BNLR/HWH5hRO73VsHcWTIGdzI9f4=; b=cRWjQ99p0mUC0HcwXofT0CN15+zYCktLg2TIgNuZMapppMykHQ/o3X3EA4PS0P0iPkO+ /eUCQxTniWDfj9TGo1UqAiScBJLX9ua70EpJSIUFcIw9LE11OlZ2cIyI5dAaIUCdaEyJ Fc174CkAIXm5wx/ebFWCq0DRzaZMoCHL5MNlKZDUl89wkRmICMAW53oq6xFSmX6ClFyC uVh1rM9zD/pugqatsg2yJqwkH2vxDkLdC2yl2Wwuvx2XO5fl/QcUhRDE/e+GI+WvglHD tvqDkkYayBh1dhripWLgpJoBwyWbHcgZ87Bz5kaCAMKn+0wpwWzYGZz//KuSN+p/VSVQ cg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb54-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:32:44 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:32:42 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:32:42 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id CFBFC3F7063; Tue, 20 Dec 2022 06:32:39 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Aakash Sasidharan , Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , Subject: [PATCH 02/17] common/cnxk: generate opad and ipad in driver Date: Tue, 20 Dec 2022 20:02:17 +0530 Message-ID: <20221220143232.2519650-3-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: JPAxUSOVnReXw8U5yrvq2wm2INfHDALD X-Proofpoint-ORIG-GUID: JPAxUSOVnReXw8U5yrvq2wm2INfHDALD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Aakash Sasidharan Generate opad and ipad in driver for SHA based crypto algos Signed-off-by: Aakash Sasidharan --- drivers/common/cnxk/cnxk_security.c | 4 +-- drivers/common/cnxk/roc_hash.c | 12 ++++--- drivers/common/cnxk/roc_hash.h | 2 +- drivers/common/cnxk/roc_se.c | 52 +++++++++++++++++++++++++++-- 4 files changed, 60 insertions(+), 10 deletions(-) diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c index 85105472a1..5034c76938 100644 --- a/drivers/common/cnxk/cnxk_security.c +++ b/drivers/common/cnxk/cnxk_security.c @@ -37,8 +37,8 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, roc_hash_sha1_gen(ipad, (uint32_t *)&hmac_opad_ipad[24]); break; case RTE_CRYPTO_AUTH_SHA256_HMAC: - roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0]); - roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64]); + roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0], 256); + roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64], 256); break; case RTE_CRYPTO_AUTH_SHA384_HMAC: roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 384); diff --git a/drivers/common/cnxk/roc_hash.c b/drivers/common/cnxk/roc_hash.c index 1b9030e693..8c451440b1 100644 --- a/drivers/common/cnxk/roc_hash.c +++ b/drivers/common/cnxk/roc_hash.c @@ -232,7 +232,7 @@ roc_hash_sha1_gen(uint8_t *msg, uint32_t *hash) * Based on implementation from RFC 3174 */ void -roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash) +roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash, int hash_size) { const uint32_t _K[] = { /* Round Constants defined in SHA-256 */ @@ -250,13 +250,17 @@ roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash) 0x682e6ff3, 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2}; - const uint32_t _H[] = {/* Initial Hash constants defined in SHA-256 */ - 0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, - 0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19}; + const uint32_t _H224[] = {/* Initial Hash constants defined in SHA-224 */ + 0xc1059ed8, 0x367cd507, 0x3070dd17, 0xf70e5939, + 0xffc00b31, 0x68581511, 0x64f98fa7, 0xbefa4fa4}; + const uint32_t _H256[] = {/* Initial Hash constants defined in SHA-256 */ + 0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, + 0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19}; int i; uint32_t temp[4], S0, S1; /* Temporary word value */ uint32_t W[64]; /* Word sequence */ uint32_t A, B, C, D, E, F, G, H; /* Word buffers */ + const uint32_t *_H = (hash_size == 224) ? _H224 : _H256; /* Initialize the first 16 words in the array W */ memcpy(&W[0], msg, 16 * sizeof(W[0])); diff --git a/drivers/common/cnxk/roc_hash.h b/drivers/common/cnxk/roc_hash.h index 8940faa6eb..c3ddb9407b 100644 --- a/drivers/common/cnxk/roc_hash.h +++ b/drivers/common/cnxk/roc_hash.h @@ -11,7 +11,7 @@ */ void __roc_api roc_hash_md5_gen(uint8_t *msg, uint32_t *hash); void __roc_api roc_hash_sha1_gen(uint8_t *msg, uint32_t *hash); -void __roc_api roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash); +void __roc_api roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash, int hash_size); void __roc_api roc_hash_sha512_gen(uint8_t *msg, uint64_t *hash, int hash_size); #endif /* _ROC_HASH_H_ */ diff --git a/drivers/common/cnxk/roc_se.c b/drivers/common/cnxk/roc_se.c index 2663480099..22df61f5f0 100644 --- a/drivers/common/cnxk/roc_se.c +++ b/drivers/common/cnxk/roc_se.c @@ -149,6 +149,53 @@ cpt_ciph_aes_key_type_set(struct roc_se_context *fctx, uint16_t key_len) fctx->enc.aes_key = aes_key_type; } +static void +cpt_hmac_opad_ipad_gen(roc_se_auth_type auth_type, const uint8_t *key, uint16_t length, + struct roc_se_hmac_context *hmac) +{ + uint8_t opad[128] = {[0 ... 127] = 0x5c}; + uint8_t ipad[128] = {[0 ... 127] = 0x36}; + uint32_t i; + + /* HMAC OPAD and IPAD */ + for (i = 0; i < 128 && i < length; i++) { + opad[i] = opad[i] ^ key[i]; + ipad[i] = ipad[i] ^ key[i]; + } + + /* Precompute hash of HMAC OPAD and IPAD to avoid + * per packet computation + */ + switch (auth_type) { + case ROC_SE_MD5_TYPE: + roc_hash_md5_gen(opad, (uint32_t *)hmac->opad); + roc_hash_md5_gen(ipad, (uint32_t *)hmac->ipad); + break; + case ROC_SE_SHA1_TYPE: + roc_hash_sha1_gen(opad, (uint32_t *)hmac->opad); + roc_hash_sha1_gen(ipad, (uint32_t *)hmac->ipad); + break; + case ROC_SE_SHA2_SHA224: + roc_hash_sha256_gen(opad, (uint32_t *)hmac->opad, 224); + roc_hash_sha256_gen(ipad, (uint32_t *)hmac->ipad, 224); + break; + case ROC_SE_SHA2_SHA256: + roc_hash_sha256_gen(opad, (uint32_t *)hmac->opad, 256); + roc_hash_sha256_gen(ipad, (uint32_t *)hmac->ipad, 256); + break; + case ROC_SE_SHA2_SHA384: + roc_hash_sha512_gen(opad, (uint64_t *)hmac->opad, 384); + roc_hash_sha512_gen(ipad, (uint64_t *)hmac->ipad, 384); + break; + case ROC_SE_SHA2_SHA512: + roc_hash_sha512_gen(opad, (uint64_t *)hmac->opad, 512); + roc_hash_sha512_gen(ipad, (uint64_t *)hmac->ipad, 512); + break; + default: + break; + } +} + static int cpt_pdcp_key_type_set(struct roc_se_zuc_snow3g_ctx *zs_ctx, uint16_t key_len) { @@ -434,9 +481,8 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, memset(fctx->hmac.ipad, 0, sizeof(fctx->hmac.ipad)); memset(fctx->hmac.opad, 0, sizeof(fctx->hmac.opad)); - if (key_len <= 64) - memcpy(fctx->hmac.opad, key, key_len); - fctx->enc.auth_input_type = 1; + cpt_hmac_opad_ipad_gen(type, key, key_len, &fctx->hmac); + fctx->enc.auth_input_type = 0; } return 0; } From patchwork Tue Dec 20 14:32:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121077 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 87A8CA0545; Tue, 20 Dec 2022 15:33:02 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D97D34114A; Tue, 20 Dec 2022 15:32:48 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id B51924114A for ; Tue, 20 Dec 2022 15:32:47 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEOLfm018975 for ; Tue, 20 Dec 2022 06:32:47 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=73dhRCJrvirECcUYJb2vLvQ6ke8AdnyCMCl+jle5y18=; b=G0E/vlgi7BHL4qTXMUzOufX6U7SCqNGWS9mfb911ajVSIWpHVIwUo4UEdC3t03BZaO51 v1EjRqiRWavoZNmdiv0oWtYjWqYnvYvFEyEeQaHqvztXvXTlMxseYpo0zk5jONwJ9m2v vekoTHRoz/vP+dMR0NyxNNVVWSBPxfN5okxXs9ZdXSNXJW3rjhCwNDt4cT5bNB0uG+rM NhnNNnh9LT8wtByfSib/vgoc3p+/oIrGBQ2Bk4ynhGYoElNab3xK8XQy0gbZxBV0BGUG 59rcR22XcrdT/qtbZDZHZEZjVxwG7ObwTDwmywe6CJ8Nhszx6VuXTFR2KGmlQ0oYLxxJ VA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb57-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:32:46 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:32:44 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:32:44 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 7EA333F704C; Tue, 20 Dec 2022 06:32:42 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Vidya Sagar Velumuri , Anoob Joseph , Gowrishankar Muthukrishnan , Volodymyr Fialko , Aakash Sasidharan , Subject: [PATCH 03/17] crypto/cnxk: update resp len calculation for IPv6 Date: Tue, 20 Dec 2022 20:02:18 +0530 Message-ID: <20221220143232.2519650-4-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: dD7NBVGHS6PSm85bt3RmjeKwmQvz9NbD X-Proofpoint-ORIG-GUID: dD7NBVGHS6PSm85bt3RmjeKwmQvz9NbD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri In IPsec lookaside path, response length for an encryption is pre calculated and updated in the mbuf as the packet length. Response length dpepends on the layer-3 header length. As IPsec security context does not provide any information about the IP version, the layer-3 header length is calculated in data path based on IP version and extension headers. Signed-off-by: Vidya Sagar Velumuri --- drivers/common/cnxk/cnxk_security.c | 4 +-- drivers/crypto/cnxk/cn9k_ipsec_la_ops.h | 36 ++++++++++++++++++++++--- 2 files changed, 35 insertions(+), 5 deletions(-) diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c index 5034c76938..79427d48fe 100644 --- a/drivers/common/cnxk/cnxk_security.c +++ b/drivers/common/cnxk/cnxk_security.c @@ -19,7 +19,7 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform, uint32_t i; /* HMAC OPAD and IPAD */ - for (i = 0; i < 127 && i < length; i++) { + for (i = 0; i < 128 && i < length; i++) { opad[i] = opad[i] ^ key[i]; ipad[i] = ipad[i] ^ key[i]; } @@ -1344,7 +1344,7 @@ cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec, } else ctx_len += sizeof(template->ip4); - ctx_len += RTE_ALIGN_CEIL(ctx_len, 8); + ctx_len = RTE_ALIGN_CEIL(ctx_len, 8); if (crypto_xform->type != RTE_CRYPTO_SYM_XFORM_AEAD) { uint8_t *hmac_opad_ipad = (uint8_t *)&out_sa->sha2; diff --git a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h index 724fc525ad..f1298017ce 100644 --- a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h @@ -13,14 +13,44 @@ #include "cnxk_security_ar.h" static __rte_always_inline int32_t -ipsec_po_out_rlen_get(struct cn9k_sec_session *sess, uint32_t plen) +ipsec_po_out_rlen_get(struct cn9k_sec_session *sess, uint32_t plen, struct rte_mbuf *m_src) { uint32_t enc_payload_len; int adj_len = 0; - if (sess->sa.out_sa.common_sa.ctl.ipsec_mode == ROC_IE_SA_MODE_TRANSPORT) + if (sess->sa.out_sa.common_sa.ctl.ipsec_mode == ROC_IE_SA_MODE_TRANSPORT) { adj_len = ROC_CPT_TUNNEL_IPV4_HDR_LEN; + uintptr_t data = (uintptr_t)m_src->buf_addr + m_src->data_off; + struct rte_ipv4_hdr *ip = (struct rte_ipv4_hdr *)data; + + if (unlikely(((ip->version_ihl & 0xf0) >> RTE_IPV4_IHL_MULTIPLIER) != IPVERSION)) { + struct rte_ipv6_hdr *ip6 = (struct rte_ipv6_hdr *)ip; + uint8_t *nxt_hdr = (uint8_t *)ip6; + uint8_t dest_op_cnt = 0; + int nh = ip6->proto; + + PLT_ASSERT(((ip->version_ihl & 0xf0) >> RTE_IPV4_IHL_MULTIPLIER) == 6); + + adj_len = ROC_CPT_TUNNEL_IPV6_HDR_LEN; + nxt_hdr += ROC_CPT_TUNNEL_IPV6_HDR_LEN; + while (nh != -EINVAL) { + size_t ext_len = 0; + + nh = rte_ipv6_get_next_ext(nxt_hdr, nh, &ext_len); + /* With multiple dest ops headers, the ESP hdr will be before + * the 2nd dest ops and after the first dest ops header + */ + if ((nh == IPPROTO_DSTOPTS) && dest_op_cnt) + break; + else if (nh == IPPROTO_DSTOPTS) + dest_op_cnt++; + adj_len += ext_len; + nxt_hdr += ext_len; + } + } + } + enc_payload_len = RTE_ALIGN_CEIL(plen + sess->rlens.roundup_len - adj_len, sess->rlens.roundup_byte); @@ -41,7 +71,7 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_sec_session *sess, struct pkt_len = rte_pktmbuf_pkt_len(m_src); dlen = pkt_len + hdr_len; - rlen = ipsec_po_out_rlen_get(sess, pkt_len); + rlen = ipsec_po_out_rlen_get(sess, pkt_len, m_src); extend_tail = rlen - dlen; if (unlikely(extend_tail > rte_pktmbuf_tailroom(m_src))) { From patchwork Tue Dec 20 14:32:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121078 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 56897A0545; Tue, 20 Dec 2022 15:33:09 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C2F2142D23; Tue, 20 Dec 2022 15:32:51 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 3EA5640395 for ; Tue, 20 Dec 2022 15:32:50 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEOVhH019007 for ; Tue, 20 Dec 2022 06:32:49 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=vLntYojYCRS2Nys1rx/DU0VE/9Qx73zW7fJyKIzpB6o=; b=TmIsflsYXOr0yiPAa0Ez3pstoXOAWrI24Clh40XzvXM+VPTRlAg6VQmuWz69njJlVFbe 6yP3tI/9Vp4Jzybtc6QwSuHHGu8hLl9N96qhpITEuKkJqYZeqfyTzK/ujOayvwwVqpuu DRG2okUAjvx0yiCw2AtWlKsEyysK/p41LdFi1yfKTw+F/CSC4iwTq90b4VV4JfYmjVvy CK+M7zFfObDFvZlYH9Zu6ftNNFQmslsqySL5fm89wG95z6XA9mPQTAb9xNzn1SninUwM tWxZEfh3NCLz9rjQHfJaEiB9us9GVZVpDlRELE+GvcaKYz3Gdail2azb4rQUTZgyNduI Dw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb59-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:32:49 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:32:47 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:32:47 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 2FDAA3F7063; Tue, 20 Dec 2022 06:32:44 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Volodymyr Fialko , Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Aakash Sasidharan , Subject: [PATCH 04/17] crypto/cnxk: add context to passthrough instruction Date: Tue, 20 Dec 2022 20:02:19 +0530 Message-ID: <20221220143232.2519650-5-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: yVWRuGmcJ2ZbZv1i67LrKXiefoneCvXl X-Proofpoint-ORIG-GUID: yVWRuGmcJ2ZbZv1i67LrKXiefoneCvXl X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Attach valid context of last packet instruction to the passthrough instruction to match hardware requirements. Signed-off-by: Volodymyr Fialko --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 7dad370047..2c750d19bf 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -36,6 +36,7 @@ struct ops_burst { struct vec_request { struct cpt_inflight_req *req; struct rte_event_vector *vec; + union cpt_inst_w7 w7; uint64_t w2; }; @@ -387,7 +388,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, struct cnxk_cpt_qp **qp, ui static inline void cn10k_cpt_vec_inst_fill(struct vec_request *vec_req, struct cpt_inst_s *inst, - struct cnxk_cpt_qp *qp) + struct cnxk_cpt_qp *qp, union cpt_inst_w7 w7) { const union cpt_res_s res = {.cn10k.compcode = CPT_COMP_NOT_DONE}; struct cpt_inflight_req *infl_req = vec_req->req; @@ -400,6 +401,8 @@ cn10k_cpt_vec_inst_fill(struct vec_request *vec_req, struct cpt_inst_s *inst, .s.dlen = 0, }; + w7.s.egrp = ROC_CPT_DFLT_ENG_GRP_SE; + infl_req->vec = vec_req->vec; infl_req->qp = qp; @@ -410,7 +413,7 @@ cn10k_cpt_vec_inst_fill(struct vec_request *vec_req, struct cpt_inst_s *inst, inst->w2.u64 = vec_req->w2; inst->w3.u64 = CNXK_CPT_INST_W3(1, infl_req); inst->w4.u64 = w4.u64; - inst->w7.u64 = ROC_CPT_DFLT_ENG_GRP_SE << 61; + inst->w7.u64 = w7.u64; } static void @@ -451,7 +454,7 @@ cn10k_cpt_vec_submit(struct vec_request vec_tbl[], uint16_t vec_tbl_len, struct again: burst_size = RTE_MIN(PKTS_PER_STEORL, vec_tbl_len); for (i = 0; i < burst_size; i++) - cn10k_cpt_vec_inst_fill(&vec_tbl[i], &inst[i * 2], qp); + cn10k_cpt_vec_inst_fill(&vec_tbl[i], &inst[i * 2], qp, vec_tbl[0].w7); do { fc.u64[0] = __atomic_load_n(fc_addr, __ATOMIC_RELAXED); @@ -590,6 +593,10 @@ next_op:; roc_lmt_submit_steorl(lmt_arg, io_addr); } + /* Store w7 of last successfully filled instruction */ + inst = &inst_base[2 * (i - 1)]; + vec_tbl[0].w7 = inst->w7; + rte_io_wmb(); put: From patchwork Tue Dec 20 14:32:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121079 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BCD1EA0545; Tue, 20 Dec 2022 15:33:16 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A83D542D29; Tue, 20 Dec 2022 15:32:54 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 1E60042D29 for ; Tue, 20 Dec 2022 15:32:53 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEOPTL018983 for ; Tue, 20 Dec 2022 06:32:52 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=dpRwqrMVDbpvPS+whSmo6Abyt17fjULTTPK5XJaZL3o=; b=iZOS7rbQUK7p1ySFPRLZvjaCWc91kI2S9L/2E0TTSVtXvTNEhJySU8kQ8P9YJzPORrnR keITX9uXJe9G+OyjT3NBiLPe4rz+YPglpwcODZe4MUeEQPVnp6oLxrt/1hk7P1EVf+rV 1avej4ioCG27iDsK9bwG9QDvADtd4icTrO+6QGFpnP4z8J5zgQ3XSZIoKHF4A+h8eq/C YytWA2CEFAsnv4EU6tfeZGfpjbskGU0IZogcfjdsXlSiO7KWqanYzr9OsLiD6MVcfk2L mD4q6nhXh7T9BBrL5VFUtqgC+Im4WhtD4OuZL9X+UiWHL2JbpUyL2J/mHQIqMUx+4pAK 6w== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:32:52 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:32:50 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:32:50 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id D5D7B3F704C; Tue, 20 Dec 2022 06:32:47 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , "Aakash Sasidharan" , Subject: [PATCH 05/17] crypto/cnxk: support truncated digest length Date: Tue, 20 Dec 2022 20:02:20 +0530 Message-ID: <20221220143232.2519650-6-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: UdstAe4pVNvZOcQT5BHu84Yl7tLC8yZg X-Proofpoint-ORIG-GUID: UdstAe4pVNvZOcQT5BHu84Yl7tLC8yZg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adding truncated digest length support. Signed-off-by: Tejasree Kondoj --- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 48 +++++++++---------- drivers/crypto/cnxk/cnxk_se.h | 19 ++++---- 2 files changed, 35 insertions(+), 32 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index 6c28f8942e..9dfbf875ec 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -108,9 +108,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 0 }, .digest_size = { - .min = 20, + .min = 1, .max = 20, - .increment = 0 + .increment = 1 }, }, } }, } @@ -128,9 +128,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 1 }, .digest_size = { - .min = 12, + .min = 1, .max = 20, - .increment = 8 + .increment = 1 }, }, } }, } @@ -148,9 +148,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 0 }, .digest_size = { - .min = 28, + .min = 1, .max = 28, - .increment = 0 + .increment = 1 }, }, } }, } @@ -168,9 +168,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 1 }, .digest_size = { - .min = 28, + .min = 1, .max = 28, - .increment = 0 + .increment = 1 }, }, } }, } @@ -188,9 +188,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 0 }, .digest_size = { - .min = 32, + .min = 1, .max = 32, - .increment = 0 + .increment = 1 }, }, } }, } @@ -208,9 +208,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 1 }, .digest_size = { - .min = 16, + .min = 1, .max = 32, - .increment = 16 + .increment = 1 }, }, } }, } @@ -228,9 +228,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 0 }, .digest_size = { - .min = 48, + .min = 1, .max = 48, - .increment = 0 + .increment = 1 }, }, } }, } @@ -248,9 +248,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 1 }, .digest_size = { - .min = 24, + .min = 1, .max = 48, - .increment = 24 + .increment = 1 }, }, } }, } @@ -268,9 +268,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 0 }, .digest_size = { - .min = 64, + .min = 1, .max = 64, - .increment = 0 + .increment = 1 }, }, } }, } @@ -288,9 +288,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 1 }, .digest_size = { - .min = 32, + .min = 1, .max = 64, - .increment = 32 + .increment = 1 }, }, } }, } @@ -308,9 +308,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 0 }, .digest_size = { - .min = 16, + .min = 1, .max = 16, - .increment = 0 + .increment = 1 }, }, } }, } @@ -328,9 +328,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { .increment = 8 }, .digest_size = { - .min = 12, + .min = 1, .max = 16, - .increment = 4 + .increment = 1 }, }, } }, } diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index b07fc22858..ce4c2bb585 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -148,30 +148,33 @@ cpt_mac_len_verify(struct rte_crypto_auth_xform *auth) uint16_t mac_len = auth->digest_length; int ret; + if ((auth->algo != RTE_CRYPTO_AUTH_NULL) && (mac_len == 0)) + return -1; + switch (auth->algo) { case RTE_CRYPTO_AUTH_MD5: case RTE_CRYPTO_AUTH_MD5_HMAC: - ret = (mac_len == 16) ? 0 : -1; + ret = (mac_len <= 16) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA1: case RTE_CRYPTO_AUTH_SHA1_HMAC: - ret = (mac_len == 20) ? 0 : -1; + ret = (mac_len <= 20) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA224: case RTE_CRYPTO_AUTH_SHA224_HMAC: - ret = (mac_len == 28) ? 0 : -1; + ret = (mac_len <= 28) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA256: case RTE_CRYPTO_AUTH_SHA256_HMAC: - ret = (mac_len == 32) ? 0 : -1; + ret = (mac_len <= 32) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA384: case RTE_CRYPTO_AUTH_SHA384_HMAC: - ret = (mac_len == 48) ? 0 : -1; + ret = (mac_len <= 48) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA512: case RTE_CRYPTO_AUTH_SHA512_HMAC: - ret = (mac_len == 64) ? 0 : -1; + ret = (mac_len <= 64) ? 0 : -1; break; case RTE_CRYPTO_AUTH_NULL: ret = 0; @@ -838,7 +841,7 @@ cpt_digest_gen_sg_ver1_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa /*GP op header */ cpt_inst_w4.s.opcode_minor = 0; - cpt_inst_w4.s.param2 = ((uint16_t)hash_type << 8); + cpt_inst_w4.s.param2 = ((uint16_t)hash_type << 8) | mac_len; if (ctx->hmac) { cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_HMAC | ROC_SE_DMA_MODE; @@ -969,7 +972,7 @@ cpt_digest_gen_sg_ver2_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa /*GP op header */ cpt_inst_w4.s.opcode_minor = 0; - cpt_inst_w4.s.param2 = ((uint16_t)hash_type << 8); + cpt_inst_w4.s.param2 = ((uint16_t)hash_type << 8) | mac_len; if (ctx->hmac) { cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_HMAC; cpt_inst_w4.s.param1 = key_len; From patchwork Tue Dec 20 14:32:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121080 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DCD37A0545; Tue, 20 Dec 2022 15:33:25 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EFE5142D35; Tue, 20 Dec 2022 15:32:56 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id ACF4142D2D for ; Tue, 20 Dec 2022 15:32:55 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEOLfn018975 for ; Tue, 20 Dec 2022 06:32:55 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Mps351WcRzhy4L4pZeAgDMNdLDKEBVI1QEFYs1ntkdc=; b=QGeSIAZy2nTdxmSzIq2739a/TeZFWhhfJDj9La6vIuLVjvJ2SIpaJa8OGv6svLrQn0a6 k+yE4niTWxaEmZd5os1zn75cdx3CtgIBF9j54pDgdnRbEwlOiEyujSCBijMEgonpzPbr QyRksd15OpJsAP2W/yk1/NfmRGnNT2gfA5BDNamksiY1DAjppZDD3jZK+J1EL8vjyWhC wdzC1VKDhswrOi8juMlazK3Q9uBM4PzWyal222ntA8IN0j/NDXRd/wXV7W1FIqHdE4hb rzfq4qjQz5oizoeDLRw6vzYKuUCB6+d1r9voiXsTkharzh5vjN6b8WJoYK8d+EvEpQvJ bQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:32:55 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:32:52 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:32:52 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 838073F7063; Tue, 20 Dec 2022 06:32:50 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Volodymyr Fialko , Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Aakash Sasidharan , Subject: [PATCH 06/17] crypto/cnxk: add queue pair check to meta set Date: Tue, 20 Dec 2022 20:02:21 +0530 Message-ID: <20221220143232.2519650-7-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 2c_9qHb1m_LSEP3ZUZFCGQ7lZenaIqMr X-Proofpoint-ORIG-GUID: 2c_9qHb1m_LSEP3ZUZFCGQ7lZenaIqMr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Check that queue pair provided by metadata information is enabled (added to crypto adapter). Signed-off-by: Volodymyr Fialko --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 2c750d19bf..1caa321112 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -296,6 +296,9 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, vo qp_id = ec_mdata->request_info.queue_pair_id; qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id]; + if (!qp->ca.enabled) + return -EINVAL; + /* Prepare w2 */ tag_type = qp->ca.vector_sz ? RTE_EVENT_TYPE_CRYPTODEV_VECTOR : RTE_EVENT_TYPE_CRYPTODEV; rsp_info = &ec_mdata->response_info; From patchwork Tue Dec 20 14:32:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121081 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A237A0545; Tue, 20 Dec 2022 15:33:33 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E415B42D30; Tue, 20 Dec 2022 15:32:59 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 81FBC42D2D for ; Tue, 20 Dec 2022 15:32:58 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEOarI019236 for ; Tue, 20 Dec 2022 06:32:57 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=QEn4ZTqw6JAdR+A+xwSWr4+hwylB81f9KooHcrxL6FA=; b=HXAB76zeK35BbLEfs4XQBiCFLwhLDgnxHqlafVHPiJIqm8WsRGo4zuzhCVHMR8MhqCPk Rkned7eryVVLFBpxn0vRMFO2sPApfotZknjmSgGz2S7cwA5c2vgwFnk4EMVhLcWn4mMR PtvjavQ0VlvgBdWWpSEFxJPquF5BpDVigDI/qTly5XAtQyrlBtIZkdWrrWo71lTQBSb0 /YOxJaKI2R46McJMeSLUnPO2u67wJeTGzJAy0ItufhzJBr4/O5FGSF0K/ZEsMcsiU2RL LayPRqIwbUINrqrFeE5TJ7GlHF9bOb2v51p+IJUz9a5TIYw1p1zYFKmz0LIsHdzriy71 Iw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:32:57 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:32:55 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:32:55 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 4E5673F704C; Tue, 20 Dec 2022 06:32:53 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Vidya Sagar Velumuri , Anoob Joseph , Gowrishankar Muthukrishnan , Volodymyr Fialko , Aakash Sasidharan , Subject: [PATCH 07/17] crypto/cnxk: update crypto completion code handling Date: Tue, 20 Dec 2022 20:02:22 +0530 Message-ID: <20221220143232.2519650-8-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: Qebon3xLLMmJh_8Wqz4gRgI2zihkDTXC X-Proofpoint-ORIG-GUID: Qebon3xLLMmJh_8Wqz4gRgI2zihkDTXC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Update crypto and IPsec completion handling as per microcode version OCPT-04-IE-IPSEC-MC-30-01-28-00 ci: skip_checkformat Signed-off-by: Vidya Sagar Velumuri --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 18 +++++++++--------- drivers/crypto/cnxk/cn10k_ipsec.c | 7 +++++-- drivers/crypto/cnxk/cn10k_ipsec.h | 1 + drivers/crypto/cnxk/cn10k_ipsec_la_ops.h | 1 + 4 files changed, 16 insertions(+), 11 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 1caa321112..5a098ffcf2 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -801,13 +801,11 @@ cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *re struct rte_mbuf *mbuf = cop->sym->m_src; const uint16_t m_len = res->rlen; - mbuf->data_len = m_len; - mbuf->pkt_len = m_len; - switch (res->uc_compcode) { case ROC_IE_OT_UCC_SUCCESS: break; case ROC_IE_OT_UCC_SUCCESS_PKT_IP_BADCSUM: + mbuf->ol_flags &= ~RTE_MBUF_F_RX_IP_CKSUM_GOOD; mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD; break; case ROC_IE_OT_UCC_SUCCESS_PKT_L4_GOODCSUM: @@ -819,15 +817,17 @@ cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *re RTE_MBUF_F_RX_IP_CKSUM_GOOD; break; case ROC_IE_OT_UCC_SUCCESS_PKT_IP_GOODCSUM: - mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD; break; case ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_FIRST: + case ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_AGAIN: cop->aux_flags = RTE_CRYPTO_OP_AUX_FLAGS_IPSEC_SOFT_EXPIRY; break; default: - plt_dp_err("Success with unknown microcode completion code"); - break; + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + return; } + mbuf->data_len = m_len; + mbuf->pkt_len = m_len; } static inline void @@ -843,7 +843,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC && cop->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { - if (likely(compcode == CPT_COMP_WARN)) { + if (likely(compcode == CPT_COMP_GOOD || compcode == CPT_COMP_WARN)) { /* Success with additional info */ cn10k_cpt_sec_post_process(cop, res); } else { @@ -860,7 +860,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, return; } - if (likely(compcode == CPT_COMP_GOOD || compcode == CPT_COMP_WARN)) { + if (likely(compcode == CPT_COMP_GOOD)) { if (unlikely(uc_compcode)) { if (uc_compcode == ROC_SE_ERR_GC_ICV_MISCOMPARE) cop->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; @@ -964,7 +964,7 @@ cn10k_cpt_crypto_adapter_vector_dequeue(uintptr_t get_work1) #ifdef CNXK_CRYPTODEV_DEBUG res.u64[0] = __atomic_load_n(&vec_infl_req->res.u64[0], __ATOMIC_RELAXED); - PLT_ASSERT(res.cn10k.compcode == CPT_COMP_WARN); + PLT_ASSERT(res.cn10k.compcode == CPT_COMP_GOOD); PLT_ASSERT(res.cn10k.uc_compcode == 0); #endif diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c index 1740a73c36..aafd461436 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.c +++ b/drivers/crypto/cnxk/cn10k_ipsec.c @@ -200,9 +200,12 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, /* Disable IP checksum verification by default */ param1.s.ip_csum_disable = ROC_IE_OT_SA_INNER_PKT_IP_CSUM_DISABLE; + /* Set the ip chksum flag in mbuf before enqueue. + * Reset the flag in post process in case of errors + */ if (ipsec_xfrm->options.ip_csum_enable) { - param1.s.ip_csum_disable = - ROC_IE_OT_SA_INNER_PKT_IP_CSUM_ENABLE; + param1.s.ip_csum_disable = ROC_IE_OT_SA_INNER_PKT_IP_CSUM_ENABLE; + sec_sess->ip_csum = RTE_MBUF_F_RX_IP_CKSUM_GOOD; } /* Disable L4 checksum verification by default */ diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h index 044fe33046..23d7a4fac4 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.h +++ b/drivers/crypto/cnxk/cn10k_ipsec.h @@ -33,6 +33,7 @@ struct cn10k_sec_session { uint16_t max_extended_len; uint16_t iv_offset; uint8_t iv_length; + uint8_t ip_csum; bool is_outbound; /** Queue pair */ struct cnxk_cpt_qp *qp; diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h index 084198b5bb..f2761a55a5 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h @@ -98,6 +98,7 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_sec_session *sess, struct inst->w4.u64 = sess->inst.w4 | rte_pktmbuf_pkt_len(m_src); dptr = rte_pktmbuf_mtod(m_src, uint64_t); inst->dptr = dptr; + m_src->ol_flags |= (uint64_t)sess->ip_csum; return 0; } From patchwork Tue Dec 20 14:32:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121082 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E96BFA0545; Tue, 20 Dec 2022 15:33:39 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EEAF841141; Tue, 20 Dec 2022 15:33:01 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 12E9342D3C for ; Tue, 20 Dec 2022 15:33:00 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEJKMU010037 for ; Tue, 20 Dec 2022 06:33:00 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=KT/L8du9s3g+qUhvbYqFxCCoZcEUCGA0FokxbMlY3GE=; b=HBxA5XdECh2Im6nOINr+nhumXsKs2Ol6fn1eADZkKTZfG8KDaUXYAnEC1Mvv0KRkwwra 16fC417PQPlc0Q9V1uhhpzKPfyU3o+GT1Tx0GTTTwnpVE28nt0sipHeORp5Z3D5RLttQ 69OR7eeRwAU6eRXtf2GameGOj/bzRvgJI+P+izQtyFOawJKXuwmsI/wFB1151l6eP435 nPYIJNooYmLjcHwKForYfEOwtIwiVFPlfvSnTOgTpJLknrlTuSAKHYwqY32MwUwL6ktb UjYLHxYSHQhp2bnTBXzCVMQKnlZVpiHwEVv3zSejIpFA5uRhydwx3sHNN/nnSsk/ECap kA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:00 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:32:58 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:32:58 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id F0B763F7063; Tue, 20 Dec 2022 06:32:55 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Gowrishankar Muthukrishnan , Anoob Joseph , Vidya Sagar Velumuri , "Volodymyr Fialko" , Aakash Sasidharan , Subject: [PATCH 08/17] crypto/cnxk: fix incorrect digest for an empty input data Date: Tue, 20 Dec 2022 20:02:23 +0530 Message-ID: <20221220143232.2519650-9-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: h3pVg56NRCLogYDkZpC9zyNlAqD0saw- X-Proofpoint-ORIG-GUID: h3pVg56NRCLogYDkZpC9zyNlAqD0saw- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Gowrishankar Muthukrishnan For an empty input data, digest value is incorrectly generated from previous auth op, in case executed. This is due to incorrect gather component used. Fixes: 7c19abdd0cf1 ("common/cnxk: support 103XX CPT") Signed-off-by: Anoob Joseph Signed-off-by: Gowrishankar Muthukrishnan --- drivers/crypto/cnxk/cnxk_se.h | 46 ++++++++++++++--------------------- 1 file changed, 18 insertions(+), 28 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index ce4c2bb585..2944d3c287 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -247,7 +247,7 @@ fill_sg_comp_from_iov(struct roc_se_sglist_comp *list, uint32_t i, uint32_t extra_len = extra_buf ? extra_buf->size : 0; uint32_t size = *psize; - for (j = 0; (j < from->buf_cnt) && size; j++) { + for (j = 0; j < from->buf_cnt; j++) { struct roc_se_sglist_comp *to = &list[i >> 2]; uint32_t buf_sz = from->bufs[j].size; void *vaddr = from->bufs[j].vaddr; @@ -314,6 +314,9 @@ fill_sg_comp_from_iov(struct roc_se_sglist_comp *list, uint32_t i, if (extra_offset) extra_offset -= size; i++; + + if (unlikely(!size)) + break; } *psize = size; @@ -370,7 +373,9 @@ fill_sg2_comp_from_iov(struct roc_se_sg2list_comp *list, uint32_t i, struct roc_ uint32_t extra_len = extra_buf ? extra_buf->size : 0; uint32_t size = *psize; - for (j = 0; (j < from->buf_cnt) && size; j++) { + rte_prefetch2(psize); + + for (j = 0; j < from->buf_cnt; j++) { struct roc_se_sg2list_comp *to = &list[i / 3]; uint32_t buf_sz = from->bufs[j].size; void *vaddr = from->bufs[j].vaddr; @@ -436,6 +441,9 @@ fill_sg2_comp_from_iov(struct roc_se_sg2list_comp *list, uint32_t i, struct roc_ if (extra_offset) extra_offset -= size; i++; + + if (unlikely(!size)) + break; } *psize = size; @@ -887,20 +895,10 @@ cpt_digest_gen_sg_ver1_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa /* input data */ size = data_len; - if (size) { - i = fill_sg_comp_from_iov(gather_comp, i, params->src_iov, 0, - &size, NULL, 0); - if (unlikely(size)) { - plt_dp_err("Insufficient dst IOV size, short by %dB", - size); - return -1; - } - } else { - /* - * Looks like we need to support zero data - * gather ptr in case of hash & hmac - */ - i++; + i = fill_sg_comp_from_iov(gather_comp, i, params->src_iov, 0, &size, NULL, 0); + if (unlikely(size)) { + plt_dp_err("Insufficient dst IOV size, short by %dB", size); + return -1; } ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_se_sglist_comp); @@ -1011,18 +1009,10 @@ cpt_digest_gen_sg_ver2_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa /* input data */ size = data_len; - if (size) { - i = fill_sg2_comp_from_iov(gather_comp, i, params->src_iov, 0, &size, NULL, 0); - if (unlikely(size)) { - plt_dp_err("Insufficient dst IOV size, short by %dB", size); - return -1; - } - } else { - /* - * Looks like we need to support zero data - * gather ptr in case of hash & hmac - */ - i++; + i = fill_sg2_comp_from_iov(gather_comp, i, params->src_iov, 0, &size, NULL, 0); + if (unlikely(size)) { + plt_dp_err("Insufficient dst IOV size, short by %dB", size); + return -1; } cpt_inst_w5.s.gather_sz = ((i + 2) / 3); From patchwork Tue Dec 20 14:32:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121083 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 43D10A0545; Tue, 20 Dec 2022 15:33:46 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 720DA410DE; Tue, 20 Dec 2022 15:33:07 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 3B05D42D40 for ; Tue, 20 Dec 2022 15:33:05 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEJKMV010037 for ; Tue, 20 Dec 2022 06:33:04 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=2jlKhFo6ZMf6TGQybe0UjNwAYA79BsjUNwc02EOKR+8=; b=GOqhrK1r6Ot2LN8Z5VLVyn5Tv7ahpszDYt4ufRjIZ1FfSpJEBuGQOZEUVPddFy2WgDlZ mwytVGxh98givB2LOHmYAn/Lr4S1+vWo1k3Sw+LdmpQbulifGBdq9/3hkJpT9tQ4AuSJ BuUc6n69IDwPBqmKLZUyf3Bnm/8rIOtU1pLoMrqdWYRcrS0k3VmwFyhzuo5Arzp21279 ISTbX8/CYtAxbsdRysJikzyDYWfBXdnivaFuWVmDUGYtxHc4YSuHk5I7uuCvXXw0a+36 0esdzksAD21vhdEx2VZA3Z0SEayLVlfgd/OymL2o/yYj4RHyrDHF8hK0aruuAqPDm4gN kg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:03 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:33:01 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:33:01 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id A0B0C3F704C; Tue, 20 Dec 2022 06:32:58 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Archana Muniganti , Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , Aakash Sasidharan , Subject: [PATCH 09/17] crypto/cnxk: add CN9K IPsec SG support Date: Tue, 20 Dec 2022 20:02:24 +0530 Message-ID: <20221220143232.2519650-10-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: nYsS0xngQnS6tZsRe6Vi0Kop6VKFtWqK X-Proofpoint-ORIG-GUID: nYsS0xngQnS6tZsRe6Vi0Kop6VKFtWqK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Archana Muniganti Added IPsec SG support in CN9K Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj --- drivers/common/cnxk/roc_api.h | 1 + drivers/common/cnxk/roc_cpt_sg.h | 37 +++ drivers/common/cnxk/roc_ie_on.h | 9 +- drivers/common/cnxk/roc_se.h | 28 -- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 75 ++--- drivers/crypto/cnxk/cn9k_ipsec_la_ops.h | 171 ++++++++-- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 25 +- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 19 ++ drivers/crypto/cnxk/cnxk_se.h | 387 +++-------------------- drivers/crypto/cnxk/cnxk_sg.h | 273 ++++++++++++++++ 10 files changed, 594 insertions(+), 431 deletions(-) create mode 100644 drivers/common/cnxk/roc_cpt_sg.h create mode 100644 drivers/crypto/cnxk/cnxk_sg.h diff --git a/drivers/common/cnxk/roc_api.h b/drivers/common/cnxk/roc_api.h index 072f16d77d..14a11321e0 100644 --- a/drivers/common/cnxk/roc_api.h +++ b/drivers/common/cnxk/roc_api.h @@ -86,6 +86,7 @@ /* CPT microcode */ #include "roc_ae.h" #include "roc_ae_fpm_tables.h" +#include "roc_cpt_sg.h" #include "roc_ie.h" #include "roc_ie_on.h" #include "roc_ie_ot.h" diff --git a/drivers/common/cnxk/roc_cpt_sg.h b/drivers/common/cnxk/roc_cpt_sg.h new file mode 100644 index 0000000000..8a97e1aa5b --- /dev/null +++ b/drivers/common/cnxk/roc_cpt_sg.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _ROC_CPT_SG_H_ +#define _ROC_CPT_SG_H_ + +#define ROC_DMA_MODE_SG (1 << 7) + +#define ROC_MAX_SG_IN_OUT_CNT 32 +#define ROC_MAX_SG_CNT (ROC_MAX_SG_IN_OUT_CNT / 2) + +#define ROC_SG_LIST_HDR_SIZE (8u) +#define ROC_SG_ENTRY_SIZE sizeof(struct roc_sglist_comp) + +struct roc_sglist_comp { + union { + uint64_t len; + struct { + uint16_t len[4]; + } s; + } u; + uint64_t ptr[4]; +}; + +struct roc_sg2list_comp { + union { + uint64_t len; + struct { + uint16_t len[3]; + uint16_t valid_segs; + } s; + } u; + uint64_t ptr[3]; +}; + +#endif /* _ROC_CPT_SG_H_ */ diff --git a/drivers/common/cnxk/roc_ie_on.h b/drivers/common/cnxk/roc_ie_on.h index 057ff95362..9933ffa148 100644 --- a/drivers/common/cnxk/roc_ie_on.h +++ b/drivers/common/cnxk/roc_ie_on.h @@ -25,10 +25,11 @@ enum roc_ie_on_ucc_ipsec { }; /* Helper macros */ -#define ROC_IE_ON_INB_RPTR_HDR 16 -#define ROC_IE_ON_MAX_IV_LEN 16 -#define ROC_IE_ON_PER_PKT_IV BIT(43) -#define ROC_IE_ON_INPLACE_BIT BIT(6) +#define ROC_IE_ON_OUTB_DPTR_HDR 16 +#define ROC_IE_ON_INB_RPTR_HDR 16 +#define ROC_IE_ON_MAX_IV_LEN 16 +#define ROC_IE_ON_PER_PKT_IV BIT(43) +#define ROC_IE_ON_INPLACE_BIT BIT(6) enum { ROC_IE_ON_SA_ENC_NULL = 0, diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index c357c19c0b..a8f0f49479 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -27,13 +27,6 @@ #define ROC_SE_MAX_MAC_LEN 64 #define ROC_SE_OFF_CTRL_LEN 8 -#define ROC_SE_DMA_MODE (1 << 7) - -#define ROC_SE_MAX_SG_IN_OUT_CNT 32 -#define ROC_SE_MAX_SG_CNT (ROC_SE_MAX_SG_IN_OUT_CNT / 2) - -#define ROC_SE_SG_LIST_HDR_SIZE (8u) -#define ROC_SE_SG_ENTRY_SIZE sizeof(struct roc_se_sglist_comp) #define ROC_SE_ZS_EA 0x1 #define ROC_SE_ZS_IA 0x2 @@ -173,27 +166,6 @@ typedef enum { ROC_SE_PDCP_MAC_LEN_128_BIT = 0x3 } roc_se_pdcp_mac_len_type; -struct roc_se_sglist_comp { - union { - uint64_t len; - struct { - uint16_t len[4]; - } s; - } u; - uint64_t ptr[4]; -}; - -struct roc_se_sg2list_comp { - union { - uint64_t len; - struct { - uint16_t len[3]; - uint16_t valid_segs; - } s; - } u; - uint64_t ptr[3]; -}; - struct roc_se_enc_context { uint64_t iv_source : 1; uint64_t aes_key : 2; diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 04c004bc7a..cfe1e08dff 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -18,13 +18,11 @@ #include "cnxk_se.h" static __rte_always_inline int __rte_hot -cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op, - struct cpt_inflight_req *infl_req, - struct cpt_inst_s *inst) +cn9k_cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, + struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst) { struct rte_crypto_sym_op *sym_op = op->sym; struct cn9k_sec_session *sec_sess; - int ret; sec_sess = (struct cn9k_sec_session *)(op->sym->session); @@ -33,22 +31,10 @@ cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op, return -ENOTSUP; } - if (unlikely(!rte_pktmbuf_is_contiguous(sym_op->m_src))) { - plt_dp_err("Scatter Gather mode is not supported"); - return -ENOTSUP; - } - if (sec_sess->is_outbound) - ret = process_outb_sa(op, sec_sess, inst); - else { - infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND; - process_inb_sa(op, sec_sess, inst); - if (unlikely(sec_sess->replay_win_sz)) - infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_INB_REPLAY; - ret = 0; - } - - return ret; + return process_outb_sa(&qp->meta_info, op, sec_sess, inst, infl_req); + else + return process_inb_sa(&qp->meta_info, op, sec_sess, inst, infl_req); } static inline struct cnxk_se_sess * @@ -94,7 +80,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst, false); inst->w7.u64 = sess->cpt_inst_w7; } else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) - ret = cn9k_cpt_sec_inst_fill(op, infl_req, inst); + ret = cn9k_cpt_sec_inst_fill(qp, op, infl_req, inst); else { sess = cn9k_cpt_sym_temp_sess_create(qp, op); if (unlikely(sess == NULL)) { @@ -522,45 +508,60 @@ cn9k_cpt_sec_post_process(struct rte_crypto_op *cop, { struct rte_crypto_sym_op *sym_op = cop->sym; struct rte_mbuf *m = sym_op->m_src; + struct roc_ie_on_inb_hdr *hdr; struct cn9k_sec_session *priv; struct rte_ipv6_hdr *ip6; struct rte_ipv4_hdr *ip; uint16_t m_len = 0; - char *data; if (infl_req->op_flags & CPT_OP_FLAGS_IPSEC_DIR_INBOUND) { - data = rte_pktmbuf_mtod(m, char *); - if (unlikely(infl_req->op_flags & - CPT_OP_FLAGS_IPSEC_INB_REPLAY)) { + hdr = (struct roc_ie_on_inb_hdr *)rte_pktmbuf_mtod(m, char *); + + if (likely(m->next == NULL)) { + ip = PLT_PTR_ADD(hdr, ROC_IE_ON_INB_RPTR_HDR); + } else { + ip = (struct rte_ipv4_hdr *)hdr; + hdr = infl_req->mdata; + } + + if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_IPSEC_INB_REPLAY)) { int ret; priv = (struct cn9k_sec_session *)(sym_op->session); - ret = ipsec_antireplay_check(priv, priv->replay_win_sz, - (struct roc_ie_on_inb_hdr *)data); + ret = ipsec_antireplay_check(priv, priv->replay_win_sz, hdr); if (unlikely(ret)) { cop->status = RTE_CRYPTO_OP_STATUS_ERROR; return; } } - ip = (struct rte_ipv4_hdr *)(data + ROC_IE_ON_INB_RPTR_HDR); - - if (((ip->version_ihl & 0xf0) >> RTE_IPV4_IHL_MULTIPLIER) == - IPVERSION) { + if (((ip->version_ihl & 0xf0) >> RTE_IPV4_IHL_MULTIPLIER) == IPVERSION) { m_len = rte_be_to_cpu_16(ip->total_length); } else { - PLT_ASSERT(((ip->version_ihl & 0xf0) >> - RTE_IPV4_IHL_MULTIPLIER) == 6); + PLT_ASSERT(((ip->version_ihl & 0xf0) >> RTE_IPV4_IHL_MULTIPLIER) == 6); ip6 = (struct rte_ipv6_hdr *)ip; - m_len = rte_be_to_cpu_16(ip6->payload_len) + - sizeof(struct rte_ipv6_hdr); + m_len = rte_be_to_cpu_16(ip6->payload_len) + sizeof(struct rte_ipv6_hdr); } - m->data_len = m_len; - m->pkt_len = m_len; - m->data_off += ROC_IE_ON_INB_RPTR_HDR; + if (likely(m->next == NULL)) { + m->data_len = m_len; + m->pkt_len = m_len; + + m->data_off += ROC_IE_ON_INB_RPTR_HDR; + } else { + struct rte_mbuf *temp = m; + uint8_t m_len_s = m_len; + + while (m_len_s - temp->data_len > 0) { + m_len_s -= temp->data_len; + temp = temp->next; + } + + temp->data_len = m_len_s; + m->pkt_len = m_len; + } } } diff --git a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h index f1298017ce..3d9c851f10 100644 --- a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h @@ -10,7 +10,9 @@ #include #include "cn9k_ipsec.h" +#include "cnxk_cryptodev_ops.h" #include "cnxk_security_ar.h" +#include "cnxk_sg.h" static __rte_always_inline int32_t ipsec_po_out_rlen_get(struct cn9k_sec_session *sess, uint32_t plen, struct rte_mbuf *m_src) @@ -58,7 +60,9 @@ ipsec_po_out_rlen_get(struct cn9k_sec_session *sess, uint32_t plen, struct rte_m } static __rte_always_inline int -process_outb_sa(struct rte_crypto_op *cop, struct cn9k_sec_session *sess, struct cpt_inst_s *inst) +process_outb_sa(struct cpt_qp_meta_info *m_info, struct rte_crypto_op *cop, + struct cn9k_sec_session *sess, struct cpt_inst_s *inst, + struct cpt_inflight_req *infl_req) { const unsigned int hdr_len = sess->custom_hdr_len; struct rte_crypto_sym_op *sym_op = cop->sym; @@ -74,24 +78,90 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_sec_session *sess, struct rlen = ipsec_po_out_rlen_get(sess, pkt_len, m_src); extend_tail = rlen - dlen; - if (unlikely(extend_tail > rte_pktmbuf_tailroom(m_src))) { - plt_dp_err("Not enough tail room (required: %d, available: %d)", - extend_tail, rte_pktmbuf_tailroom(m_src)); - return -ENOMEM; - } + pkt_len += extend_tail; - if (unlikely(hdr_len > data_off)) { - plt_dp_err("Not enough head room (required: %d, available: %d)", - hdr_len, rte_pktmbuf_headroom(m_src)); - return -ENOMEM; - } + if (likely(m_src->next == NULL)) { + if (unlikely(extend_tail > rte_pktmbuf_tailroom(m_src))) { + plt_dp_err("Not enough tail room (required: %d, available: %d)", + extend_tail, rte_pktmbuf_tailroom(m_src)); + return -ENOMEM; + } - pkt_len += extend_tail; + if (unlikely(hdr_len > data_off)) { + plt_dp_err("Not enough head room (required: %d, available: %d)", hdr_len, + rte_pktmbuf_headroom(m_src)); + return -ENOMEM; + } - m_src->data_len = pkt_len; - m_src->pkt_len = pkt_len; + m_src->data_len = pkt_len; + + hdr = PLT_PTR_ADD(m_src->buf_addr, data_off - hdr_len); + + inst->dptr = PLT_U64_CAST(hdr); + inst->w4.u64 = sess->inst.w4 | dlen; + } else { + struct roc_sglist_comp *scatter_comp, *gather_comp; + uint32_t g_size_bytes, s_size_bytes; + struct rte_mbuf *last_seg; + uint8_t *in_buffer; + void *m_data; + int i; + + last_seg = rte_pktmbuf_lastseg(m_src); + + if (unlikely(extend_tail > rte_pktmbuf_tailroom(last_seg))) { + plt_dp_err("Not enough tail room (required: %d, available: %d)", + extend_tail, rte_pktmbuf_tailroom(last_seg)); + return -ENOMEM; + } + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + hdr = m_data; + + m_data = (uint8_t *)m_data + hdr_len; + in_buffer = m_data; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* + * Input Gather List + */ + i = 0; + gather_comp = (struct roc_sglist_comp *)((uint8_t *)m_data + 8); + + i = fill_sg_comp(gather_comp, i, (uint64_t)hdr, hdr_len); + i = fill_ipsec_sg_comp_from_pkt(gather_comp, i, m_src); + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + + g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + /* + * output Scatter List + */ + last_seg->data_len += extend_tail; + + i = 0; + scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); + + i = fill_sg_comp(scatter_comp, i, (uint64_t)hdr, hdr_len); + i = fill_ipsec_sg_comp_from_pkt(scatter_comp, i, m_src); + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); - hdr = PLT_PTR_ADD(m_src->buf_addr, data_off - hdr_len); + s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + dlen = g_size_bytes + s_size_bytes + ROC_SG_LIST_HDR_SIZE; + + inst->dptr = (uint64_t)in_buffer; + + inst->w4.u64 = sess->inst.w4 | dlen; + inst->w4.s.opcode_major |= (uint64_t)ROC_DMA_MODE_SG; + } #ifdef LA_IPSEC_DEBUG if (sess->inst.w4 & ROC_IE_ON_PER_PKT_IV) { @@ -101,6 +171,7 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_sec_session *sess, struct } #endif + m_src->pkt_len = pkt_len; esn = ++sess->esn; /* Set ESN seq hi */ @@ -114,22 +185,80 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_sec_session *sess, struct hdr->ip_id = seq_lo; /* Prepare CPT instruction */ - inst->w4.u64 = sess->inst.w4 | dlen; - inst->dptr = PLT_U64_CAST(hdr); inst->w7.u64 = sess->inst.w7; return 0; } -static __rte_always_inline void -process_inb_sa(struct rte_crypto_op *cop, struct cn9k_sec_session *sess, struct cpt_inst_s *inst) +static __rte_always_inline int +process_inb_sa(struct cpt_qp_meta_info *m_info, struct rte_crypto_op *cop, struct cn9k_sec_session *sess, struct cpt_inst_s *inst, struct cpt_inflight_req *infl_req) { + const unsigned int hdr_len = ROC_IE_ON_INB_RPTR_HDR; struct rte_crypto_sym_op *sym_op = cop->sym; struct rte_mbuf *m_src = sym_op->m_src; + struct roc_ie_on_inb_hdr *hdr; + uint32_t dlen; + + infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND; + if (likely(m_src->next == NULL)) { + dlen = rte_pktmbuf_pkt_len(m_src); + inst->dptr = rte_pktmbuf_mtod(m_src, uint64_t); + inst->w4.u64 = sess->inst.w4 | dlen; + } else { + struct roc_sglist_comp *scatter_comp, *gather_comp; + uint32_t g_size_bytes, s_size_bytes; + uint8_t *in_buffer; + void *m_data; + int i; + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + hdr = m_data; + + m_data = (uint8_t *)m_data + hdr_len; + in_buffer = m_data; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* + * Input Gather List + */ + i = 0; + gather_comp = (struct roc_sglist_comp *)((uint8_t *)m_data + 8); + i = fill_ipsec_sg_comp_from_pkt(gather_comp, i, m_src); + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + + g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + /* + * Output Scatter List + */ + i = 0; + scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); + i = fill_sg_comp(scatter_comp, i, (uint64_t)hdr, hdr_len); + i = fill_ipsec_sg_comp_from_pkt(scatter_comp, i, m_src); + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + + s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + dlen = g_size_bytes + s_size_bytes + ROC_SG_LIST_HDR_SIZE; + + inst->dptr = (uint64_t)in_buffer; + inst->w4.u64 = sess->inst.w4 | dlen; + inst->w4.s.opcode_major |= (uint64_t)ROC_DMA_MODE_SG; + } /* Prepare CPT instruction */ - inst->w4.u64 = sess->inst.w4 | rte_pktmbuf_pkt_len(m_src); - inst->dptr = rte_pktmbuf_mtod(m_src, uint64_t); inst->w7.u64 = sess->inst.w7; + + if (unlikely(sess->replay_win_sz)) + infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_INB_REPLAY; + + return 0; } #endif /* __CN9K_IPSEC_LA_OPS_H__ */ diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index a9c42205e6..eb2ed0d103 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -30,10 +30,22 @@ cnxk_cpt_get_mlen(void) /* For PDCP_CHAIN passthrough alignment */ len += 8; len += ROC_SE_OFF_CTRL_LEN + ROC_CPT_AES_CBC_IV_LEN; - len += RTE_ALIGN_CEIL( - (ROC_SE_SG_LIST_HDR_SIZE + - (RTE_ALIGN_CEIL(ROC_SE_MAX_SG_IN_OUT_CNT, 4) >> 2) * ROC_SE_SG_ENTRY_SIZE), - 8); + len += RTE_ALIGN_CEIL((ROC_SG_LIST_HDR_SIZE + + (RTE_ALIGN_CEIL(ROC_MAX_SG_IN_OUT_CNT, 4) >> 2) * ROC_SG_ENTRY_SIZE), + 8); + + return len; +} + +static int +cnxk_cpt_sec_get_mlen(void) +{ + uint32_t len; + + len = ROC_IE_ON_OUTB_DPTR_HDR + ROC_IE_ON_MAX_IV_LEN; + len += RTE_ALIGN_CEIL((ROC_SG_LIST_HDR_SIZE + + (RTE_ALIGN_CEIL(ROC_MAX_SG_IN_OUT_CNT, 4) >> 2) * ROC_SG_ENTRY_SIZE), + 8); return len; } @@ -196,6 +208,11 @@ cnxk_cpt_metabuf_mempool_create(const struct rte_cryptodev *dev, mlen = cnxk_cpt_get_mlen(); } + if (dev->feature_flags & RTE_CRYPTODEV_FF_SECURITY) { + /* Get meta len for security operations */ + mlen = cnxk_cpt_sec_get_mlen(); + } + if (dev->feature_flags & RTE_CRYPTODEV_FF_ASYMMETRIC_CRYPTO) { /* Get meta len required for asymmetric operations */ diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h index 13c90444d6..5153d334ba 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h @@ -158,4 +158,23 @@ pending_queue_free_cnt(uint64_t head, uint64_t tail, const uint64_t mask) return mask - pending_queue_infl_cnt(head, tail, mask); } +static __rte_always_inline void * +alloc_op_meta(struct roc_se_buf_ptr *buf, int32_t len, struct rte_mempool *cpt_meta_pool, + struct cpt_inflight_req *infl_req) +{ + uint8_t *mdata; + + if (unlikely(rte_mempool_get(cpt_meta_pool, (void **)&mdata) < 0)) + return NULL; + + if (likely(buf)) { + buf->vaddr = mdata; + buf->size = len; + } + + infl_req->mdata = mdata; + infl_req->op_flags |= CPT_OP_FLAGS_METABUF; + + return mdata; +} #endif /* _CNXK_CRYPTODEV_OPS_H_ */ diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 2944d3c287..88049ac431 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -8,11 +8,12 @@ #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_ops.h" +#include "cnxk_sg.h" #define SRC_IOV_SIZE \ - (sizeof(struct roc_se_iov_ptr) + (sizeof(struct roc_se_buf_ptr) * ROC_SE_MAX_SG_CNT)) + (sizeof(struct roc_se_iov_ptr) + (sizeof(struct roc_se_buf_ptr) * ROC_MAX_SG_CNT)) #define DST_IOV_SIZE \ - (sizeof(struct roc_se_iov_ptr) + (sizeof(struct roc_se_buf_ptr) * ROC_SE_MAX_SG_CNT)) + (sizeof(struct roc_se_iov_ptr) + (sizeof(struct roc_se_buf_ptr) * ROC_MAX_SG_CNT)) enum cpt_dp_thread_type { CPT_DP_THREAD_TYPE_FC_CHAIN = 0x1, @@ -193,272 +194,14 @@ cpt_fc_salt_update(struct roc_se_ctx *se_ctx, uint8_t *salt) memcpy(fctx->enc.encr_iv, salt, 4); } -static __rte_always_inline uint32_t -fill_sg_comp(struct roc_se_sglist_comp *list, uint32_t i, phys_addr_t dma_addr, - uint32_t size) -{ - struct roc_se_sglist_comp *to = &list[i >> 2]; - - to->u.s.len[i % 4] = rte_cpu_to_be_16(size); - to->ptr[i % 4] = rte_cpu_to_be_64(dma_addr); - i++; - return i; -} - -static __rte_always_inline uint32_t -fill_sg_comp_from_buf(struct roc_se_sglist_comp *list, uint32_t i, - struct roc_se_buf_ptr *from) -{ - struct roc_se_sglist_comp *to = &list[i >> 2]; - - to->u.s.len[i % 4] = rte_cpu_to_be_16(from->size); - to->ptr[i % 4] = rte_cpu_to_be_64((uint64_t)from->vaddr); - i++; - return i; -} - -static __rte_always_inline uint32_t -fill_sg_comp_from_buf_min(struct roc_se_sglist_comp *list, uint32_t i, - struct roc_se_buf_ptr *from, uint32_t *psize) -{ - struct roc_se_sglist_comp *to = &list[i >> 2]; - uint32_t size = *psize; - uint32_t e_len; - - e_len = (size > from->size) ? from->size : size; - to->u.s.len[i % 4] = rte_cpu_to_be_16(e_len); - to->ptr[i % 4] = rte_cpu_to_be_64((uint64_t)from->vaddr); - *psize -= e_len; - i++; - return i; -} - -/* - * This fills the MC expected SGIO list - * from IOV given by user. - */ -static __rte_always_inline uint32_t -fill_sg_comp_from_iov(struct roc_se_sglist_comp *list, uint32_t i, - struct roc_se_iov_ptr *from, uint32_t from_offset, - uint32_t *psize, struct roc_se_buf_ptr *extra_buf, - uint32_t extra_offset) -{ - int32_t j; - uint32_t extra_len = extra_buf ? extra_buf->size : 0; - uint32_t size = *psize; - - for (j = 0; j < from->buf_cnt; j++) { - struct roc_se_sglist_comp *to = &list[i >> 2]; - uint32_t buf_sz = from->bufs[j].size; - void *vaddr = from->bufs[j].vaddr; - uint64_t e_vaddr; - uint32_t e_len; - - if (unlikely(from_offset)) { - if (from_offset >= buf_sz) { - from_offset -= buf_sz; - continue; - } - e_vaddr = (uint64_t)vaddr + from_offset; - e_len = (size > (buf_sz - from_offset)) ? - (buf_sz - from_offset) : - size; - from_offset = 0; - } else { - e_vaddr = (uint64_t)vaddr; - e_len = (size > buf_sz) ? buf_sz : size; - } - - to->u.s.len[i % 4] = rte_cpu_to_be_16(e_len); - to->ptr[i % 4] = rte_cpu_to_be_64(e_vaddr); - - if (extra_len && (e_len >= extra_offset)) { - /* Break the data at given offset */ - uint32_t next_len = e_len - extra_offset; - uint64_t next_vaddr = e_vaddr + extra_offset; - - if (!extra_offset) { - i--; - } else { - e_len = extra_offset; - size -= e_len; - to->u.s.len[i % 4] = rte_cpu_to_be_16(e_len); - } - - extra_len = RTE_MIN(extra_len, size); - /* Insert extra data ptr */ - if (extra_len) { - i++; - to = &list[i >> 2]; - to->u.s.len[i % 4] = - rte_cpu_to_be_16(extra_len); - to->ptr[i % 4] = rte_cpu_to_be_64( - (uint64_t)extra_buf->vaddr); - size -= extra_len; - } - - next_len = RTE_MIN(next_len, size); - /* insert the rest of the data */ - if (next_len) { - i++; - to = &list[i >> 2]; - to->u.s.len[i % 4] = rte_cpu_to_be_16(next_len); - to->ptr[i % 4] = rte_cpu_to_be_64(next_vaddr); - size -= next_len; - } - extra_len = 0; - - } else { - size -= e_len; - } - if (extra_offset) - extra_offset -= size; - i++; - - if (unlikely(!size)) - break; - } - - *psize = size; - return (uint32_t)i; -} - -static __rte_always_inline uint32_t -fill_sg2_comp(struct roc_se_sg2list_comp *list, uint32_t i, phys_addr_t dma_addr, uint32_t size) -{ - struct roc_se_sg2list_comp *to = &list[i / 3]; - - to->u.s.len[i % 3] = (size); - to->ptr[i % 3] = (dma_addr); - to->u.s.valid_segs = (i % 3) + 1; - i++; - return i; -} - -static __rte_always_inline uint32_t -fill_sg2_comp_from_buf(struct roc_se_sg2list_comp *list, uint32_t i, struct roc_se_buf_ptr *from) -{ - struct roc_se_sg2list_comp *to = &list[i / 3]; - - to->u.s.len[i % 3] = (from->size); - to->ptr[i % 3] = ((uint64_t)from->vaddr); - to->u.s.valid_segs = (i % 3) + 1; - i++; - return i; -} - -static __rte_always_inline uint32_t -fill_sg2_comp_from_buf_min(struct roc_se_sg2list_comp *list, uint32_t i, - struct roc_se_buf_ptr *from, uint32_t *psize) -{ - struct roc_se_sg2list_comp *to = &list[i / 3]; - uint32_t size = *psize; - uint32_t e_len; - - e_len = (size > from->size) ? from->size : size; - to->u.s.len[i % 3] = (e_len); - to->ptr[i % 3] = ((uint64_t)from->vaddr); - to->u.s.valid_segs = (i % 3) + 1; - *psize -= e_len; - i++; - return i; -} - -static __rte_always_inline uint32_t -fill_sg2_comp_from_iov(struct roc_se_sg2list_comp *list, uint32_t i, struct roc_se_iov_ptr *from, - uint32_t from_offset, uint32_t *psize, struct roc_se_buf_ptr *extra_buf, - uint32_t extra_offset) -{ - int32_t j; - uint32_t extra_len = extra_buf ? extra_buf->size : 0; - uint32_t size = *psize; - - rte_prefetch2(psize); - - for (j = 0; j < from->buf_cnt; j++) { - struct roc_se_sg2list_comp *to = &list[i / 3]; - uint32_t buf_sz = from->bufs[j].size; - void *vaddr = from->bufs[j].vaddr; - uint64_t e_vaddr; - uint32_t e_len; - - if (unlikely(from_offset)) { - if (from_offset >= buf_sz) { - from_offset -= buf_sz; - continue; - } - e_vaddr = (uint64_t)vaddr + from_offset; - e_len = (size > (buf_sz - from_offset)) ? (buf_sz - from_offset) : size; - from_offset = 0; - } else { - e_vaddr = (uint64_t)vaddr; - e_len = (size > buf_sz) ? buf_sz : size; - } - - to->u.s.len[i % 3] = (e_len); - to->ptr[i % 3] = (e_vaddr); - to->u.s.valid_segs = (i % 3) + 1; - - if (extra_len && (e_len >= extra_offset)) { - /* Break the data at given offset */ - uint32_t next_len = e_len - extra_offset; - uint64_t next_vaddr = e_vaddr + extra_offset; - - if (!extra_offset) { - i--; - } else { - e_len = extra_offset; - size -= e_len; - to->u.s.len[i % 3] = (e_len); - } - - extra_len = RTE_MIN(extra_len, size); - /* Insert extra data ptr */ - if (extra_len) { - i++; - to = &list[i / 3]; - to->u.s.len[i % 3] = (extra_len); - to->ptr[i % 3] = ((uint64_t)extra_buf->vaddr); - to->u.s.valid_segs = (i % 3) + 1; - size -= extra_len; - } - - next_len = RTE_MIN(next_len, size); - /* insert the rest of the data */ - if (next_len) { - i++; - to = &list[i / 3]; - to->u.s.len[i % 3] = (next_len); - to->ptr[i % 3] = (next_vaddr); - to->u.s.valid_segs = (i % 3) + 1; - size -= next_len; - } - extra_len = 0; - - } else { - size -= e_len; - } - if (extra_offset) - extra_offset -= size; - i++; - - if (unlikely(!size)) - break; - } - - *psize = size; - return (uint32_t)i; -} - static __rte_always_inline int sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t offset_ctrl, uint8_t *iv_s, int iv_len, uint8_t pack_iv, uint8_t pdcp_alg_type, int32_t inputlen, int32_t outputlen, uint32_t passthrough_len, uint32_t req_flags, int pdcp_flag, int decrypt) { + struct roc_sglist_comp *gather_comp, *scatter_comp; void *m_vaddr = params->meta_buf.vaddr; - struct roc_se_sglist_comp *gather_comp; - struct roc_se_sglist_comp *scatter_comp; struct roc_se_buf_ptr *aad_buf = NULL; uint32_t mac_len = 0, aad_len = 0; struct roc_se_ctx *se_ctx; @@ -485,7 +228,7 @@ sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t m_vaddr = (uint8_t *)m_vaddr + ROC_SE_OFF_CTRL_LEN + RTE_ALIGN_CEIL(iv_len, 8); - inst->w4.s.opcode_major |= (uint64_t)ROC_SE_DMA_MODE; + inst->w4.s.opcode_major |= (uint64_t)ROC_DMA_MODE_SG; /* iv offset is 0 */ *offset_vaddr = offset_ctrl; @@ -503,7 +246,7 @@ sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t /* DPTR has SG list */ /* TODO Add error check if space will be sufficient */ - gather_comp = (struct roc_se_sglist_comp *)((uint8_t *)m_vaddr + 8); + gather_comp = (struct roc_sglist_comp *)((uint8_t *)m_vaddr + 8); /* * Input Gather List @@ -562,13 +305,13 @@ sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t ((uint16_t *)in_buffer)[1] = 0; ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); - g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_se_sglist_comp); + g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); /* * Output Scatter List */ i = 0; - scatter_comp = (struct roc_se_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); + scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); if (zsk_flags == 0x1) { /* IV in SLIST only for EEA3 & UEA2 or for F8 */ @@ -626,9 +369,9 @@ sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t } } ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); - s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_se_sglist_comp); + s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); - size = g_size_bytes + s_size_bytes + ROC_SE_SG_LIST_HDR_SIZE; + size = g_size_bytes + s_size_bytes + ROC_SG_LIST_HDR_SIZE; /* This is DPTR len in case of SG mode */ inst->w4.s.dlen = size; @@ -643,18 +386,17 @@ sg2_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t int32_t outputlen, uint32_t passthrough_len, uint32_t req_flags, int pdcp_flag, int decrypt) { + struct roc_sg2list_comp *gather_comp, *scatter_comp; void *m_vaddr = params->meta_buf.vaddr; - uint32_t i, g_size_bytes; - struct roc_se_sg2list_comp *gather_comp; - struct roc_se_sg2list_comp *scatter_comp; struct roc_se_buf_ptr *aad_buf = NULL; + uint32_t mac_len = 0, aad_len = 0; + union cpt_inst_w5 cpt_inst_w5; + union cpt_inst_w6 cpt_inst_w6; struct roc_se_ctx *se_ctx; + uint32_t i, g_size_bytes; uint64_t *offset_vaddr; - uint32_t mac_len = 0, aad_len = 0; int zsk_flags; uint32_t size; - union cpt_inst_w5 cpt_inst_w5; - union cpt_inst_w6 cpt_inst_w6; uint8_t *iv_d; se_ctx = params->ctx; @@ -672,7 +414,7 @@ sg2_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t m_vaddr = (uint8_t *)m_vaddr + ROC_SE_OFF_CTRL_LEN + RTE_ALIGN_CEIL(iv_len, 8); - inst->w4.s.opcode_major |= (uint64_t)ROC_SE_DMA_MODE; + inst->w4.s.opcode_major |= (uint64_t)ROC_DMA_MODE_SG; /* iv offset is 0 */ *offset_vaddr = offset_ctrl; @@ -689,7 +431,7 @@ sg2_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t /* DPTR has SG list */ /* TODO Add error check if space will be sufficient */ - gather_comp = (struct roc_se_sg2list_comp *)((uint8_t *)m_vaddr); + gather_comp = (struct roc_sg2list_comp *)((uint8_t *)m_vaddr); /* * Input Gather List @@ -746,13 +488,13 @@ sg2_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t cpt_inst_w5.s.gather_sz = ((i + 2) / 3); - g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_se_sg2list_comp); + g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_sg2list_comp); /* * Output Scatter List */ i = 0; - scatter_comp = (struct roc_se_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); + scatter_comp = (struct roc_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); if (zsk_flags == 0x1) { /* IV in SLIST only for EEA3 & UEA2 or for F8 */ @@ -829,16 +571,15 @@ static __rte_always_inline int cpt_digest_gen_sg_ver1_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_params *params, struct cpt_inst_s *inst) { + struct roc_sglist_comp *gather_comp, *scatter_comp; void *m_vaddr = params->meta_buf.vaddr; - uint32_t size, i; + uint32_t g_size_bytes, s_size_bytes; uint16_t data_len, mac_len, key_len; + union cpt_inst_w4 cpt_inst_w4; roc_se_auth_type hash_type; struct roc_se_ctx *ctx; - struct roc_se_sglist_comp *gather_comp; - struct roc_se_sglist_comp *scatter_comp; uint8_t *in_buffer; - uint32_t g_size_bytes, s_size_bytes; - union cpt_inst_w4 cpt_inst_w4; + uint32_t size, i; ctx = params->ctx; @@ -851,13 +592,11 @@ cpt_digest_gen_sg_ver1_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa cpt_inst_w4.s.opcode_minor = 0; cpt_inst_w4.s.param2 = ((uint16_t)hash_type << 8) | mac_len; if (ctx->hmac) { - cpt_inst_w4.s.opcode_major = - ROC_SE_MAJOR_OP_HMAC | ROC_SE_DMA_MODE; + cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_HMAC | ROC_DMA_MODE_SG; cpt_inst_w4.s.param1 = key_len; cpt_inst_w4.s.dlen = data_len + RTE_ALIGN_CEIL(key_len, 8); } else { - cpt_inst_w4.s.opcode_major = - ROC_SE_MAJOR_OP_HASH | ROC_SE_DMA_MODE; + cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_HASH | ROC_DMA_MODE_SG; cpt_inst_w4.s.param1 = 0; cpt_inst_w4.s.dlen = data_len; } @@ -878,7 +617,7 @@ cpt_digest_gen_sg_ver1_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa ((uint16_t *)in_buffer)[1] = 0; /* TODO Add error check if space will be sufficient */ - gather_comp = (struct roc_se_sglist_comp *)((uint8_t *)m_vaddr + 8); + gather_comp = (struct roc_sglist_comp *)((uint8_t *)m_vaddr + 8); /* * Input gather list @@ -901,15 +640,14 @@ cpt_digest_gen_sg_ver1_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa return -1; } ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); - g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_se_sglist_comp); + g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); /* * Output Gather list */ i = 0; - scatter_comp = (struct roc_se_sglist_comp *)((uint8_t *)gather_comp + - g_size_bytes); + scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); if (flags & ROC_SE_VALID_MAC_BUF) { if (unlikely(params->mac_buf.size < mac_len)) { @@ -932,9 +670,9 @@ cpt_digest_gen_sg_ver1_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa } ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); - s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_se_sglist_comp); + s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); - size = g_size_bytes + s_size_bytes + ROC_SE_SG_LIST_HDR_SIZE; + size = g_size_bytes + s_size_bytes + ROC_SG_LIST_HDR_SIZE; /* This is DPTR len in case of SG mode */ cpt_inst_w4.s.dlen = size; @@ -949,17 +687,16 @@ static __rte_always_inline int cpt_digest_gen_sg_ver2_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_params *params, struct cpt_inst_s *inst) { + struct roc_sg2list_comp *gather_comp, *scatter_comp; void *m_vaddr = params->meta_buf.vaddr; - uint32_t size, i; uint16_t data_len, mac_len, key_len; - roc_se_auth_type hash_type; - struct roc_se_ctx *ctx; - struct roc_se_sg2list_comp *gather_comp; - struct roc_se_sg2list_comp *scatter_comp; + union cpt_inst_w4 cpt_inst_w4; union cpt_inst_w5 cpt_inst_w5; union cpt_inst_w6 cpt_inst_w6; + roc_se_auth_type hash_type; + struct roc_se_ctx *ctx; uint32_t g_size_bytes; - union cpt_inst_w4 cpt_inst_w4; + uint32_t size, i; ctx = params->ctx; @@ -993,7 +730,7 @@ cpt_digest_gen_sg_ver2_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa /* DPTR has SG list */ /* TODO Add error check if space will be sufficient */ - gather_comp = (struct roc_se_sg2list_comp *)((uint8_t *)m_vaddr + 0); + gather_comp = (struct roc_sg2list_comp *)((uint8_t *)m_vaddr + 0); /* * Input gather list @@ -1016,14 +753,14 @@ cpt_digest_gen_sg_ver2_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa } cpt_inst_w5.s.gather_sz = ((i + 2) / 3); - g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_se_sg2list_comp); + g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_sg2list_comp); /* * Output Gather list */ i = 0; - scatter_comp = (struct roc_se_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); + scatter_comp = (struct roc_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); if (flags & ROC_SE_VALID_MAC_BUF) { if (unlikely(params->mac_buf.size < mac_len)) { @@ -1482,8 +1219,7 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, pdcp_iv_copy(iv_d, auth_iv, pdcp_auth_alg, pack_iv); } else { - - struct roc_se_sglist_comp *scatter_comp, *gather_comp; + struct roc_sglist_comp *scatter_comp, *gather_comp; void *m_vaddr = params->meta_buf.vaddr; uint32_t i, g_size_bytes, s_size_bytes; uint8_t *in_buffer; @@ -1494,7 +1230,7 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, m_vaddr = PLT_PTR_ADD(m_vaddr, ROC_SE_OFF_CTRL_LEN + RTE_ALIGN_CEIL(hdr_len, 8)); - cpt_inst_w4.s.opcode_major |= (uint64_t)ROC_SE_DMA_MODE; + cpt_inst_w4.s.opcode_major |= (uint64_t)ROC_DMA_MODE_SG; /* DPTR has SG list */ in_buffer = m_vaddr; @@ -1502,8 +1238,7 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, ((uint16_t *)in_buffer)[0] = 0; ((uint16_t *)in_buffer)[1] = 0; - gather_comp = - (struct roc_se_sglist_comp *)((uint8_t *)m_vaddr + 8); + gather_comp = (struct roc_sglist_comp *)((uint8_t *)m_vaddr + 8); /* Input Gather List */ i = 0; @@ -1536,15 +1271,14 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, } } ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); - g_size_bytes = - ((i + 3) / 4) * sizeof(struct roc_se_sglist_comp); + g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); /* * Output Scatter List */ i = 0; - scatter_comp = (struct roc_se_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); + scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); if ((hdr_len)) { i = fill_sg_comp(scatter_comp, i, @@ -1573,10 +1307,9 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, } ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); - s_size_bytes = - ((i + 3) / 4) * sizeof(struct roc_se_sglist_comp); + s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); - size = g_size_bytes + s_size_bytes + ROC_SE_SG_LIST_HDR_SIZE; + size = g_size_bytes + s_size_bytes + ROC_SG_LIST_HDR_SIZE; /* This is DPTR len in case of SG mode */ cpt_inst_w4.s.dlen = size; @@ -1800,7 +1533,7 @@ cpt_kasumi_enc_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, } } - cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_KASUMI | ROC_SE_DMA_MODE; + cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_KASUMI | ROC_DMA_MODE_SG; /* Indicate ECB/CBC, direction, CTX from CPTR, IV from DPTR */ cpt_inst_w4.s.opcode_minor = @@ -1841,11 +1574,11 @@ cpt_kasumi_dec_prep(uint64_t d_offs, uint64_t d_lens, struct roc_se_fc_params *p flags = se_ctx->zsk_flags; cpt_inst_w4.u64 = 0; - cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_KASUMI | ROC_SE_DMA_MODE; + cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_KASUMI | ROC_DMA_MODE_SG; /* indicates ECB/CBC, direction, ctx from cptr, iv from dptr */ - cpt_inst_w4.s.opcode_minor = ((1 << 6) | (se_ctx->k_ecb << 5) | - (dir << 4) | (0 << 3) | (flags & 0x7)); + cpt_inst_w4.s.opcode_minor = + ((1 << 6) | (se_ctx->k_ecb << 5) | (dir << 4) | (0 << 3) | (flags & 0x7)); /* * GP op header, lengths are expected in bits. @@ -2290,28 +2023,8 @@ fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) return 0; } -static __rte_always_inline void * -alloc_op_meta(struct roc_se_buf_ptr *buf, int32_t len, - struct rte_mempool *cpt_meta_pool, - struct cpt_inflight_req *infl_req) -{ - uint8_t *mdata; - - if (unlikely(rte_mempool_get(cpt_meta_pool, (void **)&mdata) < 0)) - return NULL; - - buf->vaddr = mdata; - buf->size = len; - - infl_req->mdata = mdata; - infl_req->op_flags |= CPT_OP_FLAGS_METABUF; - - return mdata; -} - static __rte_always_inline uint32_t -prepare_iov_from_pkt(struct rte_mbuf *pkt, struct roc_se_iov_ptr *iovec, - uint32_t start_offset) +prepare_iov_from_pkt(struct rte_mbuf *pkt, struct roc_se_iov_ptr *iovec, uint32_t start_offset) { uint16_t index = 0; void *seg_data = NULL; diff --git a/drivers/crypto/cnxk/cnxk_sg.h b/drivers/crypto/cnxk/cnxk_sg.h new file mode 100644 index 0000000000..1dfca261cf --- /dev/null +++ b/drivers/crypto/cnxk/cnxk_sg.h @@ -0,0 +1,273 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _CNXK_SG_H_ +#define _CNXK_SG_H_ + +static __rte_always_inline uint32_t +fill_sg_comp(struct roc_sglist_comp *list, uint32_t i, phys_addr_t dma_addr, uint32_t size) +{ + struct roc_sglist_comp *to = &list[i >> 2]; + + to->u.s.len[i % 4] = rte_cpu_to_be_16(size); + to->ptr[i % 4] = rte_cpu_to_be_64(dma_addr); + return ++i; +} + +static __rte_always_inline uint32_t +fill_sg_comp_from_buf(struct roc_sglist_comp *list, uint32_t i, struct roc_se_buf_ptr *from) +{ + struct roc_sglist_comp *to = &list[i >> 2]; + + to->u.s.len[i % 4] = rte_cpu_to_be_16(from->size); + to->ptr[i % 4] = rte_cpu_to_be_64((uint64_t)from->vaddr); + return ++i; +} + +static __rte_always_inline uint32_t +fill_sg_comp_from_buf_min(struct roc_sglist_comp *list, uint32_t i, struct roc_se_buf_ptr *from, + uint32_t *psize) +{ + struct roc_sglist_comp *to = &list[i >> 2]; + uint32_t size = *psize; + uint32_t e_len; + + e_len = RTE_MIN(from->size, size); + to->u.s.len[i % 4] = rte_cpu_to_be_16(e_len); + to->ptr[i % 4] = rte_cpu_to_be_64((uint64_t)from->vaddr); + *psize -= e_len; + return ++i; +} + +/* + * This fills the MC expected SGIO list + * from IOV given by user. + */ +static __rte_always_inline uint32_t +fill_sg_comp_from_iov(struct roc_sglist_comp *list, uint32_t i, struct roc_se_iov_ptr *from, + uint32_t from_offset, uint32_t *psize, struct roc_se_buf_ptr *extra_buf, + uint32_t extra_offset) +{ + uint32_t extra_len = extra_buf ? extra_buf->size : 0; + uint32_t size = *psize; + int32_t j; + + for (j = 0; j < from->buf_cnt; j++) { + struct roc_sglist_comp *to = &list[i >> 2]; + uint32_t buf_sz = from->bufs[j].size; + void *vaddr = from->bufs[j].vaddr; + uint64_t e_vaddr; + uint32_t e_len; + + if (unlikely(from_offset)) { + if (from_offset >= buf_sz) { + from_offset -= buf_sz; + continue; + } + e_vaddr = (uint64_t)vaddr + from_offset; + e_len = RTE_MIN((buf_sz - from_offset), size); + from_offset = 0; + } else { + e_vaddr = (uint64_t)vaddr; + e_len = RTE_MIN(buf_sz, size); + } + + to->u.s.len[i % 4] = rte_cpu_to_be_16(e_len); + to->ptr[i % 4] = rte_cpu_to_be_64(e_vaddr); + + if (extra_len && (e_len >= extra_offset)) { + /* Break the data at given offset */ + uint32_t next_len = e_len - extra_offset; + uint64_t next_vaddr = e_vaddr + extra_offset; + + if (!extra_offset) { + i--; + } else { + e_len = extra_offset; + size -= e_len; + to->u.s.len[i % 4] = rte_cpu_to_be_16(e_len); + } + + extra_len = RTE_MIN(extra_len, size); + /* Insert extra data ptr */ + if (extra_len) { + i++; + to = &list[i >> 2]; + to->u.s.len[i % 4] = rte_cpu_to_be_16(extra_len); + to->ptr[i % 4] = rte_cpu_to_be_64((uint64_t)extra_buf->vaddr); + size -= extra_len; + } + + next_len = RTE_MIN(next_len, size); + /* insert the rest of the data */ + if (next_len) { + i++; + to = &list[i >> 2]; + to->u.s.len[i % 4] = rte_cpu_to_be_16(next_len); + to->ptr[i % 4] = rte_cpu_to_be_64(next_vaddr); + size -= next_len; + } + extra_len = 0; + + } else { + size -= e_len; + } + if (extra_offset) + extra_offset -= size; + i++; + + if (unlikely(!size)) + break; + } + + *psize = size; + return (uint32_t)i; +} + +static __rte_always_inline uint32_t +fill_ipsec_sg_comp_from_pkt(struct roc_sglist_comp *list, uint32_t i, struct rte_mbuf *pkt) +{ + uint32_t buf_sz; + void *vaddr; + + while (unlikely(pkt != NULL)) { + struct roc_sglist_comp *to = &list[i >> 2]; + buf_sz = pkt->data_len; + vaddr = rte_pktmbuf_mtod(pkt, void *); + + to->u.s.len[i % 4] = rte_cpu_to_be_16(buf_sz); + to->ptr[i % 4] = rte_cpu_to_be_64((uint64_t)vaddr); + + pkt = pkt->next; + i++; + } + + return i; +} + +static __rte_always_inline uint32_t +fill_sg2_comp(struct roc_sg2list_comp *list, uint32_t i, phys_addr_t dma_addr, uint32_t size) +{ + struct roc_sg2list_comp *to = &list[i / 3]; + + to->u.s.len[i % 3] = (size); + to->ptr[i % 3] = (dma_addr); + to->u.s.valid_segs = (i % 3) + 1; + return ++i; +} + +static __rte_always_inline uint32_t +fill_sg2_comp_from_buf(struct roc_sg2list_comp *list, uint32_t i, struct roc_se_buf_ptr *from) +{ + struct roc_sg2list_comp *to = &list[i / 3]; + + to->u.s.len[i % 3] = (from->size); + to->ptr[i % 3] = ((uint64_t)from->vaddr); + to->u.s.valid_segs = (i % 3) + 1; + return ++i; +} + +static __rte_always_inline uint32_t +fill_sg2_comp_from_buf_min(struct roc_sg2list_comp *list, uint32_t i, struct roc_se_buf_ptr *from, + uint32_t *psize) +{ + struct roc_sg2list_comp *to = &list[i / 3]; + uint32_t size = *psize; + uint32_t e_len; + + e_len = RTE_MIN(from->size, size); + to->u.s.len[i % 3] = (e_len); + to->ptr[i % 3] = ((uint64_t)from->vaddr); + to->u.s.valid_segs = (i % 3) + 1; + *psize -= e_len; + return ++i; +} + +static __rte_always_inline uint32_t +fill_sg2_comp_from_iov(struct roc_sg2list_comp *list, uint32_t i, struct roc_se_iov_ptr *from, + uint32_t from_offset, uint32_t *psize, struct roc_se_buf_ptr *extra_buf, + uint32_t extra_offset) +{ + uint32_t extra_len = extra_buf ? extra_buf->size : 0; + uint32_t size = *psize; + int32_t j; + + rte_prefetch2(psize); + + for (j = 0; j < from->buf_cnt; j++) { + struct roc_sg2list_comp *to = &list[i / 3]; + uint32_t buf_sz = from->bufs[j].size; + void *vaddr = from->bufs[j].vaddr; + uint64_t e_vaddr; + uint32_t e_len; + + if (unlikely(from_offset)) { + if (from_offset >= buf_sz) { + from_offset -= buf_sz; + continue; + } + e_vaddr = (uint64_t)vaddr + from_offset; + e_len = RTE_MIN((buf_sz - from_offset), size); + from_offset = 0; + } else { + e_vaddr = (uint64_t)vaddr; + e_len = RTE_MIN(buf_sz, size); + } + + to->u.s.len[i % 3] = (e_len); + to->ptr[i % 3] = (e_vaddr); + to->u.s.valid_segs = (i % 3) + 1; + + if (extra_len && (e_len >= extra_offset)) { + /* Break the data at given offset */ + uint32_t next_len = e_len - extra_offset; + uint64_t next_vaddr = e_vaddr + extra_offset; + + if (!extra_offset) { + i--; + } else { + e_len = extra_offset; + size -= e_len; + to->u.s.len[i % 3] = (e_len); + } + + extra_len = RTE_MIN(extra_len, size); + /* Insert extra data ptr */ + if (extra_len) { + i++; + to = &list[i / 3]; + to->u.s.len[i % 3] = (extra_len); + to->ptr[i % 3] = ((uint64_t)extra_buf->vaddr); + to->u.s.valid_segs = (i % 3) + 1; + size -= extra_len; + } + + next_len = RTE_MIN(next_len, size); + /* insert the rest of the data */ + if (next_len) { + i++; + to = &list[i / 3]; + to->u.s.len[i % 3] = (next_len); + to->ptr[i % 3] = (next_vaddr); + to->u.s.valid_segs = (i % 3) + 1; + size -= next_len; + } + extra_len = 0; + + } else { + size -= e_len; + } + if (extra_offset) + extra_offset -= size; + i++; + + if (unlikely(!size)) + break; + } + + *psize = size; + return (uint32_t)i; +} + +#endif /*_CNXK_SG_H_ */ From patchwork Tue Dec 20 14:32:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121084 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2E59CA0545; Tue, 20 Dec 2022 15:33:53 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4BA2342D42; Tue, 20 Dec 2022 15:33:08 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 0EC7942D3F for ; Tue, 20 Dec 2022 15:33:05 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEJKMW010037 for ; Tue, 20 Dec 2022 06:33:05 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=K+orMziC3az4mBxr1xO1P0lX9KkFpMk4bnUFnYtdiBc=; b=eLhX7QyCzNQcPrlhvjkmXohEeATMa0E5avT0ti/eFVGQDexWQKYuBaQFTMrRlXZdsTkW zeEMb8z/CTU/GYIkAYkfEP3ht3aWn2N45WEQ2DpKQ7vhD9vhEFDgkCmvVX63DBP0At9V RRl1guBSbOluWEFcP0/sMB7suE2NiP3yn5wU8bnjwRJ1U7bhlx1Qmw36fbrIcgv28XN3 HQQxSKpfofIDCcD7wB3gaa1Ig18fzOn8Vrsk1zhN1gk3+iGTxNqrxtIKJ/iiVl4BNzm4 ES755WzFev6XxUZeFhiHZCR3ddK5XW4A7Rz7U0uEtyxtVYgkZw6fcLwffwwl3UiK4KLX 6A== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5m-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:05 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:33:04 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:33:03 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id C5FA33F7070; Tue, 20 Dec 2022 06:33:01 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Volodymyr Fialko , Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Aakash Sasidharan , Subject: [PATCH 10/17] crypto/cnxk: add support for SHA3 hash Date: Tue, 20 Dec 2022 20:02:25 +0530 Message-ID: <20221220143232.2519650-11-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: KYGjNIr-W83enf03OXMN4Ro-MFGuFMEA X-Proofpoint-ORIG-GUID: KYGjNIr-W83enf03OXMN4Ro-MFGuFMEA X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Add support for SHA3 family hash and hmac operations. Signed-off-by: Volodymyr Fialko --- doc/guides/cryptodevs/cnxk.rst | 8 + doc/guides/cryptodevs/features/cn10k.ini | 8 + doc/guides/cryptodevs/features/cn9k.ini | 8 + drivers/crypto/cnxk/cnxk_cryptodev.h | 2 +- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 164 ++++++++++++++++++ drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 2 +- drivers/crypto/cnxk/cnxk_se.h | 33 +++- 7 files changed, 222 insertions(+), 3 deletions(-) diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst index baf0e3c4fd..9b01e04e5f 100644 --- a/doc/guides/cryptodevs/cnxk.rst +++ b/doc/guides/cryptodevs/cnxk.rst @@ -59,6 +59,14 @@ Hash algorithms: * ``RTE_CRYPTO_AUTH_SHA384_HMAC`` * ``RTE_CRYPTO_AUTH_SHA512`` * ``RTE_CRYPTO_AUTH_SHA512_HMAC`` +* ``RTE_CRYPTO_AUTH_SHA3_224`` +* ``RTE_CRYPTO_AUTH_SHA3_224_HMAC`` +* ``RTE_CRYPTO_AUTH_SHA3_256`` +* ``RTE_CRYPTO_AUTH_SHA3_256_HMAC`` +* ``RTE_CRYPTO_AUTH_SHA3_384`` +* ``RTE_CRYPTO_AUTH_SHA3_384_HMAC`` +* ``RTE_CRYPTO_AUTH_SHA3_512`` +* ``RTE_CRYPTO_AUTH_SHA3_512_HMAC`` * ``RTE_CRYPTO_AUTH_SNOW3G_UIA2`` * ``RTE_CRYPTO_AUTH_ZUC_EIA3`` * ``RTE_CRYPTO_AUTH_AES_CMAC`` diff --git a/doc/guides/cryptodevs/features/cn10k.ini b/doc/guides/cryptodevs/features/cn10k.ini index 6e4e0e0095..44b61663fc 100644 --- a/doc/guides/cryptodevs/features/cn10k.ini +++ b/doc/guides/cryptodevs/features/cn10k.ini @@ -63,6 +63,14 @@ ZUC EIA3 = Y AES CMAC (128) = Y AES CMAC (192) = Y AES CMAC (256) = Y +SHA3_224 = Y +SHA3_224 HMAC = Y +SHA3_256 = Y +SHA3_256 HMAC = Y +SHA3_384 = Y +SHA3_384 HMAC = Y +SHA3_512 = Y +SHA3_512 HMAC = Y ; ; Supported AEAD algorithms of 'cn10k' crypto driver. diff --git a/doc/guides/cryptodevs/features/cn9k.ini b/doc/guides/cryptodevs/features/cn9k.ini index f9c896f6bd..e7b287db26 100644 --- a/doc/guides/cryptodevs/features/cn9k.ini +++ b/doc/guides/cryptodevs/features/cn9k.ini @@ -64,6 +64,14 @@ ZUC EIA3 = Y AES CMAC (128) = Y AES CMAC (192) = Y AES CMAC (256) = Y +SHA3_224 = Y +SHA3_224 HMAC = Y +SHA3_256 = Y +SHA3_256 HMAC = Y +SHA3_384 = Y +SHA3_384 HMAC = Y +SHA3_512 = Y +SHA3_512 HMAC = Y ; ; Supported AEAD algorithms of 'cn9k' crypto driver. diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index 48bd6e144c..8241ee67d0 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -10,7 +10,7 @@ #include "roc_cpt.h" -#define CNXK_CPT_MAX_CAPS 37 +#define CNXK_CPT_MAX_CAPS 45 #define CNXK_SEC_CRYPTO_MAX_CAPS 16 #define CNXK_SEC_MAX_CAPS 9 #define CNXK_AE_EC_ID_MAX 8 diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index 9dfbf875ec..b2197a12be 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -337,6 +337,169 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = { }, }; +static const struct rte_cryptodev_capabilities caps_sha3[] = { + { /* SHA3_224 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA3_224, + .block_size = 144, + .key_size = { + .min = 0, + .max = 0, + .increment = 0 + }, + .digest_size = { + .min = 1, + .max = 28, + .increment = 1 + }, + }, } + }, } + }, + { /* SHA3_224 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA3_224_HMAC, + .block_size = 144, + .key_size = { + .min = 1, + .max = 1024, + .increment = 1 + }, + .digest_size = { + .min = 1, + .max = 28, + .increment = 1 + }, + }, } + }, } + }, + { /* SHA3_256 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA3_256, + .block_size = 136, + .key_size = { + .min = 0, + .max = 0, + .increment = 0 + }, + .digest_size = { + .min = 1, + .max = 32, + .increment = 1 + }, + }, } + }, } + }, + { /* SHA3_256 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA3_256_HMAC, + .block_size = 136, + .key_size = { + .min = 1, + .max = 1024, + .increment = 1 + }, + .digest_size = { + .min = 1, + .max = 32, + .increment = 1 + }, + }, } + }, } + }, + { /* SHA3_384 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA3_384, + .block_size = 104, + .key_size = { + .min = 0, + .max = 0, + .increment = 0 + }, + .digest_size = { + .min = 1, + .max = 48, + .increment = 0 + }, + }, } + }, } + }, + { /* SHA3_384 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA3_384_HMAC, + .block_size = 104, + .key_size = { + .min = 1, + .max = 1024, + .increment = 1 + }, + .digest_size = { + .min = 1, + .max = 48, + .increment = 1 + }, + }, } + }, } + }, + { /* SHA3_512 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA3_512, + .block_size = 72, + .key_size = { + .min = 0, + .max = 0, + .increment = 0 + }, + .digest_size = { + .min = 1, + .max = 64, + .increment = 1 + }, + }, } + }, } + }, + { /* SHA3_512 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA3_512_HMAC, + .block_size = 72, + .key_size = { + .min = 1, + .max = 1024, + .increment = 1 + }, + .digest_size = { + .min = 1, + .max = 64, + .increment = 1 + }, + }, } + }, } + }, +}; + static const struct rte_cryptodev_capabilities caps_chacha20[] = { { /* Chacha20-Poly1305 */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, @@ -1265,6 +1428,7 @@ crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, mul); CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, sha1_sha2); + CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, sha3); CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, chacha20); CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, zuc_snow3g); CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, aes); diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index eb2ed0d103..92e8755671 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -627,7 +627,7 @@ cnxk_cpt_inst_w7_get(struct cnxk_se_sess *sess, struct roc_cpt *roc_cpt) inst_w7.s.cptr = (uint64_t)&sess->roc_se_ctx.se_ctx; /* Set the engine group */ - if (sess->zsk_flag || sess->aes_ctr_eea2) + if (sess->zsk_flag || sess->aes_ctr_eea2 || sess->is_sha3) inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_SE]; else inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_IE]; diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 88049ac431..092cdd88e7 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -39,6 +39,8 @@ struct cnxk_se_sess { uint16_t zs_auth : 4; uint16_t dp_thr_type : 8; uint16_t aad_length; + uint8_t is_sha3 : 1; + uint8_t rsvd : 7; uint8_t mac_len; uint8_t iv_length; uint8_t auth_iv_length; @@ -163,18 +165,26 @@ cpt_mac_len_verify(struct rte_crypto_auth_xform *auth) break; case RTE_CRYPTO_AUTH_SHA224: case RTE_CRYPTO_AUTH_SHA224_HMAC: + case RTE_CRYPTO_AUTH_SHA3_224: + case RTE_CRYPTO_AUTH_SHA3_224_HMAC: ret = (mac_len <= 28) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA256: case RTE_CRYPTO_AUTH_SHA256_HMAC: + case RTE_CRYPTO_AUTH_SHA3_256: + case RTE_CRYPTO_AUTH_SHA3_256_HMAC: ret = (mac_len <= 32) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA384: case RTE_CRYPTO_AUTH_SHA384_HMAC: + case RTE_CRYPTO_AUTH_SHA3_384: + case RTE_CRYPTO_AUTH_SHA3_384_HMAC: ret = (mac_len <= 48) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA512: case RTE_CRYPTO_AUTH_SHA512_HMAC: + case RTE_CRYPTO_AUTH_SHA3_512: + case RTE_CRYPTO_AUTH_SHA3_512_HMAC: ret = (mac_len <= 64) ? 0 : -1; break; case RTE_CRYPTO_AUTH_NULL: @@ -1848,7 +1858,7 @@ fill_sess_cipher(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) static __rte_always_inline int fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) { - uint8_t zsk_flag = 0, zs_auth = 0, aes_gcm = 0, is_null = 0; + uint8_t zsk_flag = 0, zs_auth = 0, aes_gcm = 0, is_null = 0, is_sha3 = 0; struct rte_crypto_auth_xform *a_form; roc_se_auth_type auth_type = 0; /* NULL Auth type */ @@ -1900,6 +1910,26 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) case RTE_CRYPTO_AUTH_SHA384: auth_type = ROC_SE_SHA2_SHA384; break; + case RTE_CRYPTO_AUTH_SHA3_224_HMAC: + case RTE_CRYPTO_AUTH_SHA3_224: + is_sha3 = 1; + auth_type = ROC_SE_SHA3_SHA224; + break; + case RTE_CRYPTO_AUTH_SHA3_256_HMAC: + case RTE_CRYPTO_AUTH_SHA3_256: + is_sha3 = 1; + auth_type = ROC_SE_SHA3_SHA256; + break; + case RTE_CRYPTO_AUTH_SHA3_384_HMAC: + case RTE_CRYPTO_AUTH_SHA3_384: + is_sha3 = 1; + auth_type = ROC_SE_SHA3_SHA384; + break; + case RTE_CRYPTO_AUTH_SHA3_512_HMAC: + case RTE_CRYPTO_AUTH_SHA3_512: + is_sha3 = 1; + auth_type = ROC_SE_SHA3_SHA512; + break; case RTE_CRYPTO_AUTH_MD5_HMAC: case RTE_CRYPTO_AUTH_MD5: auth_type = ROC_SE_MD5_TYPE; @@ -1959,6 +1989,7 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) sess->aes_gcm = aes_gcm; sess->mac_len = a_form->digest_length; sess->is_null = is_null; + sess->is_sha3 = is_sha3; if (zsk_flag) { sess->auth_iv_offset = a_form->iv.offset; sess->auth_iv_length = a_form->iv.length; From patchwork Tue Dec 20 14:32:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121085 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 074E1A0545; Tue, 20 Dec 2022 15:34:02 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 74B7B42D4D; Tue, 20 Dec 2022 15:33:10 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A552C42D47 for ; Tue, 20 Dec 2022 15:33:09 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEN2CW016597 for ; Tue, 20 Dec 2022 06:33:09 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=DFognEOZ37pLd91l+YgjfTEyIJ/KAyVs0g59TIEcNSs=; b=KcsHrBRpiJBj9I+vm395MO0oty+1QEnmPtnRteJk6ftFMVIV2Wc7SV7XeRjOAybAQzLI kWdlSKV8pg15mYGdQUy+WYNBH2m8AxjVUQdYD4JK/cOKLpWg7pSQZ0LjyfeujUnMXxrx K9NYvg8kOmxwB3qPNOpd91GoDTs6Z0sW7goEe5capAhn2XbzUxYliqjD5gA8x7f+l5p9 UtG1UyzZP8pu+art5o3ZlE8kYWPS1Vo1sNxZot/j0UrqnJ9AX/2PMeqRcdL1BIHDx+Si 70Oo3VQ8yhraWrIsoZUcS2R1pJA7GG3N8sKJtIddGmq67f4KquqlAp3NW4ESDA1SMqOx fw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:08 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:33:06 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:33:06 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 73FE43F7063; Tue, 20 Dec 2022 06:33:04 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Volodymyr Fialko , Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Aakash Sasidharan , Subject: [PATCH 11/17] common/cnxk: skip hmac hash precomputation Date: Tue, 20 Dec 2022 20:02:26 +0530 Message-ID: <20221220143232.2519650-12-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: pul0wjJlbcRBhTL-b8YoVfuXZOSYcf2R X-Proofpoint-ORIG-GUID: pul0wjJlbcRBhTL-b8YoVfuXZOSYcf2R X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Operations with FC opcode requires precomputed ipad and opad hashes, but for auth only (HMAC opcode) this is not required, thus could be skipped. Signed-off-by: Volodymyr Fialko --- drivers/common/cnxk/roc_se.c | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/drivers/common/cnxk/roc_se.c b/drivers/common/cnxk/roc_se.c index 22df61f5f0..aba7f9416d 100644 --- a/drivers/common/cnxk/roc_se.c +++ b/drivers/common/cnxk/roc_se.c @@ -308,6 +308,7 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, struct roc_se_context *fctx; uint8_t opcode_minor; uint8_t pdcp_alg; + bool chained_op; int ret; if (se_ctx == NULL) @@ -318,12 +319,12 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, k_ctx = &se_ctx->se_ctx.k_ctx; fctx = &se_ctx->se_ctx.fctx; + chained_op = se_ctx->ciph_then_auth || se_ctx->auth_then_ciph; + if ((type >= ROC_SE_ZUC_EIA3) && (type <= ROC_SE_KASUMI_F9_ECB)) { uint8_t *zuc_const; uint32_t keyx[4]; uint8_t *ci_key; - bool chained_op = - se_ctx->ciph_then_auth || se_ctx->auth_then_ciph; if (!key_len) return -1; @@ -470,19 +471,25 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, se_ctx->mac_len = mac_len; if (key_len) { - se_ctx->hmac = 1; - - se_ctx->auth_key = plt_zmalloc(key_len, 8); - if (se_ctx->auth_key == NULL) - return -1; + /* + * Chained operation (FC opcode) requires precomputed ipad and opad hashes, but for + * auth only (HMAC opcode) this is not required + */ + if (chained_op) { + memset(fctx->hmac.ipad, 0, sizeof(fctx->hmac.ipad)); + memset(fctx->hmac.opad, 0, sizeof(fctx->hmac.opad)); + cpt_hmac_opad_ipad_gen(type, key, key_len, &fctx->hmac); + fctx->enc.auth_input_type = 0; + } else { + se_ctx->hmac = 1; - memcpy(se_ctx->auth_key, key, key_len); - se_ctx->auth_key_len = key_len; - memset(fctx->hmac.ipad, 0, sizeof(fctx->hmac.ipad)); - memset(fctx->hmac.opad, 0, sizeof(fctx->hmac.opad)); + se_ctx->auth_key = plt_zmalloc(key_len, 8); + if (se_ctx->auth_key == NULL) + return -1; - cpt_hmac_opad_ipad_gen(type, key, key_len, &fctx->hmac); - fctx->enc.auth_input_type = 0; + memcpy(se_ctx->auth_key, key, key_len); + se_ctx->auth_key_len = key_len; + } } return 0; } From patchwork Tue Dec 20 14:32:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121086 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E7550A0545; Tue, 20 Dec 2022 15:34:07 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 94A1342D56; Tue, 20 Dec 2022 15:33:13 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 734F642D47 for ; Tue, 20 Dec 2022 15:33:12 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEOVhI019007 for ; Tue, 20 Dec 2022 06:33:11 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=W1LY99sygzDd49+Vbeqe2mXOauUw7TCBGZWva1zjg88=; b=Eg97wOuDOC9kYNk4BGzzbvyos1bPGMgbgYLTEilKIrZ+dkJSdF0D5ZBYWo5NbAhEcGfN 71j0N59BNGRC2P52tukH++BxUaNZ1GilebfVQIPhqpKnqc/ZBRQmuN8qY+Hrhs+1LPJE 3NlbCDlUQVuLA7bPG+JnTTtnth+Z12EhbbV7zNkHoSv26CB8l7UpyyIIuI79A0E9LOm6 jL2onJXEg7w3dPpQlYuVWoSizDPj+k4kMwfBb9BoRKW+3XN2TnZ9iWZU9TPUCTJYC0Vn wVGSOjKQk1v3RXpNcNQc8zS8y63rCKg3h+JzoiNmo1O+FRIWTB3ABhQIGyHUNBPlYtNi EQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:11 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:33:09 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:33:09 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 223F53F7070; Tue, 20 Dec 2022 06:33:06 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , "Aakash Sasidharan" , Subject: [PATCH 12/17] crypto/octeontx: support truncated digest size Date: Tue, 20 Dec 2022 20:02:27 +0530 Message-ID: <20221220143232.2519650-13-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: JTI9hHhDCJYnf_GQbllPfrAr64HQgHoo X-Proofpoint-ORIG-GUID: JTI9hHhDCJYnf_GQbllPfrAr64HQgHoo X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Support truncated digest size for auth only mode. Signed-off-by: Tejasree Kondoj --- drivers/common/cpt/cpt_ucode.h | 17 ++++--- .../octeontx/otx_cryptodev_capabilities.c | 48 +++++++++---------- 2 files changed, 34 insertions(+), 31 deletions(-) diff --git a/drivers/common/cpt/cpt_ucode.h b/drivers/common/cpt/cpt_ucode.h index 22aabab6ac..b393be4cf6 100644 --- a/drivers/common/cpt/cpt_ucode.h +++ b/drivers/common/cpt/cpt_ucode.h @@ -41,30 +41,33 @@ cpt_mac_len_verify(struct rte_crypto_auth_xform *auth) uint16_t mac_len = auth->digest_length; int ret; + if ((auth->algo != RTE_CRYPTO_AUTH_NULL) && (mac_len == 0)) + return -1; + switch (auth->algo) { case RTE_CRYPTO_AUTH_MD5: case RTE_CRYPTO_AUTH_MD5_HMAC: - ret = (mac_len == 16) ? 0 : -1; + ret = (mac_len <= 16) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA1: case RTE_CRYPTO_AUTH_SHA1_HMAC: - ret = (mac_len == 20) ? 0 : -1; + ret = (mac_len <= 20) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA224: case RTE_CRYPTO_AUTH_SHA224_HMAC: - ret = (mac_len == 28) ? 0 : -1; + ret = (mac_len <= 28) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA256: case RTE_CRYPTO_AUTH_SHA256_HMAC: - ret = (mac_len == 32) ? 0 : -1; + ret = (mac_len <= 32) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA384: case RTE_CRYPTO_AUTH_SHA384_HMAC: - ret = (mac_len == 48) ? 0 : -1; + ret = (mac_len <= 48) ? 0 : -1; break; case RTE_CRYPTO_AUTH_SHA512: case RTE_CRYPTO_AUTH_SHA512_HMAC: - ret = (mac_len == 64) ? 0 : -1; + ret = (mac_len <= 64) ? 0 : -1; break; case RTE_CRYPTO_AUTH_NULL: ret = 0; @@ -523,7 +526,7 @@ cpt_digest_gen_prep(uint32_t flags, /*GP op header */ vq_cmd_w0.s.opcode.minor = 0; - vq_cmd_w0.s.param2 = ((uint16_t)hash_type << 8); + vq_cmd_w0.s.param2 = ((uint16_t)hash_type << 8) | mac_len; if (ctx->hmac) { vq_cmd_w0.s.opcode.major = CPT_MAJOR_OP_HMAC | CPT_DMA_MODE; vq_cmd_w0.s.param1 = key_len; diff --git a/drivers/crypto/octeontx/otx_cryptodev_capabilities.c b/drivers/crypto/octeontx/otx_cryptodev_capabilities.c index 3f734b232c..80a9fe2123 100644 --- a/drivers/crypto/octeontx/otx_cryptodev_capabilities.c +++ b/drivers/crypto/octeontx/otx_cryptodev_capabilities.c @@ -86,9 +86,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 0 }, .digest_size = { - .min = 16, + .min = 1, .max = 16, - .increment = 0 + .increment = 1 }, }, } }, } @@ -106,9 +106,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 8 }, .digest_size = { - .min = 16, + .min = 1, .max = 16, - .increment = 0 + .increment = 1 }, }, } }, } @@ -126,9 +126,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 0 }, .digest_size = { - .min = 20, + .min = 1, .max = 20, - .increment = 0 + .increment = 1 }, }, } }, } @@ -146,9 +146,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 1 }, .digest_size = { - .min = 20, + .min = 1, .max = 20, - .increment = 0 + .increment = 1 }, }, } }, } @@ -166,9 +166,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 0 }, .digest_size = { - .min = 28, + .min = 1, .max = 28, - .increment = 0 + .increment = 1 }, }, } }, } @@ -186,9 +186,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 1 }, .digest_size = { - .min = 28, + .min = 1, .max = 28, - .increment = 0 + .increment = 1 }, }, } }, } @@ -206,9 +206,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 0 }, .digest_size = { - .min = 32, + .min = 1, .max = 32, - .increment = 0 + .increment = 1 }, }, } }, } @@ -226,9 +226,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 1 }, .digest_size = { - .min = 32, + .min = 1, .max = 32, - .increment = 0 + .increment = 1 }, }, } }, } @@ -246,9 +246,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 0 }, .digest_size = { - .min = 48, + .min = 1, .max = 48, - .increment = 0 + .increment = 1 }, }, } }, } @@ -266,9 +266,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 1 }, .digest_size = { - .min = 48, + .min = 1, .max = 48, - .increment = 0 + .increment = 1 }, }, } }, } @@ -286,9 +286,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 0 }, .digest_size = { - .min = 64, + .min = 1, .max = 64, - .increment = 0 + .increment = 1 }, }, } }, } @@ -306,9 +306,9 @@ static const struct rte_cryptodev_capabilities otx_sym_capabilities[] = { .increment = 1 }, .digest_size = { - .min = 64, + .min = 1, .max = 64, - .increment = 0 + .increment = 1 }, }, } }, } From patchwork Tue Dec 20 14:32:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121087 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AB3D9A0545; Tue, 20 Dec 2022 15:34:13 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8482A42D4B; Tue, 20 Dec 2022 15:33:16 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id DBE7640E6E for ; Tue, 20 Dec 2022 15:33:14 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEOQ0u018991 for ; Tue, 20 Dec 2022 06:33:14 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=DvWtWsokIokPQHKPnU0MZWPRROifP3e4zhZP7aQGT+Q=; b=fiMLZrk9jv5a57oAf11D7jzz1HfkXJDoMRrtIuiIH76JUha7JyT4aBZBrVbhx0e1Iyad mSZ0/eFcHsY5R0nLKnmO0G8cdHR+clyvzpxJDzb6aHhYtJL5fvE/lwfEfsXVbDcoW2Rq dlBG9zPRI3Q6D6kdNOTdcWJo6JC76AdZialO3jPiOw6lS08iLCAQNyS0iYEBSPnxJj3b +IN+iAQHDUQubfBMLcQyNuiY6MMmXqt8QDzbvl/VvBKENx6BzUkIKeqqiAFDsKAaP1XK +ELDcGJhLT9tqhNqX/+g0AeYEbjdRlu4/JaogDXAw7MRRm+nrHdlu9L6DORreK/u5Luj Og== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:14 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:33:12 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:33:12 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id C4E5E3F704C; Tue, 20 Dec 2022 06:33:09 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , "Aakash Sasidharan" , Subject: [PATCH 13/17] crypto/cnxk: set device ops to null in PCI remove Date: Tue, 20 Dec 2022 20:02:28 +0530 Message-ID: <20221220143232.2519650-14-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: di12-j_Mkd-HjCybripa0dr__YY2o1sp X-Proofpoint-ORIG-GUID: di12-j_Mkd-HjCybripa0dr__YY2o1sp X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Setting dev_ops to NULL in pci_remove to avoid device close after removal. Signed-off-by: Tejasree Kondoj --- drivers/crypto/cnxk/cn10k_cryptodev.c | 1 + drivers/crypto/cnxk/cn9k_cryptodev.c | 1 + 2 files changed, 2 insertions(+) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev.c b/drivers/crypto/cnxk/cn10k_cryptodev.c index 52de9b9657..2fd4df3c5d 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev.c @@ -138,6 +138,7 @@ cn10k_cpt_pci_remove(struct rte_pci_device *pci_dev) cnxk_crypto_sec_ctx_destroy(dev); if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + dev->dev_ops = NULL; vf = dev->data->dev_private; ret = roc_cpt_dev_fini(&vf->cpt); if (ret) diff --git a/drivers/crypto/cnxk/cn9k_cryptodev.c b/drivers/crypto/cnxk/cn9k_cryptodev.c index 4cfc1f2150..47b0874185 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev.c @@ -138,6 +138,7 @@ cn9k_cpt_pci_remove(struct rte_pci_device *pci_dev) cnxk_crypto_sec_ctx_destroy(dev); if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + dev->dev_ops = NULL; vf = dev->data->dev_private; ret = roc_cpt_dev_fini(&vf->cpt); if (ret) From patchwork Tue Dec 20 14:32:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121088 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C3175A0545; Tue, 20 Dec 2022 15:34:20 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BA13642D5D; Tue, 20 Dec 2022 15:33:19 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 12FCC40E6E for ; Tue, 20 Dec 2022 15:33:17 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEObIW019239 for ; Tue, 20 Dec 2022 06:33:17 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=KAA8k9xQ9vy67kwSIx2WyUl+mVAr3Syy5+01rE2uHfs=; b=Gqd9dqcOR9cDnDy7CYSiPGx917LHMpXsWtdduadqWoYExqLdEstAO09RxnNje5syHWBZ yo1LYBDRRybd7FBRl6cMj6sOq+j1Z2hM+5/3eOBxoW2EM3oaEjNxGFOr5TtYKCzljBEr NbmrXXPj6Pb/F0Frm0H87aBQEBCjUVYAzVWn/uO76fLYkCqabKg0X08w3HFKJgorp+mR j7ghjwsLggJUnDNLhyYpdf7ajgOwkiaa3a6Y08TRgVBQat7UDmhO7u1OsloXJI17/zsL h4lSkj634kP+akh7mjoRFZcPV2HI1qSuejdFrhSnIAZEs/oKs/f3DgTNsayoavDQ/PGj Ew== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:17 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:33:14 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:33:14 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 7DECC3F7063; Tue, 20 Dec 2022 06:33:12 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , "Aakash Sasidharan" , Subject: [PATCH 14/17] crypto/cnxk: add CTX for non IPsec operations Date: Tue, 20 Dec 2022 20:02:29 +0530 Message-ID: <20221220143232.2519650-15-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: rveCv-FwVBHqeZUcoTOG_kpUjNa-ZoSU X-Proofpoint-ORIG-GUID: rveCv-FwVBHqeZUcoTOG_kpUjNa-ZoSU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Support context cache with non IPsec operations. Signed-off-by: Tejasree Kondoj Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_platform.h | 3 +- drivers/common/cnxk/roc_se.c | 47 +++++++++++++++++++++++ drivers/common/cnxk/roc_se.h | 42 +++++++++++++------- drivers/common/cnxk/version.map | 1 + drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 17 ++++---- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 14 +++---- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 38 +++++++++++++----- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 13 +++---- drivers/crypto/cnxk/cnxk_se.h | 10 +++-- 9 files changed, 134 insertions(+), 51 deletions(-) diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 1a48ff3db4..8ba28e69fa 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -57,9 +57,10 @@ #define PLT_ALIGN RTE_ALIGN #define PLT_ALIGN_MUL_CEIL RTE_ALIGN_MUL_CEIL #define PLT_MODEL_MZ_NAME "roc_model_mz" -#define PLT_CACHE_LINE_SIZE RTE_CACHE_LINE_SIZE +#define PLT_CACHE_LINE_SIZE RTE_CACHE_LINE_SIZE #define BITMASK_ULL GENMASK_ULL #define PLT_ALIGN_CEIL RTE_ALIGN_CEIL +#define PLT_ALIGN_FLOOR RTE_ALIGN_FLOOR #define PLT_INIT RTE_INIT #define PLT_MAX_ETHPORTS RTE_MAX_ETHPORTS #define PLT_TAILQ_FOREACH_SAFE RTE_TAILQ_FOREACH_SAFE diff --git a/drivers/common/cnxk/roc_se.c b/drivers/common/cnxk/roc_se.c index aba7f9416d..8c19c5fccc 100644 --- a/drivers/common/cnxk/roc_se.c +++ b/drivers/common/cnxk/roc_se.c @@ -726,3 +726,50 @@ roc_se_ctx_swap(struct roc_se_ctx *se_ctx) zs_ctx->zuc.otk_ctx.w0.u64 = htobe64(zs_ctx->zuc.otk_ctx.w0.u64); } + +void +roc_se_ctx_init(struct roc_se_ctx *roc_se_ctx) +{ + struct se_ctx_s *ctx = &roc_se_ctx->se_ctx; + uint64_t ctx_len, *uc_ctx; + uint8_t i; + + switch (roc_se_ctx->fc_type) { + case ROC_SE_FC_GEN: + ctx_len = sizeof(struct roc_se_context); + break; + case ROC_SE_PDCP: + ctx_len = sizeof(struct roc_se_zuc_snow3g_ctx); + break; + case ROC_SE_KASUMI: + ctx_len = sizeof(struct roc_se_kasumi_ctx); + break; + case ROC_SE_PDCP_CHAIN: + ctx_len = sizeof(struct roc_se_zuc_snow3g_chain_ctx); + break; + default: + ctx_len = 0; + } + + ctx_len = PLT_ALIGN_CEIL(ctx_len, 8); + + /* Skip w0 for swap */ + uc_ctx = PLT_PTR_ADD(ctx, sizeof(ctx->w0)); + for (i = 0; i < (ctx_len / 8); i++) + uc_ctx[i] = plt_cpu_to_be_64(((uint64_t *)uc_ctx)[i]); + + /* Include w0 */ + ctx_len += sizeof(ctx->w0); + ctx_len = PLT_ALIGN_CEIL(ctx_len, 8); + + ctx->w0.s.aop_valid = 1; + ctx->w0.s.ctx_hdr_size = 0; + + ctx->w0.s.ctx_size = PLT_ALIGN_FLOOR(ctx_len, 128); + if (ctx->w0.s.ctx_size == 0) + ctx->w0.s.ctx_size = 1; + + ctx->w0.s.ctx_push_size = ctx_len / 8; + if (ctx->w0.s.ctx_push_size > 32) + ctx->w0.s.ctx_push_size = 32; +} diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index a8f0f49479..d1ef71577c 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -300,14 +300,31 @@ struct roc_se_ctx { uint64_t rsvd : 17; union cpt_inst_w4 template_w4; /* Below fields are accessed by hardware */ - union { - struct roc_se_context fctx; - struct roc_se_zuc_snow3g_ctx zs_ctx; - struct roc_se_zuc_snow3g_chain_ctx zs_ch_ctx; - struct roc_se_kasumi_ctx k_ctx; - } se_ctx; + struct se_ctx_s { + /* Word0 */ + union { + struct { + uint64_t rsvd : 48; + + uint64_t ctx_push_size : 7; + uint64_t rsvd1 : 1; + + uint64_t ctx_hdr_size : 2; + uint64_t aop_valid : 1; + uint64_t rsvd2 : 1; + uint64_t ctx_size : 4; + } s; + uint64_t u64; + } w0; + union { + struct roc_se_context fctx; + struct roc_se_zuc_snow3g_ctx zs_ctx; + struct roc_se_zuc_snow3g_chain_ctx zs_ch_ctx; + struct roc_se_kasumi_ctx k_ctx; + }; + } se_ctx __plt_aligned(ROC_ALIGN); uint8_t *auth_key; -}; +} __plt_aligned(ROC_ALIGN); struct roc_se_fc_params { union { @@ -349,14 +366,13 @@ roc_se_zuc_bytes_swap(uint8_t *arr, int len) } } -int __roc_api roc_se_auth_key_set(struct roc_se_ctx *se_ctx, - roc_se_auth_type type, const uint8_t *key, - uint16_t key_len, uint16_t mac_len); +int __roc_api roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, + const uint8_t *key, uint16_t key_len, uint16_t mac_len); -int __roc_api roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, - roc_se_cipher_type type, const uint8_t *key, - uint16_t key_len, uint8_t *salt); +int __roc_api roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, + const uint8_t *key, uint16_t key_len, uint8_t *salt); void __roc_api roc_se_ctx_swap(struct roc_se_ctx *se_ctx); +void __roc_api roc_se_ctx_init(struct roc_se_ctx *se_ctx); #endif /* __ROC_SE_H__ */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 17f0ec6b48..ee283d2392 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -99,6 +99,7 @@ INTERNAL { roc_model; roc_se_auth_key_set; roc_se_ciph_key_set; + roc_se_ctx_init; roc_nix_bpf_alloc; roc_nix_bpf_config; roc_nix_bpf_connect; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 5a098ffcf2..57b44210c2 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -52,14 +52,13 @@ cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op) if (rte_mempool_get(qp->sess_mp, (void **)&sess) < 0) return NULL; - ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform, - sess); + ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform, sess, true); if (ret) { rte_mempool_put(qp->sess_mp, (void *)sess); goto sess_put; } - priv = (void *)sess->driver_priv_data; + priv = (void *)sess; sym_op->session = sess; return priv; @@ -121,13 +120,13 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], struct if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { - sec_sess = (struct cn10k_sec_session *)(sym_op->session); + sec_sess = (struct cn10k_sec_session *)sym_op->session; ret = cpt_sec_inst_fill(qp, op, sec_sess, &inst[0]); if (unlikely(ret)) return 0; w7 = sec_sess->inst.w7; } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { - sess = CRYPTODEV_GET_SYM_SESS_PRIV(sym_op->session); + sess = (struct cnxk_se_sess *)(sym_op->session); ret = cpt_sym_inst_fill(qp, op, sess, infl_req, &inst[0], is_sg_ver2); if (unlikely(ret)) return 0; @@ -141,7 +140,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], struct ret = cpt_sym_inst_fill(qp, op, sess, infl_req, &inst[0], is_sg_ver2); if (unlikely(ret)) { - sym_session_clear(op->sym->session); + sym_session_clear(op->sym->session, true); rte_mempool_put(qp->sess_mp, op->sym->session); return 0; } @@ -316,7 +315,7 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, vo } else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { struct cnxk_se_sess *priv; - priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess); + priv = (struct cnxk_se_sess *)sess; priv->qp = qp; priv->cpt_inst_w2 = w2; } else @@ -351,7 +350,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, struct cnxk_cpt_qp **qp, ui } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { struct cnxk_se_sess *priv; - priv = CRYPTODEV_GET_SYM_SESS_PRIV(op->sym->session); + priv = (struct cnxk_se_sess *)op->sym->session; *qp = priv->qp; *w2 = priv->cpt_inst_w2; } else { @@ -914,7 +913,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, temp_sess_free: if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { - sym_session_clear(cop->sym->session); + sym_session_clear(cop->sym->session, true); rte_mempool_put(qp->sess_mp, cop->sym->session); cop->sym->session = NULL; } diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index cfe1e08dff..e3784e34c9 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -49,11 +49,11 @@ cn9k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op) if (rte_mempool_get(qp->sess_mp, (void **)&sess) < 0) return NULL; - ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform, sess); + ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform, sess, true); if (ret) goto sess_put; - priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess); + priv = (struct cnxk_se_sess *)sess; sym_op->session = sess; @@ -76,7 +76,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { sym_op = op->sym; - sess = CRYPTODEV_GET_SYM_SESS_PRIV(sym_op->session); + sess = (struct cnxk_se_sess *)sym_op->session; ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst, false); inst->w7.u64 = sess->cpt_inst_w7; } else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) @@ -90,7 +90,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst, false); if (unlikely(ret)) { - sym_session_clear(op->sym->session); + sym_session_clear(op->sym->session, true); rte_mempool_put(qp->sess_mp, op->sym->session); } inst->w7.u64 = sess->cpt_inst_w7; @@ -326,7 +326,7 @@ cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, } else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { struct cnxk_se_sess *priv; - priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess); + priv = (struct cnxk_se_sess *)sess; priv->qp = qp; priv->cpt_inst_w2 = w2; } else @@ -361,7 +361,7 @@ cn9k_ca_meta_info_extract(struct rte_crypto_op *op, } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { struct cnxk_se_sess *priv; - priv = CRYPTODEV_GET_SYM_SESS_PRIV(op->sym->session); + priv = (struct cnxk_se_sess *)op->sym->session; *qp = priv->qp; inst->w2.u64 = priv->cpt_inst_w2; } else { @@ -630,7 +630,7 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, temp_sess_free: if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { - sym_session_clear(cop->sym->session); + sym_session_clear(cop->sym->session, true); rte_mempool_put(qp->sess_mp, cop->sym->session); cop->sym->session = NULL; } diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 92e8755671..2e845afce9 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -626,6 +626,11 @@ cnxk_cpt_inst_w7_get(struct cnxk_se_sess *sess, struct roc_cpt *roc_cpt) inst_w7.s.cptr = (uint64_t)&sess->roc_se_ctx.se_ctx; + if (roc_cpt->cpt_revision == ROC_CPT_REVISION_ID_106XX) + inst_w7.s.ctx_val = 1; + else + inst_w7.s.cptr += 8; + /* Set the engine group */ if (sess->zsk_flag || sess->aes_ctr_eea2 || sess->is_sha3) inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_SE]; @@ -636,19 +641,22 @@ cnxk_cpt_inst_w7_get(struct cnxk_se_sess *sess, struct roc_cpt *roc_cpt) } int -sym_session_configure(struct roc_cpt *roc_cpt, - struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess) +sym_session_configure(struct roc_cpt *roc_cpt, struct rte_crypto_sym_xform *xform, + struct rte_cryptodev_sym_session *sess, bool is_session_less) { enum cpt_dp_thread_type thr_type; - struct cnxk_se_sess *sess_priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess); + struct cnxk_se_sess *sess_priv = (struct cnxk_se_sess *)sess; int ret; - memset(sess_priv, 0, sizeof(struct cnxk_se_sess)); + if (is_session_less) + memset(sess_priv, 0, sizeof(struct cnxk_se_sess)); + ret = cnxk_sess_fill(roc_cpt, xform, sess_priv); if (ret) goto priv_put; + sess_priv->lf = roc_cpt->lf[0]; + if (sess_priv->cpt_op & ROC_SE_OP_CIPHER_MASK) { switch (sess_priv->roc_se_ctx.fc_type) { case ROC_SE_FC_GEN: @@ -690,6 +698,10 @@ sym_session_configure(struct roc_cpt *roc_cpt, } sess_priv->cpt_inst_w7 = cnxk_cpt_inst_w7_get(sess_priv, roc_cpt); + + if (roc_cpt->cpt_revision == ROC_CPT_REVISION_ID_106XX) + roc_se_ctx_init(&sess_priv->roc_se_ctx); + return 0; priv_put: @@ -704,25 +716,31 @@ cnxk_cpt_sym_session_configure(struct rte_cryptodev *dev, struct cnxk_cpt_vf *vf = dev->data->dev_private; struct roc_cpt *roc_cpt = &vf->cpt; - return sym_session_configure(roc_cpt, xform, sess); + return sym_session_configure(roc_cpt, xform, sess, false); } void -sym_session_clear(struct rte_cryptodev_sym_session *sess) +sym_session_clear(struct rte_cryptodev_sym_session *sess, bool is_session_less) { - struct cnxk_se_sess *sess_priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess); + struct cnxk_se_sess *sess_priv = (struct cnxk_se_sess *)sess; + + /* Trigger CTX flush + invalidate to remove from CTX_CACHE */ + roc_cpt_lf_ctx_flush(sess_priv->lf, &sess_priv->roc_se_ctx.se_ctx, true); + + plt_delay_ms(1); if (sess_priv->roc_se_ctx.auth_key != NULL) plt_free(sess_priv->roc_se_ctx.auth_key); - memset(sess_priv, 0, cnxk_cpt_sym_session_get_size(NULL)); + if (is_session_less) + memset(sess_priv, 0, cnxk_cpt_sym_session_get_size(NULL)); } void cnxk_cpt_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { - return sym_session_clear(sess); + return sym_session_clear(sess, false); } unsigned int diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h index 5153d334ba..f91ad368ea 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h @@ -107,18 +107,15 @@ int cnxk_cpt_queue_pair_release(struct rte_cryptodev *dev, uint16_t qp_id); unsigned int cnxk_cpt_sym_session_get_size(struct rte_cryptodev *dev); -int cnxk_cpt_sym_session_configure(struct rte_cryptodev *dev, - struct rte_crypto_sym_xform *xform, +int cnxk_cpt_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, struct rte_cryptodev_sym_session *sess); -int sym_session_configure(struct roc_cpt *roc_cpt, - struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess); +int sym_session_configure(struct roc_cpt *roc_cpt, struct rte_crypto_sym_xform *xform, + struct rte_cryptodev_sym_session *sess, bool is_session_less); -void cnxk_cpt_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess); +void cnxk_cpt_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess); -void sym_session_clear(struct rte_cryptodev_sym_session *sess); +void sym_session_clear(struct rte_cryptodev_sym_session *sess, bool is_session_less); unsigned int cnxk_ae_session_size_get(struct rte_cryptodev *dev __rte_unused); diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 092cdd88e7..774efd5879 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -6,6 +6,8 @@ #define _CNXK_SE_H_ #include +#include + #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_ops.h" #include "cnxk_sg.h" @@ -25,6 +27,7 @@ enum cpt_dp_thread_type { }; struct cnxk_se_sess { + struct rte_cryptodev_sym_session rte_sess; uint16_t cpt_op : 4; uint16_t zsk_flag : 4; uint16_t aes_gcm : 1; @@ -51,10 +54,11 @@ struct cnxk_se_sess { uint64_t cpt_inst_w2; struct cnxk_cpt_qp *qp; struct roc_se_ctx roc_se_ctx; -} __rte_cache_aligned; + struct roc_cpt_lf *lf; +} __rte_aligned(ROC_ALIGN); -static __rte_always_inline int -fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess); +static __rte_always_inline int fill_sess_gmac(struct rte_crypto_sym_xform *xform, + struct cnxk_se_sess *sess); static inline void cpt_pack_iv(uint8_t *iv_src, uint8_t *iv_dst) From patchwork Tue Dec 20 14:32:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121089 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EFBBFA0545; Tue, 20 Dec 2022 15:34:26 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B61CB42D54; Tue, 20 Dec 2022 15:33:23 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 2A17142D22 for ; Tue, 20 Dec 2022 15:33:22 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEJKMX010037 for ; Tue, 20 Dec 2022 06:33:21 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=vkID/JDKKwAv1W8a8jA7fnVPN0UVCgQ6hByeROoLmKs=; b=XbFL6TasWco87spS3axydpgTcydQgt4kKu1HxnjgGjQMLps1haqQ0GUXbbKRLubMy6hT 9Chcp2d+wH4NxMTmEYNbUsGj5Jat0T5c60P9JPlvjrvJLT5d7Ul9v2x4ILPjb1cLXIf5 xob3lE8JOyosWW6tdbA1cUc4/FiCvSzblDtZ9NzYiFxeS53oCIk6SWmdxHtnyDsQ1sRM VL/gKdbW6f1HBNyuipdc7AiyCLhKJgxNpV0zQ1j+xaBH4v6Q67ic6mUZDkSzEIUm38C8 Fr1pSG/b2oE94jdp6hRwnoaR5GNUFbJLzX+u6ayfB2FveY0LxiWh+xoE9rtkiuyHZA+O ow== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:21 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:33:19 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:33:19 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 2D1743F709D; Tue, 20 Dec 2022 06:33:14 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , "Aakash Sasidharan" , Subject: [PATCH 15/17] crypto/cnxk: set salt in dptr as part of IV Date: Tue, 20 Dec 2022 20:02:30 +0530 Message-ID: <20221220143232.2519650-16-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: Un-tI7xt3lL0kuDPqARrQJgpNgk0ymGS X-Proofpoint-ORIG-GUID: Un-tI7xt3lL0kuDPqARrQJgpNgk0ymGS X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Set salt as part of 16B IV in dptr to avoid race condition. Signed-off-by: Tejasree Kondoj --- drivers/common/cnxk/roc_se.c | 7 +++- drivers/crypto/cnxk/cnxk_se.h | 76 +++++++++++++++++++---------------- 2 files changed, 47 insertions(+), 36 deletions(-) diff --git a/drivers/common/cnxk/roc_se.c b/drivers/common/cnxk/roc_se.c index 8c19c5fccc..f335c2367f 100644 --- a/drivers/common/cnxk/roc_se.c +++ b/drivers/common/cnxk/roc_se.c @@ -462,8 +462,10 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, return -1; /* For GMAC auth, cipher must be NULL */ - if (type == ROC_SE_GMAC_TYPE) + if (type == ROC_SE_GMAC_TYPE) { fctx->enc.enc_cipher = 0; + se_ctx->template_w4.s.opcode_minor = BIT(5); + } fctx->enc.hash_type = type; se_ctx->hash_type = type; @@ -530,6 +532,9 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, return 0; } + if (type == ROC_SE_AES_GCM) + se_ctx->template_w4.s.opcode_minor = BIT(5); + ret = cpt_ciph_type_set(type, se_ctx, key_len); if (unlikely(ret)) return -1; diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 774efd5879..1827b0d7b3 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -43,7 +43,8 @@ struct cnxk_se_sess { uint16_t dp_thr_type : 8; uint16_t aad_length; uint8_t is_sha3 : 1; - uint8_t rsvd : 7; + uint8_t short_iv : 1; + uint8_t rsvd : 6; uint8_t mac_len; uint8_t iv_length; uint8_t auth_iv_length; @@ -201,13 +202,6 @@ cpt_mac_len_verify(struct rte_crypto_auth_xform *auth) return ret; } -static __rte_always_inline void -cpt_fc_salt_update(struct roc_se_ctx *se_ctx, uint8_t *salt) -{ - struct roc_se_context *fctx = &se_ctx->se_ctx.fctx; - memcpy(fctx->enc.encr_iv, salt, 4); -} - static __rte_always_inline int sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t offset_ctrl, uint8_t *iv_s, int iv_len, uint8_t pack_iv, uint8_t pdcp_alg_type, int32_t inputlen, @@ -1703,8 +1697,17 @@ fill_sess_aead(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) sess->iv_length = aead_form->iv.length; sess->aad_length = aead_form->aad_length; - if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, - aead_form->key.data, + switch (sess->iv_length) { + case 12: + sess->short_iv = 1; + case 16: + break; + default: + plt_dp_err("Crypto: Unsupported IV length %u", sess->iv_length); + return -1; + } + + if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, aead_form->key.data, aead_form->key.length, NULL))) return -1; @@ -1712,6 +1715,8 @@ fill_sess_aead(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) aead_form->digest_length))) return -1; + if (enc_type == ROC_SE_CHACHA20) + sess->roc_se_ctx.template_w4.s.opcode_minor |= BIT(5); return 0; } @@ -1849,9 +1854,19 @@ fill_sess_cipher(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) sess->iv_length = c_form->iv.length; sess->is_null = is_null; - if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, - c_form->key.data, c_form->key.length, - NULL))) + if (aes_ctr) + switch (sess->iv_length) { + case 12: + sess->short_iv = 1; + case 16: + break; + default: + plt_dp_err("Crypto: Unsupported IV length %u", sess->iv_length); + return -1; + } + + if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, c_form->key.data, + c_form->key.length, NULL))) return -1; if ((enc_type >= ROC_SE_ZUC_EEA3) && (enc_type <= ROC_SE_AES_CTR_EEA2)) @@ -2046,9 +2061,18 @@ fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) sess->iv_length = a_form->iv.length; sess->mac_len = a_form->digest_length; - if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, - a_form->key.data, a_form->key.length, - NULL))) + switch (sess->iv_length) { + case 12: + sess->short_iv = 1; + case 16: + break; + default: + plt_dp_err("Crypto: Unsupported IV length %u", sess->iv_length); + return -1; + } + + if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, a_form->key.data, + a_form->key.length, NULL))) return -1; if (unlikely(roc_se_auth_key_set(&sess->roc_se_ctx, auth_type, NULL, 0, @@ -2192,7 +2216,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, if (likely(is_kasumi || sess->iv_length)) { flags |= ROC_SE_VALID_IV_BUF; fc_params.iv_buf = rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset); - if (!is_aead && sess->aes_ctr && unlikely(sess->iv_length != 16)) { + if (sess->short_iv) { memcpy((uint8_t *)iv_buf, rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset), 12); iv_buf[3] = rte_cpu_to_be_32(0x1); @@ -2209,7 +2233,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, if (is_aead) { struct rte_mbuf *m; - uint8_t *salt; uint8_t *aad_data; uint16_t aad_len; @@ -2234,13 +2257,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, d_lens = d_lens << 32; } - salt = fc_params.iv_buf; - if (unlikely(*(uint32_t *)salt != sess->salt)) { - cpt_fc_salt_update(&sess->roc_se_ctx, salt); - sess->salt = *(uint32_t *)salt; - } - - fc_params.iv_buf = PLT_PTR_ADD(salt, 4); m = cpt_m_dst_get(cpt_op, m_src, m_dst); /* Digest immediately following data is best case */ @@ -2266,16 +2282,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, d_lens = ci_data_length; d_lens = (d_lens << 32) | a_data_length; - /* for gmac, salt should be updated like in gcm */ - if (unlikely(sess->is_gmac)) { - uint8_t *salt; - salt = fc_params.iv_buf; - if (unlikely(*(uint32_t *)salt != sess->salt)) { - cpt_fc_salt_update(&sess->roc_se_ctx, salt); - sess->salt = *(uint32_t *)salt; - } - fc_params.iv_buf = salt + 4; - } if (likely(sess->mac_len)) { struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); From patchwork Tue Dec 20 14:32:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121090 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C7641A0545; Tue, 20 Dec 2022 15:34:32 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6237842D64; Tue, 20 Dec 2022 15:33:24 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9E2B942D54 for ; Tue, 20 Dec 2022 15:33:22 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEJKMY010037 for ; Tue, 20 Dec 2022 06:33:22 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=ZnWUqESWfg3j8OH/9Uxn7QDeTnhzTtk+vtcE3DbB068=; b=gr32pyCaIlwFktu6LKI8sGlA6iWJpY1hcD4DV6jIRF5/CSVxRocVEtX9F7qqf1xC4yUr fVi3+5341tk3fVEQ6Utq3Mg4AGxqP9rk3ZGT+ETvfZKgSYnbw2DkroL4jGrdHyO6qRCi QWSfZ7ItbyHuc690sRBWDz9UBZCYuB2rzrE9PmCEjNy/5ub3STM02d1NXgjZpkTk6Phm mvCp1EyfQ+ouPLNly8XBw/A0yaHB+iOV55MzEU59O0n4Bln8eXnroM+s5r3G0mdZHylv MkDtIUlSKmycRrKglYVCteJ81s4EFxknKy++IvhSnz4DLApeU1QC3/eP7nKy1XOYvvun 0Q== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5v-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:21 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:33:20 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:33:20 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id CEE173F70AA; Tue, 20 Dec 2022 06:33:17 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , "Aakash Sasidharan" , Subject: [PATCH 16/17] crypto/cnxk: remove null check of session priv Date: Tue, 20 Dec 2022 20:02:31 +0530 Message-ID: <20221220143232.2519650-17-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: gpyijOUUPeqLUhT9gCiaHcwLb3KFJ8vl X-Proofpoint-ORIG-GUID: gpyijOUUPeqLUhT9gCiaHcwLb3KFJ8vl X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Session private data is at an offset of session. This can never be NULL. Remove check for the same. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_ipsec.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c index aafd461436..ffd3f50eed 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.c +++ b/drivers/crypto/cnxk/cn10k_ipsec.c @@ -422,14 +422,12 @@ cn10k_sec_session_update(void *device, struct rte_security_session *sess, struct rte_security_session_conf *conf) { struct rte_cryptodev *crypto_dev = device; - struct cn10k_sec_session *priv; struct roc_cpt *roc_cpt; struct cnxk_cpt_qp *qp; struct cnxk_cpt_vf *vf; int ret; - priv = SECURITY_GET_SESS_PRIV(sess); - if (priv == NULL) + if (sess == NULL) return -EINVAL; qp = crypto_dev->data->queue_pairs[0]; From patchwork Tue Dec 20 14:32:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 121091 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C7EA4A0545; Tue, 20 Dec 2022 15:34:40 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AE90142D69; Tue, 20 Dec 2022 15:33:27 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9A2A042D10 for ; Tue, 20 Dec 2022 15:33:25 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BKEOQ0v018991 for ; Tue, 20 Dec 2022 06:33:25 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=8Lw8WQ2XJMBqwPqEp2AgDi4khP9BeF8K4dNibNQI9Rk=; b=N5fGYhW6aBoJbzgiyWrg2zq2bk69g7hqe9wZ4yEu2SPKT7dE5cT5gM3jiu97p9hzTKrw G8CLhvHB0PoteSe7QPvje0oYcfqZLR1RYDagAp4oXGDkX4WUzmAtuyz4kDAofG96sb+n 2SpytkJO2vqO9kPxj0i9/MVSoPLA6+nrfI9fmAnRm9TcVN7aTCN3Mw0Cj0/PtrKbINHT jYTIsbPpuVT9nxvmPixCj+sFqT/B5Mx0EOgfJAWKEqRzU1xNeT/77TVPSV4K7dgmL1Qn +TXDKC1kERnU0AxKTwUrUrBIuCpPm9ZSEUmfXjb7TGyfehZ4s6qHjWjbribo8d6y7CkQ Ow== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3mhe5rnb5x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 20 Dec 2022 06:33:24 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 20 Dec 2022 06:33:22 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 20 Dec 2022 06:33:22 -0800 Received: from hyd1554.marvell.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 87C1C3F7070; Tue, 20 Dec 2022 06:33:20 -0800 (PST) From: Tejasree Kondoj To: Akhil Goyal CC: Anoob Joseph , Vidya Sagar Velumuri , Gowrishankar Muthukrishnan , Volodymyr Fialko , "Aakash Sasidharan" , Subject: [PATCH 17/17] common/cnxk: remove salt from session Date: Tue, 20 Dec 2022 20:02:32 +0530 Message-ID: <20221220143232.2519650-18-ktejasree@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221220143232.2519650-1-ktejasree@marvell.com> References: <20221220143232.2519650-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: nM0wSn8c5EVtB4bV9IbsjgdBK8cSgTpe X-Proofpoint-ORIG-GUID: nM0wSn8c5EVtB4bV9IbsjgdBK8cSgTpe X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-20_05,2022-12-20_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Complete 16B IV can be passed in DPTR to microcode. Passing salt in CPTR cannot be supported as the salt need to be given in different formats when CTX is enabled. Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_se.c | 17 +++-------------- drivers/common/cnxk/roc_se.h | 2 +- drivers/crypto/cnxk/cnxk_se.h | 6 +++--- 3 files changed, 7 insertions(+), 18 deletions(-) diff --git a/drivers/common/cnxk/roc_se.c b/drivers/common/cnxk/roc_se.c index f335c2367f..6f0a81b22c 100644 --- a/drivers/common/cnxk/roc_se.c +++ b/drivers/common/cnxk/roc_se.c @@ -497,8 +497,9 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type, } int -roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, - const uint8_t *key, uint16_t key_len, uint8_t *salt) +roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, const uint8_t *key, + uint16_t key_len) + { bool chained_op = se_ctx->ciph_then_auth || se_ctx->auth_then_ciph; struct roc_se_zuc_snow3g_ctx *zs_ctx = &se_ctx->se_ctx.zs_ctx; @@ -520,18 +521,6 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, zuc_const = zs_ctx->zuc.otk_ctx.zuc_const; } - /* For AES-GCM, salt is taken from ctx even if IV source - * is from DPTR - */ - if ((salt != NULL) && (type == ROC_SE_AES_GCM)) { - memcpy(fctx->enc.encr_iv, salt, 4); - /* Assuming it was just salt update - * and nothing else - */ - if (key == NULL) - return 0; - } - if (type == ROC_SE_AES_GCM) se_ctx->template_w4.s.opcode_minor = BIT(5); diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index d1ef71577c..e9415f21a5 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -370,7 +370,7 @@ int __roc_api roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type ty const uint8_t *key, uint16_t key_len, uint16_t mac_len); int __roc_api roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, - const uint8_t *key, uint16_t key_len, uint8_t *salt); + const uint8_t *key, uint16_t key_len); void __roc_api roc_se_ctx_swap(struct roc_se_ctx *se_ctx); void __roc_api roc_se_ctx_init(struct roc_se_ctx *se_ctx); diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 1827b0d7b3..0e5d2dde39 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -1708,7 +1708,7 @@ fill_sess_aead(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) } if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, aead_form->key.data, - aead_form->key.length, NULL))) + aead_form->key.length))) return -1; if (unlikely(roc_se_auth_key_set(&sess->roc_se_ctx, auth_type, NULL, 0, @@ -1866,7 +1866,7 @@ fill_sess_cipher(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) } if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, c_form->key.data, - c_form->key.length, NULL))) + c_form->key.length))) return -1; if ((enc_type >= ROC_SE_ZUC_EEA3) && (enc_type <= ROC_SE_AES_CTR_EEA2)) @@ -2072,7 +2072,7 @@ fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) } if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, a_form->key.data, - a_form->key.length, NULL))) + a_form->key.length))) return -1; if (unlikely(roc_se_auth_key_set(&sess->roc_se_ctx, auth_type, NULL, 0,