From patchwork Tue Aug 9 10:53:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114745 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 06A00A00C2; Tue, 9 Aug 2022 12:54:12 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CE06A4069C; Tue, 9 Aug 2022 12:54:06 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id AFC8140143 for ; Tue, 9 Aug 2022 12:54:05 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791GEhn015744 for ; Tue, 9 Aug 2022 03:54:04 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=OnIKocMc22pkSG+IkxP5G4WyVzBqGbZQm5JJM091grk=; b=A2NL0K3co13EQ2qWO0Gkh+lGiHr0hu5x8ESb1gZBRzRuTjDXv4ehOgisqa/euxcIcRvy fk4Gk5jNYDFTvSihIuUu5ZKMl7QQ/44DRLxg/QSOgFrkn2iVNN2Hz6xNERQcmPrdaDCd Wm23yoI+ps28PL/pgM3reDTG/Y/1nTdX87Z2ArGNEB7pkxFwSTQ/t5wIcnRO914P2MGr tSoP66yTa71H5ongqXvfZIf3/4ZS8Yz7YOnSjlEHpk93gv1LYEZe0xbGKhNR1YMxcKmt 34kHDtZtr9IOfP4LwDN0kQHXRPvIwdfE2iOBEVcKqU2rgnCBBjzr6ise4UvmM9aRhxDc 0A== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3huds2sq6c-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:04 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Aug 2022 03:54:01 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:01 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id B5CAD3F7048; Tue, 9 Aug 2022 03:53:59 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 01/18] crypto/cnxk: add AES-CCM support Date: Tue, 9 Aug 2022 16:23:39 +0530 Message-ID: <20220809105356.561-2-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: lUcCAAiOTJ7UUbP6JDRPaV0ksgVzOc-k X-Proofpoint-GUID: lUcCAAiOTJ7UUbP6JDRPaV0ksgVzOc-k X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Archana Muniganti Add lookaside IPsec AES-CCM support in CN9K & CN10K PMDs. Signed-off-by: Archana Muniganti --- doc/guides/rel_notes/release_22_11.rst | 4 ++ drivers/common/cnxk/cnxk_security.c | 38 ++++++++++++-- drivers/common/cnxk/roc_cpt.h | 13 ++--- drivers/crypto/cnxk/cn10k_ipsec_la_ops.h | 1 + drivers/crypto/cnxk/cnxk_cryptodev.h | 2 +- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 49 ++++++++++++++++--- drivers/crypto/cnxk/cnxk_ipsec.h | 3 +- 7 files changed, 93 insertions(+), 17 deletions(-) diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 8c021cf050..333f66bef3 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -55,6 +55,10 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Updated Marvell cnxk crypto driver.** + + * Added AES-CCM support in lookaside protocol (IPsec) for CN9K & CN10K. + Removed Items ------------- diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c index dca8742be3..8a0e4dea4c 100644 --- a/drivers/common/cnxk/cnxk_security.c +++ b/drivers/common/cnxk/cnxk_security.c @@ -58,6 +58,7 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, { struct rte_crypto_sym_xform *auth_xfrm, *cipher_xfrm; const uint8_t *key = NULL; + uint8_t ccm_flag = 0; uint32_t *tmp_salt; uint64_t *tmp_key; int i, length = 0; @@ -113,6 +114,15 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, tmp_salt = (uint32_t *)salt_key; *tmp_salt = rte_be_to_cpu_32(*tmp_salt); break; + case RTE_CRYPTO_AEAD_AES_CCM: + w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CCM; + w2->s.auth_type = ROC_IE_OT_SA_AUTH_NULL; + ccm_flag = 0x07 & ~ROC_CPT_AES_CCM_CTR_LEN; + *salt_key = ccm_flag; + memcpy(PLT_PTR_ADD(salt_key, 1), &ipsec_xfrm->salt, 3); + tmp_salt = (uint32_t *)salt_key; + *tmp_salt = rte_be_to_cpu_32(*tmp_salt); + break; default: return -ENOTSUP; } @@ -204,6 +214,7 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2, w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CCM || w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CTR || w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM || + w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CCM || w2->s.auth_type == ROC_IE_OT_SA_AUTH_AES_GMAC) { switch (length) { case ROC_CPT_AES128_KEY_LEN: @@ -612,6 +623,7 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt, struct rte_crypto_sym_xform *auth_xfrm, *cipher_xfrm; int rc, length, auth_key_len; const uint8_t *key = NULL; + uint8_t ccm_flag = 0; /* Set direction */ switch (ipsec_xfrm->direction) { @@ -663,6 +675,14 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt, memcpy(salt, &ipsec_xfrm->salt, 4); key = crypto_xfrm->aead.key.data; break; + case RTE_CRYPTO_AEAD_AES_CCM: + ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CCM; + ctl->auth_type = ROC_IE_ON_SA_AUTH_NULL; + ccm_flag = 0x07 & ~ROC_CPT_AES_CCM_CTR_LEN; + *salt = ccm_flag; + memcpy(PLT_PTR_ADD(salt, 1), &ipsec_xfrm->salt, 3); + key = crypto_xfrm->aead.key.data; + break; default: return -ENOTSUP; } @@ -810,7 +830,7 @@ cnxk_ipsec_ivlen_get(enum rte_crypto_cipher_algorithm c_algo, { uint8_t ivlen = 0; - if (aead_algo == RTE_CRYPTO_AEAD_AES_GCM) + if ((aead_algo == RTE_CRYPTO_AEAD_AES_GCM) || (aead_algo == RTE_CRYPTO_AEAD_AES_CCM)) ivlen = 8; switch (c_algo) { @@ -873,6 +893,7 @@ cnxk_ipsec_icvlen_get(enum rte_crypto_cipher_algorithm c_algo, switch (aead_algo) { case RTE_CRYPTO_AEAD_AES_GCM: + case RTE_CRYPTO_AEAD_AES_CCM: icv = 16; break; default: @@ -888,7 +909,7 @@ cnxk_ipsec_outb_roundup_byte(enum rte_crypto_cipher_algorithm c_algo, { uint8_t roundup_byte = 4; - if (aead_algo == RTE_CRYPTO_AEAD_AES_GCM) + if ((aead_algo == RTE_CRYPTO_AEAD_AES_GCM) || (aead_algo == RTE_CRYPTO_AEAD_AES_CCM)) return roundup_byte; switch (c_algo) { @@ -1023,6 +1044,10 @@ on_ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec, ctl->enc_type = ROC_IE_ON_SA_ENC_AES_GCM; aes_key_len = crypto_xform->aead.key.length; break; + case RTE_CRYPTO_AEAD_AES_CCM: + ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CCM; + aes_key_len = crypto_xform->aead.key.length; + break; default: plt_err("Unsupported AEAD algorithm"); return -ENOTSUP; @@ -1087,6 +1112,7 @@ on_ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec, ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CCM || ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CTR || ctl->enc_type == ROC_IE_ON_SA_ENC_AES_GCM || + ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CCM || ctl->auth_type == ROC_IE_ON_SA_AUTH_AES_GMAC) { switch (aes_key_len) { case 16: @@ -1129,6 +1155,7 @@ on_fill_ipsec_common_sa(struct rte_security_ipsec_xform *ipsec, struct rte_crypto_sym_xform *cipher_xform, *auth_xform; const uint8_t *cipher_key; int cipher_key_len = 0; + uint8_t ccm_flag = 0; int ret; ret = on_ipsec_sa_ctl_set(ipsec, crypto_xform, &common_sa->ctl); @@ -1146,6 +1173,11 @@ on_fill_ipsec_common_sa(struct rte_security_ipsec_xform *ipsec, if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) memcpy(common_sa->iv.gcm.nonce, &ipsec->salt, 4); + else if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_CCM) { + ccm_flag = 0x07 & ~ROC_CPT_AES_CCM_CTR_LEN; + *common_sa->iv.gcm.nonce = ccm_flag; + memcpy(PLT_PTR_ADD(common_sa->iv.gcm.nonce, 1), &ipsec->salt, 3); + } cipher_key = crypto_xform->aead.key.data; cipher_key_len = crypto_xform->aead.key.length; } else { @@ -1194,7 +1226,7 @@ cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec, return ret; if (ctl->enc_type == ROC_IE_ON_SA_ENC_AES_GCM || - ctl->auth_type == ROC_IE_ON_SA_AUTH_NULL || + ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CCM || ctl->auth_type == ROC_IE_ON_SA_AUTH_NULL || ctl->auth_type == ROC_IE_ON_SA_AUTH_AES_GMAC) { template = &out_sa->aes_gcm.template; ctx_len = offsetof(struct roc_ie_on_outb_sa, aes_gcm.template); diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h index a3a65f1e94..0cebc05c74 100644 --- a/drivers/common/cnxk/roc_cpt.h +++ b/drivers/common/cnxk/roc_cpt.h @@ -43,12 +43,13 @@ ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 14)) /* CPT helper macros */ -#define ROC_CPT_AH_HDR_LEN 12 -#define ROC_CPT_AES_GCM_IV_LEN 8 -#define ROC_CPT_AES_GCM_MAC_LEN 16 -#define ROC_CPT_AES_CBC_IV_LEN 16 -#define ROC_CPT_SHA1_HMAC_LEN 12 -#define ROC_CPT_SHA2_HMAC_LEN 16 +#define ROC_CPT_AH_HDR_LEN 12 +#define ROC_CPT_AES_GCM_IV_LEN 8 +#define ROC_CPT_AES_GCM_MAC_LEN 16 +#define ROC_CPT_AES_CCM_CTR_LEN 4 +#define ROC_CPT_AES_CBC_IV_LEN 16 +#define ROC_CPT_SHA1_HMAC_LEN 12 +#define ROC_CPT_SHA2_HMAC_LEN 16 #define ROC_CPT_DES3_KEY_LEN 24 #define ROC_CPT_AES128_KEY_LEN 16 diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h index 66cfe6ca98..e220863799 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h @@ -66,6 +66,7 @@ process_outb_sa(struct roc_cpt_lf *lf, struct rte_crypto_op *cop, #ifdef LA_IPSEC_DEBUG if (sess->out_sa.w2.s.iv_src == ROC_IE_OT_SA_IV_SRC_FROM_SA) { if (sess->out_sa.w2.s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM || + sess->out_sa.w2.s.enc_type == ROC_IE_OT_SA_ENC_AES_CCM || sess->out_sa.w2.s.auth_type == ROC_IE_OT_SA_AUTH_AES_GMAC) ipsec_po_sa_aes_gcm_iv_set(sess, cop); else diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index 8870021725..a3dcfbfa6d 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -11,7 +11,7 @@ #include "roc_cpt.h" #define CNXK_CPT_MAX_CAPS 35 -#define CNXK_SEC_CRYPTO_MAX_CAPS 13 +#define CNXK_SEC_CRYPTO_MAX_CAPS 14 #define CNXK_SEC_MAX_CAPS 9 #define CNXK_AE_EC_ID_MAX 8 /** diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index 705d67e91f..fdc646a6fc 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -775,6 +775,36 @@ static const struct rte_cryptodev_capabilities sec_caps_aes[] = { }, } }, } }, + { /* AES CCM */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_AES_CCM, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .digest_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .aad_size = { + .min = 8, + .max = 12, + .increment = 4 + }, + .iv_size = { + .min = 12, + .max = 12, + .increment = 0 + } + }, } + }, } + }, { /* AES CTR */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, {.sym = { @@ -1155,14 +1185,23 @@ cnxk_crypto_capabilities_get(struct cnxk_cpt_vf *vf) return vf->crypto_caps; } +static bool +sec_caps_limit_check(int *cur_pos, int nb_caps) +{ + if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS) { + rte_panic("Could not add sec crypto caps"); + return true; + } + + return false; +} + static void sec_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos, const struct rte_cryptodev_capabilities *caps, int nb_caps) { - if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS) { - rte_panic("Could not add sec crypto caps"); + if (sec_caps_limit_check(cur_pos, nb_caps)) return; - } memcpy(&cnxk_caps[*cur_pos], caps, nb_caps * sizeof(caps[0])); *cur_pos += nb_caps; @@ -1175,10 +1214,8 @@ cn10k_sec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[], const struct rte_cryptodev_capabilities *cap; unsigned int i; - if ((CNXK_SEC_CRYPTO_MAX_CAPS - *cur_pos) < 1) { - rte_panic("Could not add sec crypto caps"); + if (sec_caps_limit_check(cur_pos, 1)) return; - } /* NULL auth */ for (i = 0; i < RTE_DIM(caps_null); i++) { diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h index 07ab2cf4ee..00873ca6ac 100644 --- a/drivers/crypto/cnxk/cnxk_ipsec.h +++ b/drivers/crypto/cnxk/cnxk_ipsec.h @@ -87,7 +87,8 @@ ipsec_xform_aead_verify(struct rte_security_ipsec_xform *ipsec_xform, crypto_xform->aead.op != RTE_CRYPTO_AEAD_OP_DECRYPT) return -EINVAL; - if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) { + if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM || + crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_CCM) { switch (crypto_xform->aead.key.length) { case 16: case 24: From patchwork Tue Aug 9 10:53:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114746 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D0CFA00C5; Tue, 9 Aug 2022 12:54:18 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B1FF042BB1; Tue, 9 Aug 2022 12:54:09 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 1210942BAE for ; Tue, 9 Aug 2022 12:54:06 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791T39W017143 for ; Tue, 9 Aug 2022 03:54:06 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=2sEp2bU0Gn60Tb6nO1uddLqqiCn1mbPS5Z3qmxYEUY0=; b=YxJzSXxk0oV0Fm3x588jcHIn7fxaLX2KQyuGCdmJ8b/YpqFMf4DiiH0NzkgT9lCsArjE fGMH+rPr9yM73r8B0ml1O+n1NHfO/VGQpB3SeRdMYZxgFuBklSDLL1bAz/s+VL74fyZE iGHGWStQsGp6J9ppJC8dUsfYNGYqEkmIT/ZobicHORh9S/wVPW2GQOdJJ8yux4356vAs hg/dicnhp9VOLp4coOTw1KAGJHduVIHZxColntAnsz2SfzZZt4CfeMm42GTAU9agKlXd Fh2mAxP9equh4Y6jjGSTADXxvl7PfGD3gfGlxd7SLhpi9maC5y76d00fcFT1QWnZz081 Mg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hudy6spgk-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:06 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:04 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:04 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id 5C1B03F706A; Tue, 9 Aug 2022 03:54:02 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Volodymyr Fialko , Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 02/18] crypto/cnxk: add burst enqueue for event crypto Date: Tue, 9 Aug 2022 16:23:40 +0530 Message-ID: <20220809105356.561-3-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: VQrGWKKKJ33u4rtQ49mO83Oup1h22w3f X-Proofpoint-ORIG-GUID: VQrGWKKKJ33u4rtQ49mO83Oup1h22w3f X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Added support for burst enqueue for cn10k event crypto adapter. Instruction will be grouped based on the queue pair and sent in a burst. Signed-off-by: Volodymyr Fialko --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 141 +++++++++++++++------- drivers/crypto/cnxk/cn10k_cryptodev_ops.h | 7 +- drivers/crypto/cnxk/meson.build | 2 +- drivers/event/cnxk/cn10k_eventdev.c | 2 +- drivers/event/cnxk/cn10k_worker.c | 10 -- drivers/event/cnxk/cn10k_worker.h | 2 - 6 files changed, 105 insertions(+), 59 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index f761ba36e2..bfa6374005 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -9,11 +9,12 @@ #include "cn10k_cryptodev.h" #include "cn10k_cryptodev_ops.h" -#include "cn10k_ipsec_la_ops.h" #include "cn10k_ipsec.h" +#include "cn10k_ipsec_la_ops.h" #include "cnxk_ae.h" #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_ops.h" +#include "cnxk_eventdev.h" #include "cnxk_se.h" #include "roc_api.h" @@ -391,79 +392,135 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, return 0; } -uint16_t -cn10k_cpt_crypto_adapter_enqueue(uintptr_t base, struct rte_crypto_op *op) +static inline uint16_t +ca_lmtst_burst_submit(struct cn10k_sso_hws *ws, uint64_t w2[], struct cnxk_cpt_qp *qp, + struct rte_crypto_op *op[], uint16_t nb_ops) { + struct cpt_inflight_req *infl_reqs[PKTS_PER_LOOP]; + uint64_t lmt_base, lmt_arg, io_addr; + struct cpt_inst_s *inst, *inst_base; struct cpt_inflight_req *infl_req; - uint64_t lmt_base, lmt_arg, w2; - struct cpt_inst_s *inst; union cpt_fc_write_s fc; - struct cnxk_cpt_qp *qp; uint64_t *fc_addr; uint16_t lmt_id; - int ret; + int ret, i; - ret = cn10k_ca_meta_info_extract(op, &qp, &w2); - if (unlikely(ret)) { - rte_errno = EINVAL; - return 0; - } + lmt_base = qp->lmtline.lmt_base; + io_addr = qp->lmtline.io_addr; + fc_addr = qp->lmtline.fc_addr; + + const uint32_t fc_thresh = qp->lmtline.fc_thresh; + + ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); + inst_base = (struct cpt_inst_s *)lmt_base; if (unlikely(!qp->ca.enabled)) { rte_errno = EINVAL; return 0; } - if (unlikely(rte_mempool_get(qp->ca.req_mp, (void **)&infl_req))) { + if (unlikely(rte_mempool_get_bulk(qp->ca.req_mp, (void **)infl_reqs, nb_ops))) { rte_errno = ENOMEM; return 0; } - infl_req->op_flags = 0; - - lmt_base = qp->lmtline.lmt_base; - fc_addr = qp->lmtline.fc_addr; - const uint32_t fc_thresh = qp->lmtline.fc_thresh; + for (i = 0; i < nb_ops; i++) { + inst = &inst_base[2 * i]; + infl_req = infl_reqs[i]; + infl_req->op_flags = 0; - ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); - inst = (struct cpt_inst_s *)lmt_base; + ret = cn10k_cpt_fill_inst(qp, &op[i], inst, infl_req); + if (unlikely(ret != 1)) { + plt_dp_err("Could not process op: %p", op[i]); + if (i != 0) + goto submit; + else + goto put; + } - ret = cn10k_cpt_fill_inst(qp, &op, inst, infl_req); - if (unlikely(ret != 1)) { - plt_dp_err("Could not process op: %p", op); - rte_mempool_put(qp->ca.req_mp, infl_req); - return 0; + infl_req->res.cn10k.compcode = CPT_COMP_NOT_DONE; + infl_req->qp = qp; + inst->w0.u64 = 0; + inst->res_addr = (uint64_t)&infl_req->res; + inst->w2.u64 = w2[i]; + inst->w3.u64 = CNXK_CPT_INST_W3(1, infl_req); } - infl_req->cop = op; - infl_req->res.cn10k.compcode = CPT_COMP_NOT_DONE; - infl_req->qp = qp; - inst->w0.u64 = 0; - inst->res_addr = (uint64_t)&infl_req->res; - inst->w2.u64 = w2; - inst->w3.u64 = CNXK_CPT_INST_W3(1, infl_req); - fc.u64[0] = __atomic_load_n(fc_addr, __ATOMIC_RELAXED); if (unlikely(fc.s.qsize > fc_thresh)) { - rte_mempool_put(qp->ca.req_mp, infl_req); rte_errno = EAGAIN; - return 0; + i = 0; + goto put; } - if (inst->w2.s.tt == RTE_SCHED_TYPE_ORDERED) - roc_sso_hws_head_wait(base); +submit: + if (CNXK_TT_FROM_TAG(ws->gw_rdata) == SSO_TT_ORDERED) + roc_sso_hws_head_wait(ws->base); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, qp->lmtline.io_addr); + if (i > PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (PKTS_PER_STEORL - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, io_addr); + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + PKTS_PER_STEORL); + roc_lmt_submit_steorl(lmt_arg, io_addr); + } else { + lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, io_addr); + } rte_io_wmb(); - return 1; +put: + if (unlikely(i != nb_ops)) + rte_mempool_put_bulk(qp->ca.req_mp, (void *)&infl_reqs[i], nb_ops - i); + + return i; +} + +uint16_t __rte_hot +cn10k_cpt_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_events) +{ + struct rte_crypto_op *ops[PKTS_PER_LOOP], *op; + struct cnxk_cpt_qp *qp, *curr_qp = NULL; + uint64_t w2s[PKTS_PER_LOOP], w2; + uint16_t submitted, count = 0; + int ret, i, ops_len = 0; + + for (i = 0; i < nb_events; i++) { + op = ev[i].event_ptr; + ret = cn10k_ca_meta_info_extract(op, &qp, &w2); + if (unlikely(ret)) { + rte_errno = EINVAL; + return count; + } + + if (qp != curr_qp) { + if (ops_len) { + submitted = ca_lmtst_burst_submit(ws, w2s, curr_qp, ops, ops_len); + count += submitted; + if (unlikely(submitted != ops_len)) + return count; + ops_len = 0; + } + curr_qp = qp; + } + w2s[ops_len] = w2; + ops[ops_len] = op; + if (++ops_len == PKTS_PER_LOOP) { + submitted = ca_lmtst_burst_submit(ws, w2s, curr_qp, ops, ops_len); + count += submitted; + if (unlikely(submitted != ops_len)) + return count; + ops_len = 0; + } + } + if (ops_len) + count += ca_lmtst_burst_submit(ws, w2s, curr_qp, ops, ops_len); + return count; } static inline void -cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, - struct cpt_cn10k_res_s *res) +cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) { struct rte_mbuf *mbuf = cop->sym->m_src; const uint16_t m_len = res->rlen; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h index 1ad4c16873..628d6a567c 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h @@ -5,16 +5,17 @@ #ifndef _CN10K_CRYPTODEV_OPS_H_ #define _CN10K_CRYPTODEV_OPS_H_ -#include #include +#include +#include extern struct rte_cryptodev_ops cn10k_cpt_ops; void cn10k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev); __rte_internal -uint16_t cn10k_cpt_crypto_adapter_enqueue(uintptr_t base, - struct rte_crypto_op *op); +uint16_t __rte_hot cn10k_cpt_crypto_adapter_enqueue(void *ws, struct rte_event ev[], + uint16_t nb_events); __rte_internal uintptr_t cn10k_cpt_crypto_adapter_dequeue(uintptr_t get_work1); diff --git a/drivers/crypto/cnxk/meson.build b/drivers/crypto/cnxk/meson.build index 23a1cc3aac..952554ac12 100644 --- a/drivers/crypto/cnxk/meson.build +++ b/drivers/crypto/cnxk/meson.build @@ -24,7 +24,7 @@ sources = files( deps += ['bus_pci', 'common_cnxk', 'security', 'eventdev'] -includes += include_directories('../../../lib/net') +includes += include_directories('../../../lib/net', '../../event/cnxk') if get_option('buildtype').contains('debug') cflags += [ '-DLA_IPSEC_DEBUG' ] diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c index 5a0cab40a9..25c85902d6 100644 --- a/drivers/event/cnxk/cn10k_eventdev.c +++ b/drivers/event/cnxk/cn10k_eventdev.c @@ -454,7 +454,7 @@ cn10k_sso_fp_fns_set(struct rte_eventdev *event_dev) sso_hws_deq_tmo_ca_burst); } } - event_dev->ca_enqueue = cn10k_sso_hws_ca_enq; + event_dev->ca_enqueue = cn10k_cpt_crypto_adapter_enqueue; if (dev->tx_offloads & NIX_TX_MULTI_SEG_F) CN10K_SET_EVDEV_ENQ_OP(dev, event_dev->txa_enqueue, diff --git a/drivers/event/cnxk/cn10k_worker.c b/drivers/event/cnxk/cn10k_worker.c index f953e19dd0..4581c41233 100644 --- a/drivers/event/cnxk/cn10k_worker.c +++ b/drivers/event/cnxk/cn10k_worker.c @@ -64,13 +64,3 @@ cn10k_sso_hws_enq_fwd_burst(void *port, const struct rte_event ev[], return 1; } - -uint16_t __rte_hot -cn10k_sso_hws_ca_enq(void *port, struct rte_event ev[], uint16_t nb_events) -{ - struct cn10k_sso_hws *ws = port; - - RTE_SET_USED(nb_events); - - return cn10k_cpt_crypto_adapter_enqueue(ws->base, ev->event_ptr); -} diff --git a/drivers/event/cnxk/cn10k_worker.h b/drivers/event/cnxk/cn10k_worker.h index 0915f404e0..65bb08c0a1 100644 --- a/drivers/event/cnxk/cn10k_worker.h +++ b/drivers/event/cnxk/cn10k_worker.h @@ -353,8 +353,6 @@ uint16_t __rte_hot cn10k_sso_hws_enq_new_burst(void *port, uint16_t __rte_hot cn10k_sso_hws_enq_fwd_burst(void *port, const struct rte_event ev[], uint16_t nb_events); -uint16_t __rte_hot cn10k_sso_hws_ca_enq(void *port, struct rte_event ev[], - uint16_t nb_events); #define R(name, flags) \ uint16_t __rte_hot cn10k_sso_hws_deq_##name( \ From patchwork Tue Aug 9 10:53:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114747 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2AD3CA00C2; Tue, 9 Aug 2022 12:54:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E4CAF42BC8; Tue, 9 Aug 2022 12:54:10 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id DDC5942BB4 for ; Tue, 9 Aug 2022 12:54:09 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791T66D017193 for ; Tue, 9 Aug 2022 03:54:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=pAKCv13jG+RVF+gHIWW5q0DCH42DvAMXpP1xMC5zYZU=; b=Sijm9Ofo+swV2ynnoFOdrzN0MLs6Pu8gmf/vxlGkXIa3eNLwZnO/hcO21dZT/JY/DRoO t8s1MQwZUMTemiMyrVeDMOGZeQWVqdtzRwJUN2iMio5vkWza/SWurTPkndafHtXQ6jna IfJ/b9SS/terQsCT+ND1UTsrR83X5wKTuLq4a2Q1U7sw68BDVGNrhVAmyA3hkpPp6YOt rDEJbgZRs2ul1+hjyxa1y9vO6Frncbm2TJazb0+EllIHP2nAhq16IIgg3xbg2VPgdRNE JUaS3U8D+fvl9ARwyE89jgXhIOnoQgk1mMoOhjoZCY5aajvsv+y4t6ix3SlE9cN9Yiin yQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hudy6spgs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:09 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Aug 2022 03:54:07 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:07 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id 3CBFE3F7048; Tue, 9 Aug 2022 03:54:04 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 03/18] crypto/cnxk: remove zero IV Date: Tue, 9 Aug 2022 16:23:41 +0530 Message-ID: <20220809105356.561-4-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: C6MgcdHMbh1uxJ_N_gLqkP6PavFse9X2 X-Proofpoint-ORIG-GUID: C6MgcdHMbh1uxJ_N_gLqkP6PavFse9X2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Zeroising AES-CMAC IV would be done in microcode. Extra clearing in DPDK is not required. Signed-off-by: Anoob Joseph Signed-off-by: Tejasree Kondoj --- drivers/crypto/cnxk/cnxk_se.h | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index a75003f2c6..5574dbfa04 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -91,10 +91,6 @@ pdcp_iv_copy(uint8_t *iv_d, uint8_t *iv_s, const uint8_t pdcp_alg_type, memcpy(iv_d + 6, iv_s + 8, 17); } else memcpy(iv_d, iv_s, 16); - } else { - /* AES-CMAC EIA2, microcode expects 16B zeroized IV */ - for (j = 0; j < 16; j++) - iv_d[j] = 0; } } From patchwork Tue Aug 9 10:53:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114748 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E1FE1A00C2; Tue, 9 Aug 2022 12:54:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D5E3442BCD; Tue, 9 Aug 2022 12:54:13 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 7E5CA42BA7 for ; Tue, 9 Aug 2022 12:54:12 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791FxJh015646 for ; Tue, 9 Aug 2022 03:54:11 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=YgDaChHrDnDLfna2QxKdzmLTlajZWmcnGxvxA4+PZ4k=; b=RR9hx9P5TRb0rxQ1K8gJP+0VCdkNfagVS+8aj8GSI2oVyG0NRt+k1zrm4fyPcsSlgrKi I0hxwV7jYNmH9aieU4LNaEHo4Ba0JzmXRh0B69vdIjgd1bvhUwgXOGHrtKTOuHAjFC0e VshwhoRjOS/g7XMOQy0Z8Zk1CH/k46yS7jvj2/eoaaaYLlFKJBRyI7BH70rmOeCJUQnq fDoDHv4GIbioUKuG74DwGpxJifXsj3mLvYcaHfP5+iPAun/s2vnB/HU1E9bB460VpVVZ souTBYpDoyV3HFExsFDrZl67JsrpmOHw/+F/9OlYNC3V3TqiCl352CxLb3ndN4g31a92 1A== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3huds2sq71-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:11 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:10 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Aug 2022 03:54:10 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id CA4D93F704C; Tue, 9 Aug 2022 03:54:07 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , , Volodymyr Fialko Subject: [PATCH v2 04/18] crypto/cnxk: limit the meta buf cache to 128 Date: Tue, 9 Aug 2022 16:23:42 +0530 Message-ID: <20220809105356.561-5-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: wmqDh3fWUqw4G7siiFDSkNtCuIWwYAd7 X-Proofpoint-GUID: wmqDh3fWUqw4G7siiFDSkNtCuIWwYAd7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Limit meta buf pool cache size as 128. Having 512 as the cache size would cause more time for refill. Signed-off-by: Anoob Joseph Signed-off-by: Volodymyr Fialko --- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 80071872f1..a73c156d01 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -10,12 +10,13 @@ #include "cnxk_ae.h" #include "cnxk_cryptodev.h" -#include "cnxk_cryptodev_ops.h" #include "cnxk_cryptodev_capabilities.h" +#include "cnxk_cryptodev_ops.h" #include "cnxk_se.h" -#define CNXK_CPT_MAX_ASYM_OP_NUM_PARAMS 5 -#define CNXK_CPT_MAX_ASYM_OP_MOD_LEN 1024 +#define CNXK_CPT_MAX_ASYM_OP_NUM_PARAMS 5 +#define CNXK_CPT_MAX_ASYM_OP_MOD_LEN 1024 +#define CNXK_CPT_META_BUF_MAX_CACHE_SIZE 128 static int cnxk_cpt_get_mlen(void) @@ -200,7 +201,7 @@ cnxk_cpt_metabuf_mempool_create(const struct rte_cryptodev *dev, } mb_pool_sz = nb_elements; - cache_sz = RTE_MIN(RTE_MEMPOOL_CACHE_MAX_SIZE, nb_elements / 1.5); + cache_sz = RTE_MIN(CNXK_CPT_META_BUF_MAX_CACHE_SIZE, nb_elements / 1.5); /* For poll mode, core that enqueues and core that dequeues can be * different. For event mode, all cores are allowed to use same crypto From patchwork Tue Aug 9 10:53:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114749 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 39A4AA00C2; Tue, 9 Aug 2022 12:54:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CFC6942BD0; Tue, 9 Aug 2022 12:54:16 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 1502B42BBD for ; Tue, 9 Aug 2022 12:54:14 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791Fxll015643 for ; Tue, 9 Aug 2022 03:54:14 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=YWHJB1ZVudPgdZvVpaCO2oOiaSLMLL+wXZHU7dKoG/Q=; b=Tdc8+qoj7+pm6cUrfHON7SGE5dSIJ+r1V6k6ulglDEE9GsVhBNURQNRHBU+tuh8Sf8Iq f6u8d0NfmxE9ld67ciFhJZzON8/RZEb4wnEYTGStZMra02TGcIKWeXX9T6dNjNfZ94+U rkQHDPmv3VozEGxp6RgiYMMKqfB/X3ekY4+tgxHvU3Gf7UskSTPCu77bOKhyoyCoRzLE hwLynh47zRLlJGPPPFYUbsE7En2XWI1LvevhJIV7Fo9F9Rrsv1tJEXOGyKA+Km1p594x LajxJG//YC7R1uEfPydIRj7LWo0JTPceTTjJc3K+jKaHtif/PBndyzw70AnuQtCqKzTM mA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3huds2sq76-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:14 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Aug 2022 03:54:12 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:12 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id A3F823F7048; Tue, 9 Aug 2022 03:54:10 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 05/18] crypto/cnxk: add separate path for pdcp chain opcode Date: Tue, 9 Aug 2022 16:23:43 +0530 Message-ID: <20220809105356.561-6-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: F8KEnIXTBQSRzINI3cBzLdFFIYEumKst X-Proofpoint-GUID: F8KEnIXTBQSRzINI3cBzLdFFIYEumKst X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add separate datapath for PDCP chain opcode. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 7 +- drivers/crypto/cnxk/cnxk_se.h | 229 +++++++++++++++++++---- drivers/crypto/cnxk/meson.build | 4 +- 3 files changed, 196 insertions(+), 44 deletions(-) diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 3c4cc41bdb..2182c1bd2f 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -28,11 +28,12 @@ cn9k_cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, cpt_op = sess->cpt_op; - if (cpt_op & ROC_SE_OP_CIPHER_MASK) + if (sess->roc_se_ctx.fc_type == ROC_SE_PDCP_CHAIN) + ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); + else if (cpt_op & ROC_SE_OP_CIPHER_MASK) ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst); else - ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, - inst); + ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, inst); return ret; } diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 5574dbfa04..2b477284c0 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -94,6 +94,47 @@ pdcp_iv_copy(uint8_t *iv_d, uint8_t *iv_s, const uint8_t pdcp_alg_type, } } +/* + * Digest immediately at the end of the data is the best case. Switch to SG if + * that cannot be ensured. + */ +static inline void +cpt_digest_buf_lb_check(const struct cnxk_se_sess *sess, struct rte_mbuf *m, + struct roc_se_fc_params *fc_params, uint32_t *flags, + struct rte_crypto_sym_op *sym_op, bool *inplace, uint32_t a_data_off, + uint32_t a_data_len, uint32_t c_data_off, uint32_t c_data_len, + const bool is_pdcp_chain) +{ + const uint32_t auth_end = a_data_off + a_data_len; + uint32_t mc_hash_off; + + /* PDCP_CHAIN only supports auth_first */ + + if (is_pdcp_chain || sess->auth_first) + mc_hash_off = auth_end; + else + mc_hash_off = RTE_MAX(c_data_off + c_data_len, auth_end); + + /* Digest immediately following data is best case */ + + if (unlikely(rte_pktmbuf_mtod_offset(m, uint8_t *, mc_hash_off) != + sym_op->auth.digest.data)) { + *flags |= ROC_SE_VALID_MAC_BUF; + fc_params->mac_buf.size = sess->mac_len; + fc_params->mac_buf.vaddr = sym_op->auth.digest.data; + *inplace = false; + } +} + +static inline struct rte_mbuf * +cpt_m_dst_get(uint8_t cpt_op, struct rte_mbuf *m_src, struct rte_mbuf *m_dst) +{ + if (m_dst != NULL && (cpt_op & ROC_SE_OP_ENCODE)) + return m_dst; + else + return m_src; +} + static __rte_always_inline int cpt_mac_len_verify(struct rte_crypto_auth_xform *auth) { @@ -1828,9 +1869,6 @@ cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst); } else if (fc_type == ROC_SE_KASUMI) { ret = cpt_kasumi_dec_prep(d_offs, d_lens, fc_params, inst); - } else if (fc_type == ROC_SE_PDCP_CHAIN) { - ret = cpt_pdcp_chain_alg_prep(flags, d_offs, d_lens, fc_params, - inst); } /* @@ -1858,13 +1896,9 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, } else if (fc_type == ROC_SE_PDCP) { ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst); } else if (fc_type == ROC_SE_KASUMI) { - ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params, - inst); + ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params, inst); } else if (fc_type == ROC_SE_HASH_HMAC) { ret = cpt_digest_gen_prep(flags, d_lens, fc_params, inst); - } else if (fc_type == ROC_SE_PDCP_CHAIN) { - ret = cpt_pdcp_chain_alg_prep(flags, d_offs, d_lens, fc_params, - inst); } return ret; @@ -2384,15 +2418,11 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, uint8_t inplace = 1; #endif struct roc_se_fc_params fc_params; - bool chain = sess->chained_op; char src[SRC_IOV_SIZE]; char dst[SRC_IOV_SIZE]; uint32_t iv_buf[4]; - bool pdcp_chain; int ret; - pdcp_chain = chain && (sess->zs_auth || sess->zs_cipher); - fc_params.cipher_iv_len = sess->iv_length; fc_params.auth_iv_len = sess->auth_iv_length; @@ -2410,11 +2440,11 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } - if (sess->zsk_flag || sess->zs_auth) { + if (sess->zsk_flag) { if (sess->auth_iv_length) - fc_params.auth_iv_buf = rte_crypto_op_ctod_offset( - cop, uint8_t *, sess->auth_iv_offset); - if ((!chain) && (sess->zsk_flag != ROC_SE_ZS_EA)) + fc_params.auth_iv_buf = + rte_crypto_op_ctod_offset(cop, uint8_t *, sess->auth_iv_offset); + if (sess->zsk_flag != ROC_SE_ZS_EA) inplace = 0; } m_src = sym_op->m_src; @@ -2453,11 +2483,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } fc_params.iv_buf = salt + 4; if (likely(sess->mac_len)) { - struct rte_mbuf *m = - (cpt_op & ROC_SE_OP_ENCODE) ? m_dst : m_src; - - if (!m) - m = m_src; + struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); /* hmac immediately following data is best case */ if (unlikely(rte_pktmbuf_mtod(m, uint8_t *) + @@ -2476,17 +2502,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, uint32_t a_data_length = sym_op->auth.data.length; uint32_t a_data_offset = sym_op->auth.data.offset; - if (pdcp_chain) { - if (sess->zs_cipher) { - ci_data_length /= 8; - ci_data_offset /= 8; - } - if (sess->zs_auth) { - a_data_length /= 8; - a_data_offset /= 8; - } - } - d_offs = ci_data_offset; d_offs = (d_offs << 16) | a_data_offset; @@ -2512,11 +2527,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.iv_buf = salt + 4; } if (likely(sess->mac_len)) { - struct rte_mbuf *m; - - m = (cpt_op & ROC_SE_OP_ENCODE) ? m_dst : m_src; - if (!m) - m = m_src; + struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); /* hmac immediately following data is best case */ if (!(op_minor & ROC_SE_FC_MINOR_OP_HMAC_FIRST) && @@ -2533,8 +2544,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; - if (!(sess->auth_first) && (!pdcp_chain) && - unlikely(sess->is_null || sess->cpt_op == ROC_SE_OP_DECODE)) + if (!(sess->auth_first) && unlikely(sess->is_null || sess->cpt_op == ROC_SE_OP_DECODE)) inplace = 0; if (likely(!m_dst && inplace)) { @@ -2622,6 +2632,147 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, return ret; } +static __rte_always_inline int +fill_pdcp_chain_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst) +{ + uint32_t ci_data_length, ci_data_offset, a_data_length, a_data_offset; + struct rte_crypto_sym_op *sym_op = cop->sym; + struct roc_se_fc_params fc_params; + struct rte_mbuf *m_src, *m_dst; + uint8_t cpt_op = sess->cpt_op; + uint64_t d_offs, d_lens; + char src[SRC_IOV_SIZE]; + char dst[SRC_IOV_SIZE]; + bool inplace = true; + uint32_t flags = 0; + void *mdata; + int ret; + + fc_params.cipher_iv_len = sess->iv_length; + fc_params.auth_iv_len = sess->auth_iv_length; + fc_params.iv_buf = NULL; + fc_params.auth_iv_buf = NULL; + + m_src = sym_op->m_src; + m_dst = sym_op->m_dst; + + if (likely(sess->iv_length)) + fc_params.iv_buf = rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset); + + ci_data_length = sym_op->cipher.data.length; + ci_data_offset = sym_op->cipher.data.offset; + a_data_length = sym_op->auth.data.length; + a_data_offset = sym_op->auth.data.offset; + + /* + * For ZUC & SNOW, length & offset is provided in bits. Convert to + * bytes. + */ + + if (sess->zs_cipher) { + ci_data_length /= 8; + ci_data_offset /= 8; + } + + if (sess->zs_auth) { + a_data_length /= 8; + a_data_offset /= 8; + /* + * ZUC & SNOW would have valid iv_buf. AES-CMAC doesn't require + * IV from application. + */ + fc_params.auth_iv_buf = + rte_crypto_op_ctod_offset(cop, uint8_t *, sess->auth_iv_offset); +#ifdef CNXK_CRYPTODEV_DEBUG + if (sess->auth_iv_length == 0) + plt_err("Invalid auth IV length"); +#endif + } + + d_offs = ci_data_offset; + d_offs = (d_offs << 16) | a_data_offset; + d_lens = ci_data_length; + d_lens = (d_lens << 32) | a_data_length; + + if (likely(sess->mac_len)) { + struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); + + cpt_digest_buf_lb_check(sess, m, &fc_params, &flags, sym_op, &inplace, + a_data_offset, a_data_length, ci_data_offset, + ci_data_length, true); + } + + fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + + if (likely((m_dst == NULL || m_dst == m_src)) && inplace) { + fc_params.dst_iov = fc_params.src_iov = (void *)src; + prepare_iov_from_pkt_inplace(m_src, &fc_params, &flags); + } else { + /* Out of place processing */ + fc_params.src_iov = (void *)src; + fc_params.dst_iov = (void *)dst; + + /* Store SG I/O in the api for reuse */ + if (unlikely(prepare_iov_from_pkt(m_src, fc_params.src_iov, 0))) { + plt_dp_err("Could not prepare src iov"); + ret = -EINVAL; + goto err_exit; + } + + if (unlikely(m_dst != NULL)) { + uint32_t pkt_len; + + /* Try to make room as much as src has */ + pkt_len = rte_pktmbuf_pkt_len(m_dst); + + if (unlikely(pkt_len < rte_pktmbuf_pkt_len(m_src))) { + pkt_len = rte_pktmbuf_pkt_len(m_src) - pkt_len; + if (!rte_pktmbuf_append(m_dst, pkt_len)) { + plt_dp_err("Not enough space in m_dst " + "%p, need %u more", + m_dst, pkt_len); + ret = -EINVAL; + goto err_exit; + } + } + + if (unlikely(prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0))) { + plt_dp_err("Could not prepare m_dst iov %p", m_dst); + ret = -EINVAL; + goto err_exit; + } + } else { + fc_params.dst_iov = (void *)src; + } + } + + if (unlikely(!((flags & ROC_SE_SINGLE_BUF_INPLACE) && + (flags & ROC_SE_SINGLE_BUF_HEADROOM)))) { + mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, m_info->pool, infl_req); + if (unlikely(mdata == NULL)) { + plt_dp_err("Could not allocate meta buffer for request"); + return -ENOMEM; + } + } + + /* Finally prepare the instruction */ + ret = cpt_pdcp_chain_alg_prep(flags, d_offs, d_lens, &fc_params, inst); + if (unlikely(ret)) { + plt_dp_err("Could not prepare instruction"); + goto free_mdata_and_exit; + } + + return 0; + +free_mdata_and_exit: + if (infl_req->op_flags & CPT_OP_FLAGS_METABUF) + rte_mempool_put(m_info->pool, infl_req->mdata); +err_exit: + return ret; +} + static __rte_always_inline void compl_auth_verify(struct rte_crypto_op *op, uint8_t *gen_mac, uint64_t mac_len) { diff --git a/drivers/crypto/cnxk/meson.build b/drivers/crypto/cnxk/meson.build index 952554ac12..8db861f908 100644 --- a/drivers/crypto/cnxk/meson.build +++ b/drivers/crypto/cnxk/meson.build @@ -27,7 +27,7 @@ deps += ['bus_pci', 'common_cnxk', 'security', 'eventdev'] includes += include_directories('../../../lib/net', '../../event/cnxk') if get_option('buildtype').contains('debug') - cflags += [ '-DLA_IPSEC_DEBUG' ] + cflags += [ '-DLA_IPSEC_DEBUG','-DCNXK_CRYPTODEV_DEBUG' ] else - cflags += [ '-ULA_IPSEC_DEBUG' ] + cflags += [ '-ULA_IPSEC_DEBUG','-UCNXK_CRYPTODEV_DEBUG' ] endif From patchwork Tue Aug 9 10:53:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114750 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CA7A9A00C2; Tue, 9 Aug 2022 12:54:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C731B42BDC; Tue, 9 Aug 2022 12:54:18 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id F3FC042BDA for ; Tue, 9 Aug 2022 12:54:17 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791TKc5017274 for ; Tue, 9 Aug 2022 03:54:17 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=ZoncwHScaA1cg6x9JgJyPj146wXnAIm6ACWMN/APXOg=; b=D5U6TsctK0Hj1dw6q1nAUCc0KavUmGznBTarMh9c6fojIzCvsE0q/8UROgh2NjOnib4+ 2vXnYlRGUOXt7/CM3xEi0ock2NCiZXlwtjXYczQXiIuDWf118AX1Q+Y7CY9TdWlsqwYG 4EzV9ukb4BqEBLcpqs9BNqmu2TrwKTst6lwC9m6x4ifZn5SgYknC3Yg3iSFL0R3mn7Gw c/eAW9zDufWrAjreBHxW6vSnoZN8jkIb8X2vyF0ouXhxU3Ujpp1HowCmifZZMa5VbJu3 BG3Mu+i8phChkghhfiq0GZg0Y6nE/FTcEl2F5h9DRpJGfT/jJIXnnDUsrdmXwdYFq1lD 9A== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hudy6sphe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:17 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:15 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Aug 2022 03:54:15 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id 33F723F7065; Tue, 9 Aug 2022 03:54:12 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 06/18] crypto/cnxk: add separate datapath for pdcp cipher operation Date: Tue, 9 Aug 2022 16:23:44 +0530 Message-ID: <20220809105356.561-7-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: U2JVJJNPXUChQwdW1QkNYBkBJhDFW14l X-Proofpoint-ORIG-GUID: U2JVJJNPXUChQwdW1QkNYBkBJhDFW14l X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add separate datapath for PDCP opcode performing cipher operation. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 19 --- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 27 +--- drivers/crypto/cnxk/cnxk_se.h | 177 +++++++++++++++++++--- 3 files changed, 158 insertions(+), 65 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index bfa6374005..1b70d02e2a 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -77,25 +77,6 @@ cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, return ret; } -static __rte_always_inline int __rte_hot -cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, - struct cnxk_se_sess *sess, struct cpt_inflight_req *infl_req, - struct cpt_inst_s *inst) -{ - uint64_t cpt_op; - int ret; - - cpt_op = sess->cpt_op; - - if (cpt_op & ROC_SE_OP_CIPHER_MASK) - ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst); - else - ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, - inst); - - return ret; -} - static inline int cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], struct cpt_inst_s inst[], struct cpt_inflight_req *infl_req) diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 2182c1bd2f..3d69723809 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -17,27 +17,6 @@ #include "cnxk_cryptodev_ops.h" #include "cnxk_se.h" -static __rte_always_inline int __rte_hot -cn9k_cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, - struct cnxk_se_sess *sess, - struct cpt_inflight_req *infl_req, - struct cpt_inst_s *inst) -{ - uint64_t cpt_op; - int ret; - - cpt_op = sess->cpt_op; - - if (sess->roc_se_ctx.fc_type == ROC_SE_PDCP_CHAIN) - ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); - else if (cpt_op & ROC_SE_OP_CIPHER_MASK) - ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst); - else - ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, inst); - - return ret; -} - static __rte_always_inline int __rte_hot cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op, struct cpt_inflight_req *infl_req, @@ -118,8 +97,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, sym_op = op->sym; sess = get_sym_session_private_data( sym_op->session, cn9k_cryptodev_driver_id); - ret = cn9k_cpt_sym_inst_fill(qp, op, sess, infl_req, - inst); + ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst); inst->w7.u64 = sess->cpt_inst_w7; } else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) ret = cn9k_cpt_sec_inst_fill(op, infl_req, inst); @@ -130,8 +108,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, return -1; } - ret = cn9k_cpt_sym_inst_fill(qp, op, sess, infl_req, - inst); + ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst); if (unlikely(ret)) { sym_session_clear(cn9k_cryptodev_driver_id, op->sym->session); diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 2b477284c0..35d074ea34 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -1865,8 +1865,6 @@ cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, if (likely(fc_type == ROC_SE_FC_GEN)) { ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, inst); - } else if (fc_type == ROC_SE_PDCP) { - ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst); } else if (fc_type == ROC_SE_KASUMI) { ret = cpt_kasumi_dec_prep(d_offs, d_lens, fc_params, inst); } @@ -2400,8 +2398,8 @@ prepare_iov_from_pkt_inplace(struct rte_mbuf *pkt, static __rte_always_inline int fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, - struct cpt_qp_meta_info *m_info, - struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst) + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst, const bool is_kasumi) { struct roc_se_ctx *ctx = &sess->roc_se_ctx; uint8_t op_minor = ctx->template_w4.s.opcode_minor; @@ -2424,7 +2422,9 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, int ret; fc_params.cipher_iv_len = sess->iv_length; - fc_params.auth_iv_len = sess->auth_iv_length; + fc_params.auth_iv_len = 0; + fc_params.auth_iv_buf = NULL; + fc_params.iv_buf = NULL; if (likely(sess->iv_length)) { flags |= ROC_SE_VALID_IV_BUF; @@ -2440,13 +2440,15 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } - if (sess->zsk_flag) { + /* Kasumi would need auth IV */ + if (is_kasumi && sess->zsk_flag) { + fc_params.auth_iv_len = sess->auth_iv_length; if (sess->auth_iv_length) fc_params.auth_iv_buf = rte_crypto_op_ctod_offset(cop, uint8_t *, sess->auth_iv_offset); - if (sess->zsk_flag != ROC_SE_ZS_EA) - inplace = 0; + inplace = 0; } + m_src = sym_op->m_src; m_dst = sym_op->m_dst; @@ -2508,14 +2510,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, d_lens = ci_data_length; d_lens = (d_lens << 32) | a_data_length; - if (sess->auth_first) - mc_hash_off = a_data_offset + a_data_length; - else - mc_hash_off = ci_data_offset + ci_data_length; - - if (mc_hash_off < (a_data_offset + a_data_length)) { - mc_hash_off = (a_data_offset + a_data_length); - } /* for gmac, salt should be updated like in gcm */ if (unlikely(sess->is_gmac)) { uint8_t *salt; @@ -2529,6 +2523,14 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, if (likely(sess->mac_len)) { struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); + if (sess->auth_first) + mc_hash_off = a_data_offset + a_data_length; + else + mc_hash_off = ci_data_offset + ci_data_length; + + if (mc_hash_off < (a_data_offset + a_data_length)) + mc_hash_off = (a_data_offset + a_data_length); + /* hmac immediately following data is best case */ if (!(op_minor & ROC_SE_FC_MINOR_OP_HMAC_FIRST) && (unlikely(rte_pktmbuf_mtod(m, uint8_t *) + @@ -2599,11 +2601,8 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } if (unlikely(!((flags & ROC_SE_SINGLE_BUF_INPLACE) && - (flags & ROC_SE_SINGLE_BUF_HEADROOM) && - ((ctx->fc_type != ROC_SE_KASUMI) && - (ctx->fc_type != ROC_SE_HASH_HMAC))))) { - mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, - m_info->pool, infl_req); + (flags & ROC_SE_SINGLE_BUF_HEADROOM) && !is_kasumi))) { + mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, m_info->pool, infl_req); if (mdata == NULL) { plt_dp_err("Error allocating meta buffer for request"); return -ENOMEM; @@ -2632,6 +2631,112 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, return ret; } +static __rte_always_inline int +fill_pdcp_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst) +{ + struct rte_crypto_sym_op *sym_op = cop->sym; + struct roc_se_fc_params fc_params; + uint32_t c_data_len, c_data_off; + struct rte_mbuf *m_src, *m_dst; + uint64_t d_offs, d_lens; + char src[SRC_IOV_SIZE]; + char dst[SRC_IOV_SIZE]; + void *mdata = NULL; + uint32_t flags = 0; + int ret; + + /* Cipher only */ + + fc_params.cipher_iv_len = sess->iv_length; + fc_params.auth_iv_len = 0; + fc_params.iv_buf = NULL; + fc_params.auth_iv_buf = NULL; + + if (likely(sess->iv_length)) + fc_params.iv_buf = rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset); + + m_src = sym_op->m_src; + m_dst = sym_op->m_dst; + + c_data_len = sym_op->cipher.data.length; + c_data_off = sym_op->cipher.data.offset; + + d_offs = (uint64_t)c_data_off << 16; + d_lens = (uint64_t)c_data_len << 32; + + fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + + if (likely(m_dst == NULL || m_src == m_dst)) { + fc_params.dst_iov = fc_params.src_iov = (void *)src; + prepare_iov_from_pkt_inplace(m_src, &fc_params, &flags); + } else { + /* Out of place processing */ + fc_params.src_iov = (void *)src; + fc_params.dst_iov = (void *)dst; + + /* Store SG I/O in the api for reuse */ + if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0)) { + plt_dp_err("Prepare src iov failed"); + ret = -EINVAL; + goto err_exit; + } + + if (unlikely(m_dst != NULL)) { + uint32_t pkt_len; + + /* Try to make room as much as src has */ + pkt_len = rte_pktmbuf_pkt_len(m_dst); + + if (unlikely(pkt_len < rte_pktmbuf_pkt_len(m_src))) { + pkt_len = rte_pktmbuf_pkt_len(m_src) - pkt_len; + if (!rte_pktmbuf_append(m_dst, pkt_len)) { + plt_dp_err("Not enough space in " + "m_dst %p, need %u" + " more", + m_dst, pkt_len); + ret = -EINVAL; + goto err_exit; + } + } + + if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0)) { + plt_dp_err("Prepare dst iov failed for " + "m_dst %p", + m_dst); + ret = -EINVAL; + goto err_exit; + } + } else { + fc_params.dst_iov = (void *)src; + } + } + + if (unlikely(!((flags & ROC_SE_SINGLE_BUF_INPLACE) && + (flags & ROC_SE_SINGLE_BUF_HEADROOM)))) { + mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, m_info->pool, infl_req); + if (mdata == NULL) { + plt_dp_err("Could not allocate meta buffer"); + return -ENOMEM; + } + } + + ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, &fc_params, inst); + if (unlikely(ret)) { + plt_dp_err("Could not prepare instruction"); + goto free_mdata_and_exit; + } + + return 0; + +free_mdata_and_exit: + if (infl_req->op_flags & CPT_OP_FLAGS_METABUF) + rte_mempool_put(m_info->pool, infl_req->mdata); +err_exit: + return ret; +} + static __rte_always_inline int fill_pdcp_chain_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, @@ -2974,4 +3079,34 @@ fill_digest_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, err_exit: return ret; } + +static __rte_always_inline int __rte_hot +cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cnxk_se_sess *sess, + struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst) +{ + uint64_t cpt_op = sess->cpt_op; + int ret; + + if (cpt_op & ROC_SE_OP_CIPHER_MASK) { + switch (sess->roc_se_ctx.fc_type) { + case ROC_SE_PDCP_CHAIN: + ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); + break; + case ROC_SE_PDCP: + ret = fill_pdcp_params(op, sess, &qp->meta_info, infl_req, inst); + break; + case ROC_SE_KASUMI: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, true); + break; + default: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, false); + break; + } + } else { + ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, inst); + } + + return ret; +} + #endif /*_CNXK_SE_H_ */ From patchwork Tue Aug 9 10:53:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114751 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ABE81A00C2; Tue, 9 Aug 2022 12:54:55 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4383342BE5; Tue, 9 Aug 2022 12:54:23 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id D24A742BBD for ; Tue, 9 Aug 2022 12:54:21 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791T66E017193 for ; Tue, 9 Aug 2022 03:54:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=2IejpeZY/nkwaNuPwnjsg58FbpV/xxhag3Aym+WoklQ=; b=hHCcNdt+SlzNb9Obmj58DtA6Sj9M45UliLK5OrwMmNcJN4l/Ea2lPp+xnp8XThpGT4bq 0dt8kRfyl3sCkghJvMaynmOQnBMd9UUiJJbGDCUwjlzoQcNGIPSeIf5AJ8PVuUXMT8ME 27+N3OYEiP983y052jY2wsTy2wrKbPeuCuJno1b+bQqSd+I9IjTu4pbVxL08Wevfg5T2 Bp+tkdpg/zKJbTa8vAWGMKs9FNf4p+Oc4P7ViLiZPmii1BJ8DqO+6Nlhg2xz2hcrR1N7 K9VHI4jiH0EEhWw0u3UkGsmuJCj2czXtirh4C6c/SzwHyokndGb65YJBBK6fzpVyb9Tx 7Q== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hudy6sphq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:21 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Aug 2022 03:54:19 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Aug 2022 03:54:19 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id B29E43F7072; Tue, 9 Aug 2022 03:54:15 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 07/18] crypto/cnxk: remove MAC len check for AEAD Date: Tue, 9 Aug 2022 16:23:45 +0530 Message-ID: <20220809105356.561-8-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: yXGNQsDIcR9xUYTu3OuwUqDKj4fvV0T9 X-Proofpoint-ORIG-GUID: yXGNQsDIcR9xUYTu3OuwUqDKj4fvV0T9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For both AES-GCM and ChaChaPoly, digest size must be non-zero. So remove the conditional handling in the datapath. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 35d074ea34..2c44dba9a0 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2453,6 +2453,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, m_dst = sym_op->m_dst; if (sess->aes_gcm || sess->chacha_poly) { + struct rte_mbuf *m; uint8_t *salt; uint8_t *aad_data; uint16_t aad_len; @@ -2483,20 +2484,19 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, cpt_fc_salt_update(&sess->roc_se_ctx, salt); sess->salt = *(uint32_t *)salt; } - fc_params.iv_buf = salt + 4; - if (likely(sess->mac_len)) { - struct rte_mbuf *m = cpt_m_dst_get(cpt_op, m_src, m_dst); - /* hmac immediately following data is best case */ - if (unlikely(rte_pktmbuf_mtod(m, uint8_t *) + - mc_hash_off != - (uint8_t *)sym_op->aead.digest.data)) { - flags |= ROC_SE_VALID_MAC_BUF; - fc_params.mac_buf.size = sess->mac_len; - fc_params.mac_buf.vaddr = - sym_op->aead.digest.data; - inplace = 0; - } + fc_params.iv_buf = PLT_PTR_ADD(salt, 4); + fc_params.mac_buf.size = 0; + fc_params.mac_buf.vaddr = NULL; + m = cpt_m_dst_get(cpt_op, m_src, m_dst); + + /* Digest immediately following data is best case */ + if (unlikely(rte_pktmbuf_mtod(m, uint8_t *) + mc_hash_off != + (uint8_t *)sym_op->aead.digest.data)) { + flags |= ROC_SE_VALID_MAC_BUF; + fc_params.mac_buf.size = sess->mac_len; + fc_params.mac_buf.vaddr = sym_op->aead.digest.data; + inplace = 0; } } else { uint32_t ci_data_length = sym_op->cipher.data.length; From patchwork Tue Aug 9 10:53:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114752 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 40041A00C2; Tue, 9 Aug 2022 12:55:01 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4532342BE7; Tue, 9 Aug 2022 12:54:25 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 7A42542C0A for ; Tue, 9 Aug 2022 12:54:23 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791G2gB015667 for ; Tue, 9 Aug 2022 03:54:22 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Lt5En0iNnf3iEtTDhzPIHZ6WpGSX/nQJAm2ATQ3LWuM=; b=DEMEDaTqBfhPv5lJcooYp8LsXfkWxxL3tvMeC4wRa8xNqEwRJ0hc4OhoDYD8E7l49oWj GP+dK0hDrq1+RG6Wc7TeTT23TO7Rdh3XKlg4TL5txujtC1TxqtBtE4lwNvJkV17GzHzO JISFpPZdpQ39XhOR2ABoRMwkiVQC9ABppzwm+w2Q/kPXtag9hlkT/bfB4cVi+NsYhHhB U5KbLCVlsINLa4Yx3o9PnnIOG2lRVrCyNs97fYYsqh2ji7fcABXrztOb0EqVwauHdjtX 08uy580obrzgKWxHqJuNVqnkFPHXekUTAqA5YdyHRclb6TGTO9totR+63S/uv/fCyLr0 vQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3huds2sq7m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:22 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:20 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:20 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id 6464D5B6970; Tue, 9 Aug 2022 03:54:18 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Tejasree Kondoj , Archana Muniganti , Subject: [PATCH v2 08/18] crypto/cnxk: fix endianness in anti-replay Date: Tue, 9 Aug 2022 16:23:46 +0530 Message-ID: <20220809105356.561-9-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: ZQJx-T9Humc5Vnp-1SxX6HJ4MuBe1fhr X-Proofpoint-GUID: ZQJx-T9Humc5Vnp-1SxX6HJ4MuBe1fhr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Tejasree Kondoj Fixing anti-replay endianness issue in lookaside IPsec. Fixes: c1eac1b966c2 ("crypto/cnxk: add anti-replay as per new firmware") Cc: ktejasree@marvell.com Signed-off-by: Tejasree Kondoj --- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index 3d69723809..b753c1cb4b 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -487,22 +487,19 @@ cn9k_cpt_crypto_adapter_enqueue(uintptr_t base, struct rte_crypto_op *op) } static inline int -ipsec_antireplay_check(struct cn9k_ipsec_sa *sa, uint32_t win_sz, - struct roc_ie_on_inb_hdr *data) +ipsec_antireplay_check(struct cn9k_ipsec_sa *sa, uint32_t win_sz, struct roc_ie_on_inb_hdr *data) { + uint32_t esn_low, esn_hi, seql, seqh = 0; struct roc_ie_on_common_sa *common_sa; struct roc_ie_on_inb_sa *in_sa; - struct roc_ie_on_sa_ctl *ctl; - uint32_t seql, seqh = 0; - uint64_t seq; + uint64_t seq, seq_in_sa; uint8_t esn; int ret; in_sa = &sa->in_sa; common_sa = &in_sa->common_sa; - ctl = &common_sa->ctl; - esn = ctl->esn_en; + esn = common_sa->ctl.esn_en; seql = rte_be_to_cpu_32(data->seql); if (!esn) { @@ -517,9 +514,13 @@ ipsec_antireplay_check(struct cn9k_ipsec_sa *sa, uint32_t win_sz, ret = cnxk_on_anti_replay_check(seq, &sa->ar, win_sz); if (esn && !ret) { - common_sa = &sa->in_sa.common_sa; - if (seq > common_sa->seq_t.u64) - common_sa->seq_t.u64 = seq; + esn_low = rte_be_to_cpu_32(common_sa->seq_t.tl); + esn_hi = rte_be_to_cpu_32(common_sa->seq_t.th); + seq_in_sa = ((uint64_t)esn_hi << 32) | esn_low; + if (seq > seq_in_sa) { + common_sa->seq_t.tl = rte_cpu_to_be_32(seql); + common_sa->seq_t.th = rte_cpu_to_be_32(seqh); + } } return ret; From patchwork Tue Aug 9 10:53:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114753 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8BC64A00C2; Tue, 9 Aug 2022 12:55:07 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4AEA942C2F; Tue, 9 Aug 2022 12:54:28 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id E9B0742BBA for ; Tue, 9 Aug 2022 12:54:26 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791T66F017193 for ; Tue, 9 Aug 2022 03:54:26 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=iov3QG+i9R6z+G6aagRW5EAdN3cD+ASTPQV4VrwtfVU=; b=QsawMnmM1RT6oVT9+xf2mSZnJuTWu7kGsUqteYGSeVwUL4BUImqYiB2ewMYb0OrL5dcr hGGJ7FraWWIV42+koW8RvoMJuDD58yLtwEEGkC3+u1M6lJdNO/nvJCcXcH5OEz5IqlYm k8BrSqr9D11x40+w9FVvOBedaRgG+XKFP+oisN3Gde3hul6Pb5y5jyAg+vPyjHMAIbZl EsKoSnGYDtVKT1UqH3UIfS/fR2XMGSePHNEPkoWKNStFpCZvg8q9zpj9JqlgdqV9n5AA 8gTTlDUjczX1jZwB60tUMlfG+BFXUUyg5bSOxElpNr1OjSDWihoUb3nkPoVDEFeAEXBy qA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hudy6sphy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:26 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:22 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Aug 2022 03:54:23 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id F1FD93F7048; Tue, 9 Aug 2022 03:54:20 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 09/18] crypto/cnxk: remove extra indirection for FC and Kasumi Date: Tue, 9 Aug 2022 16:23:47 +0530 Message-ID: <20220809105356.561-10-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: by8M1Koaiukbm8nKwcs3aT-ala4XC5HE X-Proofpoint-ORIG-GUID: by8M1Koaiukbm8nKwcs3aT-ala4XC5HE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Fill FC params routine now caters to only KASUMI & FC_GEN. Call appropriate functions directly. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 55 +++++++++++------------------------ 1 file changed, 17 insertions(+), 38 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 2c44dba9a0..64100e8376 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -1852,36 +1852,9 @@ cpt_kasumi_dec_prep(uint64_t d_offs, uint64_t d_lens, return 0; } -static __rte_always_inline int -cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, - struct roc_se_fc_params *fc_params, - struct cpt_inst_s *inst) -{ - struct roc_se_ctx *ctx = fc_params->ctx_buf.vaddr; - uint8_t fc_type; - int ret = -1; - - fc_type = ctx->fc_type; - - if (likely(fc_type == ROC_SE_FC_GEN)) { - ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, inst); - } else if (fc_type == ROC_SE_KASUMI) { - ret = cpt_kasumi_dec_prep(d_offs, d_lens, fc_params, inst); - } - - /* - * For AUTH_ONLY case, - * MC only supports digest generation and verification - * should be done in software by memcmp() - */ - - return ret; -} - static __rte_always_inline int cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, - struct roc_se_fc_params *fc_params, - struct cpt_inst_s *inst) + struct roc_se_fc_params *fc_params, struct cpt_inst_s *inst) { struct roc_se_ctx *ctx = fc_params->ctx_buf.vaddr; uint8_t fc_type; @@ -2425,6 +2398,8 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.auth_iv_len = 0; fc_params.auth_iv_buf = NULL; fc_params.iv_buf = NULL; + fc_params.mac_buf.size = 0; + fc_params.mac_buf.vaddr = 0; if (likely(sess->iv_length)) { flags |= ROC_SE_VALID_IV_BUF; @@ -2486,8 +2461,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } fc_params.iv_buf = PLT_PTR_ADD(salt, 4); - fc_params.mac_buf.size = 0; - fc_params.mac_buf.vaddr = NULL; m = cpt_m_dst_get(cpt_op, m_src, m_dst); /* Digest immediately following data is best case */ @@ -2600,8 +2573,8 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } - if (unlikely(!((flags & ROC_SE_SINGLE_BUF_INPLACE) && - (flags & ROC_SE_SINGLE_BUF_HEADROOM) && !is_kasumi))) { + if (unlikely(is_kasumi || !((flags & ROC_SE_SINGLE_BUF_INPLACE) && + (flags & ROC_SE_SINGLE_BUF_HEADROOM)))) { mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, m_info->pool, infl_req); if (mdata == NULL) { plt_dp_err("Error allocating meta buffer for request"); @@ -2610,12 +2583,18 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } /* Finally prepare the instruction */ - if (cpt_op & ROC_SE_OP_ENCODE) - ret = cpt_fc_enc_hmac_prep(flags, d_offs, d_lens, &fc_params, - inst); - else - ret = cpt_fc_dec_hmac_prep(flags, d_offs, d_lens, &fc_params, - inst); + + if (is_kasumi) { + if (cpt_op & ROC_SE_OP_ENCODE) + ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, &fc_params, inst); + else + ret = cpt_kasumi_dec_prep(d_offs, d_lens, &fc_params, inst); + } else { + if (cpt_op & ROC_SE_OP_ENCODE) + ret = cpt_enc_hmac_prep(flags, d_offs, d_lens, &fc_params, inst); + else + ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, &fc_params, inst); + } if (unlikely(ret)) { plt_dp_err("Preparing request failed due to bad input arg"); From patchwork Tue Aug 9 10:53:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114754 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5721FA00C2; Tue, 9 Aug 2022 12:55:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3222842C3C; Tue, 9 Aug 2022 12:54:29 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id F396142C1D for ; Tue, 9 Aug 2022 12:54:27 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791FxJi015646 for ; Tue, 9 Aug 2022 03:54:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=5Mmj233L9vQJWpbC9/4lC+rqIkF850Ti39Q9suwaA5I=; b=k4r57m92WXnSK4UlQL45SDtDWtd/+nwARG8Cb4icweiehOd4i9qn49gmSFLHlJCfIwK+ GwbkUdB2AEo6ZEmTxkIRNSLh5/0GtRzB3AkMNCMIseHoypYpnfQCYkrHi9jeoEhB27Io pUVtJFdsNVOfp0HKQrpK7aXIAKmdBrWqCk8H2xCMnnLicx59whj7LvOK4vjBib2GrU2T TGk0SWKKvi1/M4loAkn2koZOAV9xwYZHd287piVUYDpT0qg0EaL2Yn2Gkxr7gJkV4ELM sAw8xV62ygMSDQL9MdUVCpadY8Rpv0W+SptERYOqmBiFgFXhN146LC+Oyey4VEgolWRd FA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3huds2sq7x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:27 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Aug 2022 03:54:25 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Aug 2022 03:54:25 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id 86AF23F704C; Tue, 9 Aug 2022 03:54:23 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 10/18] crypto/cnxk: remove extra digest len check Date: Tue, 9 Aug 2022 16:23:48 +0530 Message-ID: <20220809105356.561-11-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: nl1OVLVPHGUaYRxlcuU9n-cHY_9rI3VL X-Proofpoint-GUID: nl1OVLVPHGUaYRxlcuU9n-cHY_9rI3VL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When the flag ROC_SE_VALID_MAC_BUF is set, digest length will be non-zero. Remove extra check in datapath. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 64100e8376..f382194c27 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -680,11 +680,9 @@ cpt_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, return -1; } } - /* mac_data */ - if (mac_len) { - i = fill_sg_comp_from_buf(scatter_comp, i, - &fc_params->mac_buf); - } + + /* Digest buffer */ + i = fill_sg_comp_from_buf(scatter_comp, i, &fc_params->mac_buf); } else { /* Output including mac */ size = outputlen - iv_len; From patchwork Tue Aug 9 10:53:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114755 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C841AA00C2; Tue, 9 Aug 2022 12:55:18 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3A80842C4F; Tue, 9 Aug 2022 12:54:32 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 24CFA42C4E for ; Tue, 9 Aug 2022 12:54:31 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791T39c017143 for ; Tue, 9 Aug 2022 03:54:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Q1BRmYBqO7/dMhODHbJA40rgZTl/81Rg3ay8Ge6BZwk=; b=ZjO+GAB9gzG9XTCsOEG0eSg0DwLkbBf99eG5w1xreYB3LZSurVyWAxOEHR6+CbiLw0/w LoEuBn1NLFtJc8mW1sSllgWVQPi8F7FSFgDZaB2PQAoeBTglXonyJwZmJS4pVAkFEj/Z WoLKr2O51xYXyvU6Y3NpzoViCoshuehsAYpJ/ktlbByUlkxR+L1SXPhT5qnZu9B4Cm5Y lLGjVXXWAjw1p9Ud44ZwnVpBZrqTmA7PYR+pOCMG3kbfGwUSzmxke1tePlu5hnxZDTD3 Fve17/0p3HS3L0NTNZcL5YP8WLIO04P70d413pVWqnTCuIcvPgwp7qPOvZ6RcGDWhbBf aA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hudy6spjb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:30 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Aug 2022 03:54:28 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:28 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id 211F53F7048; Tue, 9 Aug 2022 03:54:25 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 11/18] crypto/cnxk: avoid accessing se ctx in aes gcm path Date: Tue, 9 Aug 2022 16:23:49 +0530 Message-ID: <20220809105356.561-12-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: wFfawQR009cqmSajZ2HEeoBM-7oZxJQB X-Proofpoint-ORIG-GUID: wFfawQR009cqmSajZ2HEeoBM-7oZxJQB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Field op_minor is required only for digest encrypted cases with chained ops. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index f382194c27..a75e98362d 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2372,8 +2372,6 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst, const bool is_kasumi) { - struct roc_se_ctx *ctx = &sess->roc_se_ctx; - uint8_t op_minor = ctx->template_w4.s.opcode_minor; struct rte_crypto_sym_op *sym_op = cop->sym; void *mdata = NULL; uint32_t mc_hash_off; @@ -2474,6 +2472,9 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, uint32_t ci_data_offset = sym_op->cipher.data.offset; uint32_t a_data_length = sym_op->auth.data.length; uint32_t a_data_offset = sym_op->auth.data.offset; + struct roc_se_ctx *ctx = &sess->roc_se_ctx; + + const uint8_t op_minor = ctx->template_w4.s.opcode_minor; d_offs = ci_data_offset; d_offs = (d_offs << 16) | a_data_offset; From patchwork Tue Aug 9 10:53:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114756 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 87B70A00C2; Tue, 9 Aug 2022 12:55:24 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 29A6B42BED; Tue, 9 Aug 2022 12:54:35 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 3810942BED for ; Tue, 9 Aug 2022 12:54:33 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791G2gD015667 for ; Tue, 9 Aug 2022 03:54:32 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=EVuUaMfzF+ZHnSP7AIhAnVRtbpc4kDu6vISTqzQjIgc=; b=W6W9MHv39pWimFVOXsOuGl7vqzx8Aq1T5zNWsbzIsFLw8BRioOtGIaddQ96Ak3YBWneo fxkZFV6vpS8OqFzd0C43yQ7uIzkbZe9lM9QkbBrjOzX9vr02D8goOEUE7jtJVW5TmWZI l1U9DCnbamNOfgL5ulqAJ9mfwmKepbgpIxUTTEBKH+A+VMftfVXnUHyEjxIxEEE3AWin 5qnSFPKyB6jEK3KTEZs9L/zd/bMg7L6jeHiXCmP4DO2UmRN6U9WNRvRNcp0m2Z2dROwp rR7xJexzEE5cWSETheg/LRARiRtmySAJt/6CDtGq7cCs/bi53gbX1XDU7zCv5dqbPTlH WQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3huds2sq8b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:32 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Aug 2022 03:54:30 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:30 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id B3DB13F704C; Tue, 9 Aug 2022 03:54:28 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 12/18] crypto/cnxk: remove auth iv from kasumi cipher Date: Tue, 9 Aug 2022 16:23:50 +0530 Message-ID: <20220809105356.561-13-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 54Z80otpQvwjlbjqMOj9ZyjTij9JOVn7 X-Proofpoint-GUID: 54Z80otpQvwjlbjqMOj9ZyjTij9JOVn7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Auth IV is not applicable for kasumi cipher operation. Remove the same. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index a75e98362d..d8126567f9 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2411,14 +2411,9 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } - /* Kasumi would need auth IV */ - if (is_kasumi && sess->zsk_flag) { - fc_params.auth_iv_len = sess->auth_iv_length; - if (sess->auth_iv_length) - fc_params.auth_iv_buf = - rte_crypto_op_ctod_offset(cop, uint8_t *, sess->auth_iv_offset); + /* Kasumi would need SG mode */ + if (is_kasumi) inplace = 0; - } m_src = sym_op->m_src; m_dst = sym_op->m_dst; From patchwork Tue Aug 9 10:53:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114757 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 23397A00C2; Tue, 9 Aug 2022 12:55:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1C5DA42C23; Tue, 9 Aug 2022 12:54:37 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 671EE42BD7 for ; Tue, 9 Aug 2022 12:54:35 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791G3aL015673 for ; Tue, 9 Aug 2022 03:54:34 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=tIc0GqriIATRd3YsoFGFFX/ADaDsq+Z9u4nLJpbomsE=; b=AdkUPZFfQ1PXl9+jhCEW034EpnTlUj+GmU2LH8jj16XLrIpMlWAbTXVNfxg7oREOQBG9 3vnsyC4Du+DOGTWnIK/dLBIRBnsePfV2b9vqMe3xn6LYee13raUWEuvhGNo9pySMsFf+ 8ujvzVgCwZptwA7Nqj9F7VJbN8W/jHnHb8lY+P4LcyBNVRam37e8YpmzCcYoklum2rEw xDy40/xzzf/faYwZEa2YKY8Ja9NSP+R1qnfWu7AcwAOZ8w2n42AgHw1oy6+vNjsWzyI/ lWMvKe3Xz/owgnL7AIRciiznTSmWTwASKLMUr3ZWPiiCAJVw+OUD5byi61A7q6bHfB+m bw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3huds2sq8g-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:34 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:33 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:33 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id 51C143F7065; Tue, 9 Aug 2022 03:54:31 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Tejasree Kondoj , Archana Muniganti , Subject: [PATCH v2 13/18] crypto/cnxk: enable IE engine for Chacha-Poly Date: Tue, 9 Aug 2022 16:23:51 +0530 Message-ID: <20220809105356.561-14-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: ZzndQBnvEtFpCshI-cY19oiqNQgMJ8io X-Proofpoint-GUID: ZzndQBnvEtFpCshI-cY19oiqNQgMJ8io X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Tejasree Kondoj Change engine group of Chacha20-Poly1305 as it is supported on IE engines also. Signed-off-by: Tejasree Kondoj --- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index a73c156d01..85bb1d27a1 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -611,7 +611,7 @@ cnxk_cpt_inst_w7_get(struct cnxk_se_sess *sess, struct roc_cpt *roc_cpt) inst_w7.s.cptr = (uint64_t)&sess->roc_se_ctx.se_ctx; /* Set the engine group */ - if (sess->zsk_flag || sess->chacha_poly || sess->aes_ctr_eea2) + if (sess->zsk_flag || sess->aes_ctr_eea2) inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_SE]; else inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_IE]; From patchwork Tue Aug 9 10:53:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114758 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 23AB1A00C2; Tue, 9 Aug 2022 12:55:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 766C742C7C; Tue, 9 Aug 2022 12:54:39 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 81F9942C77 for ; Tue, 9 Aug 2022 12:54:38 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791TKcB017274 for ; Tue, 9 Aug 2022 03:54:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=G9+4OASF4f4mKbD48Et0mGVaWH51wKm4v6VDnHglbaE=; b=XaNBezPkHzqfe9kaKkWM30gpwGAInYPgl3zwy2qdzLPUim0krdO/PKnVkQ+iTrroHnjQ HOgK3UmNB4ApzzGmsiAaJO/0/H2q7ISKOkkjtgixLhBhrHjrnD+JBDPQWAuQMffyCZCv zk/Kixc8e+87Gu3cM3ryHd5vX6Wum7AlKyxHqk+Ic+qa9tkzZovGh1YkytRawOLlabFr egytPGUFkyXhrOg3MWi+/OiojYQ9qmELUCVKb0ySRM+a7pGeGU3aHKe8HCC5fgZtK0ye ofrih+SC0tatMQ5BwbVrCgTkvUT8LgfYaPOtfNY7gAelDZs0wifkBhhG4mQ7GRmoVznL dQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hudy6spk0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:37 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:35 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Aug 2022 03:54:36 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id E35F83F7048; Tue, 9 Aug 2022 03:54:33 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 14/18] crypto/cnxk: use dedicated dp threads depending on operation Date: Tue, 9 Aug 2022 16:23:52 +0530 Message-ID: <20220809105356.561-15-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: OVTw0YSR7thvEVuNAv4qVZgIAKMrUcU_ X-Proofpoint-ORIG-GUID: OVTw0YSR7thvEVuNAv4qVZgIAKMrUcU_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Identify the datapath thread to be used during session create. This can be used to call right function early on to avoid multiple session specific checks in datapath functions. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 29 +++++++++++++ drivers/crypto/cnxk/cnxk_se.h | 53 +++++++++++++++--------- 2 files changed, 62 insertions(+), 20 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 85bb1d27a1..cf91b92c2c 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -625,6 +625,7 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id, struct rte_cryptodev_sym_session *sess, struct rte_mempool *pool) { + enum cpt_dp_thread_type thr_type; struct cnxk_se_sess *sess_priv; void *priv; int ret; @@ -642,6 +643,34 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id, if (ret) goto priv_put; + if (sess_priv->cpt_op & ROC_SE_OP_CIPHER_MASK) { + switch (sess_priv->roc_se_ctx.fc_type) { + case ROC_SE_FC_GEN: + if (sess_priv->aes_gcm || sess_priv->chacha_poly) + thr_type = CPT_DP_THREAD_TYPE_FC_AEAD; + else + thr_type = CPT_DP_THREAD_TYPE_FC_CHAIN; + break; + case ROC_SE_PDCP: + thr_type = CPT_DP_THREAD_TYPE_PDCP; + break; + case ROC_SE_KASUMI: + thr_type = CPT_DP_THREAD_TYPE_KASUMI; + break; + case ROC_SE_PDCP_CHAIN: + thr_type = CPT_DP_THREAD_TYPE_PDCP_CHAIN; + break; + default: + plt_err("Invalid op type"); + ret = -ENOTSUP; + goto priv_put; + } + } else { + thr_type = CPT_DP_THREAD_AUTH_ONLY; + } + + sess_priv->dp_thr_type = thr_type; + if ((sess_priv->roc_se_ctx.fc_type == ROC_SE_HASH_HMAC) && cpt_mac_len_verify(&xform->auth)) { plt_dp_err("MAC length is not supported"); diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index d8126567f9..99fb3b899e 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -16,6 +16,15 @@ (sizeof(struct roc_se_iov_ptr) + \ (sizeof(struct roc_se_buf_ptr) * ROC_SE_MAX_SG_CNT)) +enum cpt_dp_thread_type { + CPT_DP_THREAD_TYPE_FC_CHAIN = 0x1, + CPT_DP_THREAD_TYPE_FC_AEAD, + CPT_DP_THREAD_TYPE_PDCP, + CPT_DP_THREAD_TYPE_PDCP_CHAIN, + CPT_DP_THREAD_TYPE_KASUMI, + CPT_DP_THREAD_AUTH_ONLY, +}; + struct cnxk_se_sess { uint16_t cpt_op : 4; uint16_t zsk_flag : 4; @@ -29,7 +38,7 @@ struct cnxk_se_sess { uint16_t aes_ctr_eea2 : 1; uint16_t zs_cipher : 4; uint16_t zs_auth : 4; - uint16_t rsvd2 : 8; + uint16_t dp_thr_type : 8; uint16_t aad_length; uint8_t mac_len; uint8_t iv_length; @@ -2370,7 +2379,7 @@ prepare_iov_from_pkt_inplace(struct rte_mbuf *pkt, static __rte_always_inline int fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, - struct cpt_inst_s *inst, const bool is_kasumi) + struct cpt_inst_s *inst, const bool is_kasumi, const bool is_aead) { struct rte_crypto_sym_op *sym_op = cop->sym; void *mdata = NULL; @@ -2418,7 +2427,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, m_src = sym_op->m_src; m_dst = sym_op->m_dst; - if (sess->aes_gcm || sess->chacha_poly) { + if (is_aead) { struct rte_mbuf *m; uint8_t *salt; uint8_t *aad_data; @@ -2567,6 +2576,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } + fc_params.meta_buf.vaddr = NULL; if (unlikely(is_kasumi || !((flags & ROC_SE_SINGLE_BUF_INPLACE) && (flags & ROC_SE_SINGLE_BUF_HEADROOM)))) { mdata = alloc_op_meta(&fc_params.meta_buf, m_info->mlen, m_info->pool, infl_req); @@ -3057,26 +3067,29 @@ static __rte_always_inline int __rte_hot cpt_sym_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cnxk_se_sess *sess, struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst) { - uint64_t cpt_op = sess->cpt_op; int ret; - if (cpt_op & ROC_SE_OP_CIPHER_MASK) { - switch (sess->roc_se_ctx.fc_type) { - case ROC_SE_PDCP_CHAIN: - ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); - break; - case ROC_SE_PDCP: - ret = fill_pdcp_params(op, sess, &qp->meta_info, infl_req, inst); - break; - case ROC_SE_KASUMI: - ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, true); - break; - default: - ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, false); - break; - } - } else { + switch (sess->dp_thr_type) { + case CPT_DP_THREAD_TYPE_PDCP: + ret = fill_pdcp_params(op, sess, &qp->meta_info, infl_req, inst); + break; + case CPT_DP_THREAD_TYPE_FC_CHAIN: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, false, false); + break; + case CPT_DP_THREAD_TYPE_FC_AEAD: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, false, true); + break; + case CPT_DP_THREAD_TYPE_PDCP_CHAIN: + ret = fill_pdcp_chain_params(op, sess, &qp->meta_info, infl_req, inst); + break; + case CPT_DP_THREAD_TYPE_KASUMI: + ret = fill_fc_params(op, sess, &qp->meta_info, infl_req, inst, true, false); + break; + case CPT_DP_THREAD_AUTH_ONLY: ret = fill_digest_params(op, sess, &qp->meta_info, infl_req, inst); + break; + default: + ret = -EINVAL; } return ret; From patchwork Tue Aug 9 10:53:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114759 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BB41EA00C2; Tue, 9 Aug 2022 12:55:42 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6BEA642C61; Tue, 9 Aug 2022 12:54:42 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id BD72B42C72 for ; Tue, 9 Aug 2022 12:54:40 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791G3aN015673 for ; Tue, 9 Aug 2022 03:54:40 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=AH0Waz0PuA6mT91suK6N2P/S1TtgWI0nOXw3uEUg1ac=; b=SiDQO8P3njjc/7PnbF9Lv9A4rrUTuI3pNmcxt6Sa8Inn1xO1NGb9P79i2iADOEICOCgv IlE6u+EveGI4HZAxjvcQlpLTYVLr1R0IjM/z5C3KGjbomc6XDA3r5nlLkpNeFQisBQKx xf45SE+Fk/rXsGWBr6wsBd4yWph4Deo49LikPQr4o2wQ6vzNzCzUxpHzeMUNM6c373h4 4R12+vgyh/2XZ8NwLOcV6aJ2w0gYTCWdju2SdxHOWeFftwmD0aEpOQM6CX+tYK1n1oxU Q/uU0xiXDczLvA9vwZwDLDSmtJdwn2KHLIQCxPEaaEU9sEKhv4qsVdaKU+XbCTEN/QoK EA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3huds2sq8w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:39 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:38 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:38 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id 74FD13F7065; Tue, 9 Aug 2022 03:54:36 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 15/18] crypto/cnxk: remove unused ctx buf len Date: Tue, 9 Aug 2022 16:23:53 +0530 Message-ID: <20220809105356.561-16-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: oMAqxr8VMngy4XENzcMXFp8KvA4glbsP X-Proofpoint-GUID: oMAqxr8VMngy4XENzcMXFp8KvA4glbsP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Ctx len buf field is unused. Remove it. Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_se.h | 45 +++++++++++++++-------------------- drivers/crypto/cnxk/cnxk_se.h | 27 +++++++++++---------- 2 files changed, 33 insertions(+), 39 deletions(-) diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index 86bb3aa79d..d1a87a96da 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -287,32 +287,6 @@ struct roc_se_iov_ptr { struct roc_se_buf_ptr bufs[]; }; -struct roc_se_fc_params { - /* 0th cache line */ - union { - struct roc_se_buf_ptr bufs[1]; - struct { - struct roc_se_iov_ptr *src_iov; - struct roc_se_iov_ptr *dst_iov; - }; - }; - void *iv_buf; - void *auth_iv_buf; - struct roc_se_buf_ptr meta_buf; - struct roc_se_buf_ptr ctx_buf; - uint32_t rsvd2; - uint8_t rsvd3; - uint8_t iv_ovr; - uint8_t cipher_iv_len; - uint8_t auth_iv_len; - - /* 1st cache line */ - struct roc_se_buf_ptr aad_buf __plt_cache_aligned; - struct roc_se_buf_ptr mac_buf; -}; - -PLT_STATIC_ASSERT((offsetof(struct roc_se_fc_params, aad_buf) % 128) == 0); - #define ROC_SE_PDCP_ALG_TYPE_ZUC 0 #define ROC_SE_PDCP_ALG_TYPE_SNOW3G 1 #define ROC_SE_PDCP_ALG_TYPE_AES_CTR 2 @@ -349,6 +323,25 @@ struct roc_se_ctx { uint8_t *auth_key; }; +struct roc_se_fc_params { + union { + struct roc_se_buf_ptr bufs[1]; + struct { + struct roc_se_iov_ptr *src_iov; + struct roc_se_iov_ptr *dst_iov; + }; + }; + void *iv_buf; + void *auth_iv_buf; + struct roc_se_ctx *ctx; + struct roc_se_buf_ptr meta_buf; + uint8_t cipher_iv_len; + uint8_t auth_iv_len; + + struct roc_se_buf_ptr aad_buf; + struct roc_se_buf_ptr mac_buf; +}; + static inline void roc_se_zuc_bytes_swap(uint8_t *arr, int len) { diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 99fb3b899e..33e65eb4e3 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -334,7 +334,7 @@ cpt_digest_gen_prep(uint32_t flags, uint64_t d_lens, uint32_t g_size_bytes, s_size_bytes; union cpt_inst_w4 cpt_inst_w4; - ctx = params->ctx_buf.vaddr; + ctx = params->ctx; hash_type = ctx->hash_type; mac_len = ctx->mac_len; @@ -478,7 +478,8 @@ cpt_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, aad_len = fc_params->aad_buf.size; aad_buf = &fc_params->aad_buf; } - se_ctx = fc_params->ctx_buf.vaddr; + + se_ctx = fc_params->ctx; cipher_type = se_ctx->enc_cipher; hash_type = se_ctx->hash_type; mac_len = se_ctx->mac_len; @@ -777,7 +778,7 @@ cpt_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, aad_buf = &fc_params->aad_buf; } - se_ctx = fc_params->ctx_buf.vaddr; + se_ctx = fc_params->ctx; hash_type = se_ctx->hash_type; mac_len = se_ctx->mac_len; op_minor = se_ctx->template_w4.s.opcode_minor; @@ -1087,7 +1088,7 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, return -1; } - se_ctx = params->ctx_buf.vaddr; + se_ctx = params->ctx; mac_len = se_ctx->mac_len; pdcp_ci_alg = se_ctx->pdcp_ci_alg; pdcp_auth_alg = se_ctx->pdcp_auth_alg; @@ -1281,7 +1282,7 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, uint8_t pack_iv = 0; union cpt_inst_w4 cpt_inst_w4; - se_ctx = params->ctx_buf.vaddr; + se_ctx = params->ctx; flags = se_ctx->zsk_flags; mac_len = se_ctx->mac_len; @@ -1562,7 +1563,7 @@ cpt_kasumi_enc_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, encr_data_len = ROC_SE_ENCR_DLEN(d_lens); auth_data_len = ROC_SE_AUTH_DLEN(d_lens); - se_ctx = params->ctx_buf.vaddr; + se_ctx = params->ctx; flags = se_ctx->zsk_flags; mac_len = se_ctx->mac_len; @@ -1751,7 +1752,7 @@ cpt_kasumi_dec_prep(uint64_t d_offs, uint64_t d_lens, encr_offset = ROC_SE_ENCR_OFFSET(d_offs) / 8; encr_data_len = ROC_SE_ENCR_DLEN(d_lens); - se_ctx = params->ctx_buf.vaddr; + se_ctx = params->ctx; flags = se_ctx->zsk_flags; cpt_inst_w4.u64 = 0; @@ -1863,7 +1864,7 @@ static __rte_always_inline int cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, struct roc_se_fc_params *fc_params, struct cpt_inst_s *inst) { - struct roc_se_ctx *ctx = fc_params->ctx_buf.vaddr; + struct roc_se_ctx *ctx = fc_params->ctx; uint8_t fc_type; int ret = -1; @@ -2410,7 +2411,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, flags |= ROC_SE_VALID_IV_BUF; fc_params.iv_buf = rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset); - if (sess->aes_ctr && unlikely(sess->iv_length != 16)) { + if (!is_aead && sess->aes_ctr && unlikely(sess->iv_length != 16)) { memcpy((uint8_t *)iv_buf, rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset), @@ -2520,7 +2521,7 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, } } } - fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + fc_params.ctx = &sess->roc_se_ctx; if (!(sess->auth_first) && unlikely(sess->is_null || sess->cpt_op == ROC_SE_OP_DECODE)) inplace = 0; @@ -2649,7 +2650,7 @@ fill_pdcp_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, d_offs = (uint64_t)c_data_off << 16; d_lens = (uint64_t)c_data_len << 32; - fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + fc_params.ctx = &sess->roc_se_ctx; if (likely(m_dst == NULL || m_src == m_dst)) { fc_params.dst_iov = fc_params.src_iov = (void *)src; @@ -2792,7 +2793,7 @@ fill_pdcp_chain_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, ci_data_length, true); } - fc_params.ctx_buf.vaddr = &sess->roc_se_ctx; + fc_params.ctx = &sess->roc_se_ctx; if (likely((m_dst == NULL || m_dst == m_src)) && inplace) { fc_params.dst_iov = fc_params.src_iov = (void *)src; @@ -2991,7 +2992,7 @@ fill_digest_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, d_lens = sym_op->auth.data.length; - params.ctx_buf.vaddr = &sess->roc_se_ctx; + params.ctx = &sess->roc_se_ctx; if (auth_op == ROC_SE_OP_AUTH_GENERATE) { if (sym_op->auth.digest.data) { From patchwork Tue Aug 9 10:53:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114760 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0FEF8A00C2; Tue, 9 Aug 2022 12:55:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 77AFA42BB4; Tue, 9 Aug 2022 12:54:45 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id CC51F42BA7 for ; Tue, 9 Aug 2022 12:54:43 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791T1GE017014 for ; Tue, 9 Aug 2022 03:54:43 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=yzdtJukYBBK498KjRKiFJONTbglThZ/kE5vR9MfKkvg=; b=BNf/CMmEte1caFRvEVf7QftsyepsE9o6corbUCCYRr2vXIW85n/zCyoXTSiC0Botqdaq 9LK4UStImoxEKC+J4vP4r09w9swQ2JVUVGvnoIZ1zdnkGHqqiETLxgHGkm/70Fr/gsxX umjuPOiFBGjuIBsdqCq1VHL0ayN5aECTr8P9QbdbHRzPTJLAUQ3X8778nqUHIZ8hOZhq aGrdt2ahhvySYfjravUvnu1sgolC6WeD4c5ohej/uyquZiiKC08T66cG+zSArVA6+3sb KcyK2XEcfxNpi51o399lieRL1R3/dInVhY6gOmJ+v/vXfdpknOb/yF+fizPqmJcgtXDi tQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hudy6spkb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:43 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:41 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Aug 2022 03:54:41 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id 076CE3F7048; Tue, 9 Aug 2022 03:54:38 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Shijith Thotton , Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 16/18] drivers: change crypto adapter datapath error print to debug Date: Tue, 9 Aug 2022 16:23:54 +0530 Message-ID: <20220809105356.561-17-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: vOWuHrSihvwJOra-f1XSVIWZCLIGjz9Y X-Proofpoint-ORIG-GUID: vOWuHrSihvwJOra-f1XSVIWZCLIGjz9Y X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Shijith Thotton Errors in crypto adapter datapath can be handled gracefully. So the error print can be moved under debug. Signed-off-by: Shijith Thotton --- drivers/common/cnxk/roc_platform.h | 2 ++ drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 502f243a81..0cde3e0cc3 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -269,6 +269,8 @@ extern int cnxk_logtype_ree; RTE_LOG_DP(ERR, PMD, "%s():%u " fmt "\n", __func__, __LINE__, ##args) #define plt_dp_info(fmt, args...) \ RTE_LOG_DP(INFO, PMD, "%s():%u " fmt "\n", __func__, __LINE__, ##args) +#define plt_dp_dbg(fmt, args...) \ + RTE_LOG_DP(DEBUG, PMD, "%s():%u " fmt "\n", __func__, __LINE__, ##args) #ifdef __cplusplus #define CNXK_PCI_ID(subsystem_dev, dev) \ diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 1b70d02e2a..586941cd70 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -412,7 +412,7 @@ ca_lmtst_burst_submit(struct cn10k_sso_hws *ws, uint64_t w2[], struct cnxk_cpt_q ret = cn10k_cpt_fill_inst(qp, &op[i], inst, infl_req); if (unlikely(ret != 1)) { - plt_dp_err("Could not process op: %p", op[i]); + plt_dp_dbg("Could not process op: %p", op[i]); if (i != 0) goto submit; else From patchwork Tue Aug 9 10:53:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114761 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 64E74A00C2; Tue, 9 Aug 2022 12:55:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 513E542C6B; Tue, 9 Aug 2022 12:54:48 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id A225742B86 for ; Tue, 9 Aug 2022 12:54:46 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791Fxlt015643 for ; Tue, 9 Aug 2022 03:54:45 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=4XaXNmXDy6LZe8U10TIjmztsFqO/SLT0I/gX7GfL2DQ=; b=fYN97RyNA875pIh2ZC69bu4wTRv/5mbw3AOjKBW1Gre84jQeei8URqCzplmpvpyj+pXs wGtWrhfrVdiKHRf76uenJuuFLz5+9oNXAVznmZGIHkyRxpk0TUtLkXNSj8SQ7GWD8PB5 4sFmukIRyf3FYsyCtGy+uR9T5Qw5l+9sdrPjOhPuxaEmd4j//yG+05ejvvzvtdFiGuwO AjKCTOZlxSSV6Pm9Uj4gF0Iw6OvB3UB+amlp7L0s7/c4o+/mMYUavgG60iqmcAzd4X3p muwPDFm0eMIaGRjEfMmCbAhWemM21YpLMxZrg+eJJbcwalx5ghdU1sIAL6z2MSW3mNB9 Ow== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3huds2sq99-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:45 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 9 Aug 2022 03:54:44 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 9 Aug 2022 03:54:44 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id DF5393F704C; Tue, 9 Aug 2022 03:54:41 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Vamsi Attunuru , Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 17/18] crypto/cnxk: update flow label copy capability Date: Tue, 9 Aug 2022 16:23:55 +0530 Message-ID: <20220809105356.561-18-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: urno4tlMAZVuWLIVCpj2q3YPO8osM34W X-Proofpoint-GUID: urno4tlMAZVuWLIVCpj2q3YPO8osM34W X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vamsi Attunuru Update flow label copy capability in IPsec options. Signed-off-by: Vamsi Attunuru --- drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index fdc646a6fc..ffb0c289a0 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -1294,6 +1294,7 @@ cn10k_sec_caps_update(struct rte_security_capability *sec_cap) sec_cap->ipsec.options.l4_csum_enable = 1; sec_cap->ipsec.options.stats = 1; sec_cap->ipsec.options.esn = 1; + sec_cap->ipsec.options.copy_flabel = 1; sec_cap->ipsec.replay_win_sz_max = ROC_AR_WIN_SIZE_MAX; } From patchwork Tue Aug 9 10:53:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 114762 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B23C8A00C2; Tue, 9 Aug 2022 12:55:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3A39842BC7; Tue, 9 Aug 2022 12:54:51 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id B3DC642B86 for ; Tue, 9 Aug 2022 12:54:49 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2791THGS017221 for ; Tue, 9 Aug 2022 03:54:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=OFqBPpzYZsQNOPVb+qATknD0l8FcnDI2I5YExfr7HhM=; b=P8Zy6bBb3ujp2qghw4GicDU3sd9wbf2JsKlVrnB1WjWynBYIejYwiemzsXqcjn9khMLs sHfWOuAm37JBdwvKxzGbQANV4vhIVXs77g1PRdgjlZk1zdZo2opLC1T2gKPO4JyiDMxo B6aWcjp6i9VRfYP8zKA3clM/1OsK7m49jFqTujJIZyyKSPacHHk1U9BCFFV/vMuNLHrx BbVD3McyFfMmzltBDtNQD73rlBUMUVrIeMdiyGND8sYu3ECXRJyYspZ5izvqll7JuGA/ Wi1BWkD/sg8ZJXcSDuISW9UjE19CbsopiyPnVo7Sc+/gTt95875iu9fs0u0O/gKw4N6M aA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hudy6spkr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 09 Aug 2022 03:54:49 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Aug 2022 03:54:47 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Aug 2022 03:54:47 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.69.70]) by maili.marvell.com (Postfix) with ESMTP id BDABC3F7048; Tue, 9 Aug 2022 03:54:44 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Volodymyr Fialko , Archana Muniganti , Tejasree Kondoj , Subject: [PATCH v2 18/18] crypto/cnxk: add support for DOCSIS algorithm Date: Tue, 9 Aug 2022 16:23:56 +0530 Message-ID: <20220809105356.561-19-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220809105356.561-1-anoobj@marvell.com> References: <20220808080606.220-1-anoobj@marvell.com> <20220809105356.561-1-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: go_AOOElWdAM2Agwt0UqJQUORbbCB_wr X-Proofpoint-ORIG-GUID: go_AOOElWdAM2Agwt0UqJQUORbbCB_wr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-09_03,2022-08-09_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Add support for offloading RTE_CRYPTO_CIPHER_AES_DOCSISBPI and RTE_CRYPTO_CIPHER_DES_DOCSISBPI algorithms to symmetric crypto session. Signed-off-by: Volodymyr Fialko --- doc/guides/cryptodevs/features/cn9k.ini | 2 + doc/guides/rel_notes/release_22_11.rst | 1 + drivers/common/cnxk/roc_se.c | 25 ++++++++- drivers/common/cnxk/roc_se.h | 11 ++-- drivers/crypto/cnxk/cnxk_cryptodev.h | 2 +- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 52 +++++++++++++++++++ drivers/crypto/cnxk/cnxk_se.h | 12 +++++ 7 files changed, 99 insertions(+), 6 deletions(-) diff --git a/doc/guides/cryptodevs/features/cn9k.ini b/doc/guides/cryptodevs/features/cn9k.ini index 98ad7cf10a..c3d131db1a 100644 --- a/doc/guides/cryptodevs/features/cn9k.ini +++ b/doc/guides/cryptodevs/features/cn9k.ini @@ -35,6 +35,8 @@ DES CBC = Y KASUMI F8 = Y SNOW3G UEA2 = Y ZUC EEA3 = Y +AES DOCSIS BPI = Y +DES DOCSIS BPI = Y ; ; Supported authentication algorithms of 'cn9k' crypto driver. diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 333f66bef3..7fab9d6550 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -58,6 +58,7 @@ New Features * **Updated Marvell cnxk crypto driver.** * Added AES-CCM support in lookaside protocol (IPsec) for CN9K & CN10K. + * Added AES & DES DOCSIS algorithm support in lookaside crypto for CN9K. Removed Items diff --git a/drivers/common/cnxk/roc_se.c b/drivers/common/cnxk/roc_se.c index 8d6446c3a0..2663480099 100644 --- a/drivers/common/cnxk/roc_se.c +++ b/drivers/common/cnxk/roc_se.c @@ -63,6 +63,7 @@ cpt_ciph_type_set(roc_se_cipher_type type, struct roc_se_ctx *ctx, break; case ROC_SE_DES3_CBC: case ROC_SE_DES3_ECB: + case ROC_SE_DES_DOCSISBPI: fc_type = ROC_SE_FC_GEN; break; case ROC_SE_AES_CBC: @@ -70,6 +71,7 @@ cpt_ciph_type_set(roc_se_cipher_type type, struct roc_se_ctx *ctx, case ROC_SE_AES_CFB: case ROC_SE_AES_CTR: case ROC_SE_AES_GCM: + case ROC_SE_AES_DOCSISBPI: if (unlikely(cpt_ciph_aes_key_validate(key_len) != 0)) return -1; fc_type = ROC_SE_FC_GEN; @@ -451,7 +453,7 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, uint8_t *zuc_const; uint32_t keyx[4]; uint8_t *ci_key; - int ret; + int i, ret; zs_ch_ctx = &se_ctx->se_ctx.zs_ch_ctx; @@ -531,6 +533,27 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, memset(fctx->hmac.ipad, 0, sizeof(fctx->hmac.ipad)); memcpy(fctx->hmac.ipad, &key[key_len], key_len); break; + case ROC_SE_AES_DOCSISBPI: + /* + * DOCSIS uses the combination of AES-CBC and residual termination blocks that are + * less than 128. Pass it as regular AES-CBC cipher to CPT, but keep type in + * se_ctx as AES_DOCSISBPI to skip block size checks in instruction preparation. + */ + cpt_ciph_aes_key_type_set(fctx, key_len); + fctx->enc.enc_cipher = ROC_SE_AES_CBC; + memcpy(fctx->enc.encr_key, key, key_len); + goto success; + case ROC_SE_DES_DOCSISBPI: + /* See case ROC_SE_DES3_CBC: for explanation */ + for (i = 0; i < 3; i++) + memcpy(fctx->enc.encr_key + key_len * i, key, key_len); + /* + * DOCSIS uses DES-CBC mode with special handling of residual termination blocks + * that are less than 64 bits. Pass it as regular DES-CBC, but keep type in + * se_ctx as DES_DOCSISBPI to skip block size checks in instruction preparation. + */ + fctx->enc.enc_cipher = ROC_SE_DES3_CBC; + goto success; case ROC_SE_SNOW3G_UEA2: if (chained_op == true) { struct roc_se_onk_zuc_chain_ctx *ctx = diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h index d1a87a96da..e70a197d4f 100644 --- a/drivers/common/cnxk/roc_se.h +++ b/drivers/common/cnxk/roc_se.h @@ -10,6 +10,7 @@ #define ROC_SE_FC_MINOR_OP_ENCRYPT 0x0 #define ROC_SE_FC_MINOR_OP_DECRYPT 0x1 #define ROC_SE_FC_MINOR_OP_HMAC_FIRST 0x10 +#define ROC_SE_FC_MINOR_OP_DOCSIS 0x40 #define ROC_SE_MAJOR_OP_HASH 0x34 #define ROC_SE_MAJOR_OP_HMAC 0x35 @@ -17,10 +18,10 @@ #define ROC_SE_MAJOR_OP_KASUMI 0x38 #define ROC_SE_MAJOR_OP_PDCP_CHAIN 0x3C -#define ROC_SE_MAJOR_OP_MISC 0x01 -#define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03 -#define ROC_SE_MISC_MINOR_OP_DUMMY 0x04 -#define ROC_SE_MISC_MINOR_OP_HW_SUPPORT 0x08 +#define ROC_SE_MAJOR_OP_MISC 0x01ULL +#define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03ULL +#define ROC_SE_MISC_MINOR_OP_DUMMY 0x04ULL +#define ROC_SE_MISC_MINOR_OP_HW_SUPPORT 0x08ULL #define ROC_SE_MAX_AAD_SIZE 64 #define ROC_SE_MAX_MAC_LEN 64 @@ -125,6 +126,8 @@ typedef enum { ROC_SE_AES_CTR_EEA2 = 0x92, ROC_SE_KASUMI_F8_CBC = 0x93, ROC_SE_KASUMI_F8_ECB = 0x94, + ROC_SE_AES_DOCSISBPI = 0x95, + ROC_SE_DES_DOCSISBPI = 0x96, } roc_se_cipher_type; typedef enum { diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index a3dcfbfa6d..588760cfb0 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -10,7 +10,7 @@ #include "roc_cpt.h" -#define CNXK_CPT_MAX_CAPS 35 +#define CNXK_CPT_MAX_CAPS 37 #define CNXK_SEC_CRYPTO_MAX_CAPS 14 #define CNXK_SEC_MAX_CAPS 9 #define CNXK_AE_EC_ID_MAX 8 diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index ffb0c289a0..1fb35f54cd 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -697,6 +697,49 @@ static const struct rte_cryptodev_capabilities caps_des[] = { }, }; +static const struct rte_cryptodev_capabilities caps_docsis[] = { + { /* AES DOCSIS BPI */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_DOCSISBPI, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 16 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* DES DOCSIS BPI */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_DES_DOCSISBPI, + .block_size = 8, + .key_size = { + .min = 8, + .max = 8, + .increment = 0 + }, + .iv_size = { + .min = 8, + .max = 8, + .increment = 0 + } + }, } + }, } + }, +}; + static const struct rte_cryptodev_capabilities caps_null[] = { { /* NULL (AUTH) */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, @@ -1158,6 +1201,12 @@ cn10k_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[]) } } +static void +cn9k_crypto_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos) +{ + cpt_caps_add(cnxk_caps, cur_pos, caps_docsis, RTE_DIM(caps_docsis)); +} + static void crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], union cpt_eng_caps *hw_caps) @@ -1172,6 +1221,9 @@ crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[], CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, kasumi); CPT_CAPS_ADD(cnxk_caps, &cur_pos, hw_caps, des); + if (!roc_model_is_cn10k()) + cn9k_crypto_caps_add(cnxk_caps, &cur_pos); + cpt_caps_add(cnxk_caps, &cur_pos, caps_null, RTE_DIM(caps_null)); cpt_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end)); diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 33e65eb4e3..54a78d0a5a 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2032,6 +2032,18 @@ fill_sess_cipher(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) enc_type = ROC_SE_AES_ECB; cipher_key_len = 16; break; + case RTE_CRYPTO_CIPHER_AES_DOCSISBPI: + /* Set DOCSIS flag */ + sess->roc_se_ctx.template_w4.s.opcode_minor |= ROC_SE_FC_MINOR_OP_DOCSIS; + enc_type = ROC_SE_AES_DOCSISBPI; + cipher_key_len = 16; + break; + case RTE_CRYPTO_CIPHER_DES_DOCSISBPI: + /* Set DOCSIS flag */ + sess->roc_se_ctx.template_w4.s.opcode_minor |= ROC_SE_FC_MINOR_OP_DOCSIS; + enc_type = ROC_SE_DES_DOCSISBPI; + cipher_key_len = 8; + break; case RTE_CRYPTO_CIPHER_3DES_CTR: case RTE_CRYPTO_CIPHER_AES_F8: case RTE_CRYPTO_CIPHER_ARC4: