From patchwork Wed Jun 26 10:55:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141761 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E2A1745500; Wed, 26 Jun 2024 13:00:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A83B842EE0; Wed, 26 Jun 2024 13:00:21 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 9226740B95 for ; Wed, 26 Jun 2024 12:55:43 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q9jdjo020506 for ; Wed, 26 Jun 2024 03:55:42 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=4 S4ksnydXUEsJuuAdvroPu1vA+FAxc9EZ2KxLR9G0eU=; b=Tjp88QSKU1PGlVkLG zvKadO4864hOYxbU77NDE9ysWOzitd3gVLtjEfb3rkjcJ80QsrBS5bsPEOFpcvy8 glx4dA3HcCIAjrbxe6zCax3mhufJSnpDs4jiYhDEeojc2qpBz8D5IdYJ9vQLhQEW 7SbfC0LGcrtgkCLS9VK0/wOrRRjltBe2o6PDyhDLjp0HbpjlTfZsriwVI/kA9G4E 9fLjZSgKpdn4eMdkUZ0zxqEL23H3L+Hm8X8Wi2LxwiLE0R446yKwrpo1jnsSWIQi uuflHLZ6BZFBFir49sfBCjEmDMZqGPEVbxxgwjXSCV8/v3LllMK1RkaJnU4weguD pvwNA== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 400ftngbvv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:55:42 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:55:41 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:55:41 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id CBBC33F7079; Wed, 26 Jun 2024 03:55:37 -0700 (PDT) From: Aakash Sasidharan To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra CC: , , , , , Subject: [PATCH v3 01/12] common/cnxk: add comments to denote skipped entries Date: Wed, 26 Jun 2024 16:25:23 +0530 Message-ID: <20240626105534.1386528-2-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: uu6xcIHnPWgAYaTg8Duk0UA9UnN6zJLd X-Proofpoint-GUID: uu6xcIHnPWgAYaTg8Duk0UA9UnN6zJLd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Add comments to denote unused table entries. Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_ae.c | 6 +++--- drivers/common/cnxk/roc_ae_fpm_tables.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/common/cnxk/roc_ae.c b/drivers/common/cnxk/roc_ae.c index e6a013d7c4..7ef0efe2b3 100644 --- a/drivers/common/cnxk/roc_ae.c +++ b/drivers/common/cnxk/roc_ae.c @@ -151,9 +151,9 @@ const struct roc_ae_ec_group ae_ec_grp[ROC_AE_EC_ID_PMAX] = { 0x3F, 0x00}, .length = 66}, }, - {}, - {}, - {}, + { /* ROC_AE_EC_ID_P160 */ }, + { /* ROC_AE_EC_ID_P320 */ }, + { /* ROC_AE_EC_ID_P512 */ }, { .prime = {.data = {0xFF, 0xFF, 0xFF, 0xFE, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, diff --git a/drivers/common/cnxk/roc_ae_fpm_tables.c b/drivers/common/cnxk/roc_ae_fpm_tables.c index ead3128e7f..942657b56a 100644 --- a/drivers/common/cnxk/roc_ae_fpm_tables.c +++ b/drivers/common/cnxk/roc_ae_fpm_tables.c @@ -1261,9 +1261,9 @@ const struct ae_fpm_entry ae_fpm_tbl_scalar[ROC_AE_EC_ID_PMAX] = { .data = ae_fpm_tbl_p521, .len = sizeof(ae_fpm_tbl_p521) }, - {}, - {}, - {}, + { /* ROC_AE_EC_ID_P160 */ }, + { /* ROC_AE_EC_ID_P320 */ }, + { /* ROC_AE_EC_ID_P512 */ }, { .data = ae_fpm_tbl_p256_sm2, .len = sizeof(ae_fpm_tbl_p256_sm2) From patchwork Wed Jun 26 10:55:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141762 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7FB3C45500; Wed, 26 Jun 2024 13:00:32 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EA4C342EFB; Wed, 26 Jun 2024 13:00:22 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 4D75240B95; Wed, 26 Jun 2024 12:55:46 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q8oNRG024693; Wed, 26 Jun 2024 03:55:45 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=l WuPr8cIy5lc3vLBtWqdwcu8bmRHPe4rKpNHy18XeMM=; b=UTgsRcPWPCwQeLHv1 GltvZfS2Aq6/l0rvvXuURK2ocFy0o68jB6QnDoMWiD6gIRJtAZI7GS0Of5N8bUH8 IqyD9s9sVGXe43LSlKb9kCqUc8NGTGKl383mUukC4eoAQGm5koi3I1cLH9kyb4pY A5xoF9AInPg+PQTMzIRvzcPN6cauDFwHtBsUS4ivEITDIRKkJ16YI4kuquBtfIuq s2b832s+iCBw783N3VFhXVkmXU5xECn9eVIHAuKdmJPTJJXzYB91Y8TL3fvHwtrB QGrUzO9la0Zf2MPJnogB/xSTV0wEuKPEso6ni9Bx96qEaGNsFhyi4CjRPizZN4RS 0at+w== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 400ftngbw0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 26 Jun 2024 03:55:45 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:55:44 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:55:44 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id BC7DC3F7079; Wed, 26 Jun 2024 03:55:41 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Akhil Goyal CC: , , , , Subject: [PATCH v3 02/12] crypto/cnxk: update version map file with PMD APIs Date: Wed, 26 Jun 2024 16:25:24 +0530 Message-ID: <20240626105534.1386528-3-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: mDBiXsTePXQNwdUDWTvlZY7aQJZEgbfE X-Proofpoint-GUID: mDBiXsTePXQNwdUDWTvlZY7aQJZEgbfE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Update version map with details of PMD APIs added. Fixes: 26bb5c4de63e ("crypto/cnxk: add CPT raw submission PMD API") Cc: stable@dpdk.org Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h | 2 ++ drivers/crypto/cnxk/version.map | 8 ++++++++ 2 files changed, 10 insertions(+) diff --git a/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h b/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h index 8b0a5ba0f2..eab1243065 100644 --- a/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h +++ b/drivers/crypto/cnxk/rte_pmd_cnxk_crypto.h @@ -23,6 +23,7 @@ * @return * Pointer to queue pair structure that would be the input to submit APIs. */ +__rte_experimental void *rte_pmd_cnxk_crypto_qptr_get(uint8_t dev_id, uint16_t qp_id); /** @@ -41,6 +42,7 @@ void *rte_pmd_cnxk_crypto_qptr_get(uint8_t dev_id, uint16_t qp_id); * @param nb_inst * Number of instructions. */ +__rte_experimental void rte_pmd_cnxk_crypto_submit(void *qptr, void *inst, uint16_t nb_inst); #endif /* _PMD_CNXK_CRYPTO_H_ */ diff --git a/drivers/crypto/cnxk/version.map b/drivers/crypto/cnxk/version.map index 5789a6bfc9..7a77607774 100644 --- a/drivers/crypto/cnxk/version.map +++ b/drivers/crypto/cnxk/version.map @@ -1,3 +1,11 @@ +EXPERIMENTAL { + global: + + # added in 24.03 + rte_pmd_cnxk_crypto_submit; + rte_pmd_cnxk_crypto_qptr_get; +}; + INTERNAL { global: From patchwork Wed Jun 26 10:55:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141763 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E19CB45500; Wed, 26 Jun 2024 13:00:38 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0277442EF8; Wed, 26 Jun 2024 13:00:24 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 66EEA40B95 for ; Wed, 26 Jun 2024 12:55:50 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q5S1vV005422 for ; Wed, 26 Jun 2024 03:55:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=F 9Qc9z2CHV/xfwfzvr+jja5tzDz5xLn+Og8k8z9Zrfo=; b=kl4FKE6OvvnlOj6RT F3Lzd6J9hbD8eospognhV73fv+yMaCCXI1I92bnHIRduaAIcgTQOT5yKZxPxK1WU VkBskwn/6HlhbCcXDMQ8/rNTrNsWsqDmF2BFSKriD94v0jYS2tkzmQ9IonHD6BFr eP7i3mKoo0VPUzNjWztXy2tMbtCdtUlj91qjPNFNiZpn4Ce5I1u2roBUoeWvnAxG gHFFIXDZCNLrWQ9fM37AogJ6qodISr6cwqMlr0USYFt7lMSxtyLPRZSJi4ha6eLe 4othVm7wOip1D7ALFv6u9PrPlZNPhlJObl/yz6/i2cApgMnoSqiPDGJb11trzIZj 4rncg== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 400cur8y2v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:55:49 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:55:48 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:55:48 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 214853F7079; Wed, 26 Jun 2024 03:55:44 -0700 (PDT) From: Aakash Sasidharan To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra CC: , , , , , Subject: [PATCH v3 03/12] common/cnxk: make inline dev PF func get as idev API Date: Wed, 26 Jun 2024 16:25:25 +0530 Message-ID: <20240626105534.1386528-4-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: o0-FVbSwr10MQbs4FfviFFdLu8MfN1R_ X-Proofpoint-GUID: o0-FVbSwr10MQbs4FfviFFdLu8MfN1R_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Inline PF FUNC would be required to set SSO_PF_FUNC in the instruction for cryptodev Rx inject. Move the API to idev to allow usage of the same. Signed-off-by: Anoob Joseph --- drivers/common/cnxk/roc_idev.c | 6 ++++++ drivers/common/cnxk/roc_idev.h | 2 ++ drivers/common/cnxk/roc_nix_inl.h | 1 - drivers/common/cnxk/roc_nix_inl_dev.c | 6 ------ drivers/common/cnxk/version.map | 2 +- drivers/net/cnxk/cn10k_ethdev_sec.c | 2 +- drivers/net/cnxk/cnxk_ethdev_telemetry.c | 3 +-- 7 files changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/common/cnxk/roc_idev.c b/drivers/common/cnxk/roc_idev.c index d0307c666c..0778d51d1e 100644 --- a/drivers/common/cnxk/roc_idev.c +++ b/drivers/common/cnxk/roc_idev.c @@ -374,3 +374,9 @@ roc_idev_nix_rx_chan_set(uint16_t port, uint16_t chan) if (idev != NULL && port < PLT_MAX_ETHPORTS) __atomic_store_n(&idev->inl_rx_inj_cfg.chan[port], chan, __ATOMIC_RELEASE); } + +uint16_t +roc_idev_nix_inl_dev_pffunc_get(void) +{ + return nix_inl_dev_pffunc_get(); +} diff --git a/drivers/common/cnxk/roc_idev.h b/drivers/common/cnxk/roc_idev.h index 00664eaed6..fc0f7db54e 100644 --- a/drivers/common/cnxk/roc_idev.h +++ b/drivers/common/cnxk/roc_idev.h @@ -27,4 +27,6 @@ uint8_t __roc_api roc_idev_nix_rx_inject_get(uint16_t port); void __roc_api roc_idev_nix_rx_inject_set(uint16_t port, uint8_t enable); uint16_t *__roc_api roc_idev_nix_rx_chan_base_get(void); void __roc_api roc_idev_nix_rx_chan_set(uint16_t port, uint16_t chan); + +uint16_t __roc_api roc_idev_nix_inl_dev_pffunc_get(void); #endif /* _ROC_IDEV_H_ */ diff --git a/drivers/common/cnxk/roc_nix_inl.h b/drivers/common/cnxk/roc_nix_inl.h index ab0965e512..1a4bf8808c 100644 --- a/drivers/common/cnxk/roc_nix_inl.h +++ b/drivers/common/cnxk/roc_nix_inl.h @@ -112,7 +112,6 @@ void __roc_api roc_nix_inl_dev_lock(void); void __roc_api roc_nix_inl_dev_unlock(void); int __roc_api roc_nix_inl_dev_xaq_realloc(uint64_t aura_handle); int __roc_api roc_nix_inl_dev_stats_get(struct roc_nix_stats *stats); -uint16_t __roc_api roc_nix_inl_dev_pffunc_get(void); int __roc_api roc_nix_inl_dev_cpt_setup(bool use_inl_dev_sso); int __roc_api roc_nix_inl_dev_cpt_release(void); bool __roc_api roc_nix_inl_dev_is_multi_channel(void); diff --git a/drivers/common/cnxk/roc_nix_inl_dev.c b/drivers/common/cnxk/roc_nix_inl_dev.c index 60e6a43033..e2bbe3a67b 100644 --- a/drivers/common/cnxk/roc_nix_inl_dev.c +++ b/drivers/common/cnxk/roc_nix_inl_dev.c @@ -34,12 +34,6 @@ nix_inl_dev_pffunc_get(void) return 0; } -uint16_t -roc_nix_inl_dev_pffunc_get(void) -{ - return nix_inl_dev_pffunc_get(); -} - static void nix_inl_selftest_work_cb(uint64_t *gw, void *args, uint32_t soft_exp_event) { diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index eac2ea9ff8..f98738d07e 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -112,6 +112,7 @@ INTERNAL { roc_idev_npa_nix_get; roc_idev_num_lmtlines_get; roc_idev_nix_inl_meta_aura_get; + roc_idev_nix_inl_dev_pffunc_get; roc_idev_nix_list_get; roc_idev_nix_rx_chan_base_get; roc_idev_nix_rx_chan_set; @@ -244,7 +245,6 @@ INTERNAL { roc_nix_inl_dev_is_probed; roc_nix_inl_dev_stats_get; roc_nix_inl_dev_lock; - roc_nix_inl_dev_pffunc_get; roc_nix_inl_dev_rq; roc_nix_inl_dev_rq_get; roc_nix_inl_dev_rq_put; diff --git a/drivers/net/cnxk/cn10k_ethdev_sec.c b/drivers/net/cnxk/cn10k_ethdev_sec.c index b8b0da5ea9..5e509e97d4 100644 --- a/drivers/net/cnxk/cn10k_ethdev_sec.c +++ b/drivers/net/cnxk/cn10k_ethdev_sec.c @@ -1360,7 +1360,7 @@ cn10k_eth_sec_rx_inject_config(void *device, uint16_t port_id, bool enable) inj_cfg->io_addr = inl_lf->io_addr; inj_cfg->lmt_base = nix->lmt_base; channel = roc_nix_get_base_chan(nix); - pf_func = roc_nix_inl_dev_pffunc_get(); + pf_func = roc_idev_nix_inl_dev_pffunc_get(); inj_cfg->cmd_w0 = pf_func << 48 | inj_match_id << 32 | channel << 4; return 0; diff --git a/drivers/net/cnxk/cnxk_ethdev_telemetry.c b/drivers/net/cnxk/cnxk_ethdev_telemetry.c index 3027ca4735..a1958185f2 100644 --- a/drivers/net/cnxk/cnxk_ethdev_telemetry.c +++ b/drivers/net/cnxk/cnxk_ethdev_telemetry.c @@ -65,8 +65,7 @@ ethdev_tel_handle_info(const char *cmd __rte_unused, info = ð_info.info; dev = cnxk_eth_pmd_priv(eth_dev); if (dev) { - info->inl_dev_pf_func = - roc_nix_inl_dev_pffunc_get(); + info->inl_dev_pf_func = roc_idev_nix_inl_dev_pffunc_get(); info->pf_func = roc_nix_get_pf_func(&dev->nix); info->max_mac_entries = dev->max_mac_entries; info->dmac_filter_ena = dev->dmac_filter_enable; From patchwork Wed Jun 26 10:55:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141764 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 66BC245500; Wed, 26 Jun 2024 13:00:45 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 06F7842F09; Wed, 26 Jun 2024 13:00:25 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 5859D40B95 for ; Wed, 26 Jun 2024 12:55:53 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q9jhow020687 for ; Wed, 26 Jun 2024 03:55:52 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=+ MIyloyCBtq4mqKerP3eW+AqPVvequ2bZ51oWWqp5pE=; b=Wo9agdMueCKym6zRv yeJmXmoj9cDMi8QLDnMKc32ZezvHBLGCnmfLJq4QlkY/ciaQlzBhp9vFqLTetnPv oiU0BsX1ZANxfJC0/CMQsLqBeJP3GQpMT8T2zJYavg11LQAGbHnzzmU8bZAdm4Q+ 7OEIzlfQ+Cgev8GXYClhv64s2lau2+jGcKN7PtvU7f19NZ41achZDupz0iK+5q1V gXOpstTYjjAdhYF3wqDM9LGEYup9yp776NVRnBRRebmR+UAjUtwJUpYq1XR04bme 882YjiMLp6/PUgHxx0MbrjJpfaMa01QxqMHDFHJA0enbZC/fI1/YSjF9ivvoQoA8 x2y6g== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 400ftngbw9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:55:52 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:55:51 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:55:51 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id EDBCD3F7079; Wed, 26 Jun 2024 03:55:48 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v3 04/12] crypto/cnxk: add flow control in Rx inject path Date: Wed, 26 Jun 2024 16:25:26 +0530 Message-ID: <20240626105534.1386528-5-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: aE6KdqBg_cfEims5-F8UsuBBdmosZlt4 X-Proofpoint-GUID: aE6KdqBg_cfEims5-F8UsuBBdmosZlt4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Add flow control in Rx inject path to avoid over submission to CPT. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 07bd13b16d..673220977c 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -1405,8 +1405,10 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, struct rte_cryptodev *cdev = dev; union cpt_res_s *hw_res = NULL; struct cpt_inst_s *inst; + union cpt_fc_write_s fc; struct cnxk_cpt_vf *vf; struct rte_mbuf *m; + uint64_t *fc_addr; uint64_t dptr; int i; @@ -1418,13 +1420,24 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, lmt_base = vf->rx_inj_lmtline.lmt_base; io_addr = vf->rx_inj_lmtline.io_addr; + fc_addr = vf->rx_inj_lmtline.fc_addr; ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); pf_func = vf->rx_inj_pf_func; + const uint32_t fc_thresh = vf->rx_inj_lmtline.fc_thresh; + again: + fc.u64[0] = + rte_atomic_load_explicit((RTE_ATOMIC(uint64_t) *)fc_addr, rte_memory_order_relaxed); inst = (struct cpt_inst_s *)lmt_base; - for (i = 0; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_pkts); i++) { + + i = 0; + + if (unlikely(fc.s.qsize > fc_thresh)) + goto exit; + + for (; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_pkts); i++) { m = pkts[i]; sec_sess = (struct cn10k_sec_session *)sess[i]; @@ -1492,6 +1505,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, goto again; } +exit: return count + i; } From patchwork Wed Jun 26 10:55:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141765 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7422545500; Wed, 26 Jun 2024 13:00:52 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0EBBE42F13; Wed, 26 Jun 2024 13:00:26 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 17A0340B95 for ; Wed, 26 Jun 2024 12:55:56 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q5RNiv005061 for ; Wed, 26 Jun 2024 03:55:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=A L+CaYYa967+EA6oWL+K52CZb5C+DJlDq3Z7P6jlWIU=; b=Lzr7E3D0HyZFsx7kN uwLwDiCwSUWXfvEikRk23XcA92Pezbm0U4/farmD89yLcP6l0bfhIRzNhtIBSfPO GAqYcPfAW7wsehDRKTCVz3ZgXCPF3hQTdu9EF5BF5+hu+u2BCqAW3M1XB+gvyqJE v7PiRDAwEz8kftuhFh+tP4uNBBtAtx1vO/0l4DCQHUET8c5JReQzj/UfkkS7o3a8 StaL1ZGiOSExhpznb4xq2GggkHDj5ZRXGhJKRhr6a+/sz8m7G3n86TQal7rW8X8b 6D5/isgCCwCcidSKbxw41OleL8cqN0M4ZgqAyL2X+j5YG5zF8LMwUp8/gl6krdps BAq6g== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 400cur8y34-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:55:55 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:55:54 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:55:54 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 0E2393F7079; Wed, 26 Jun 2024 03:55:51 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v3 05/12] crypto/cnxk: use SSO PF func of inline device in inst Date: Wed, 26 Jun 2024 16:25:27 +0530 Message-ID: <20240626105534.1386528-6-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: Oww_vzHS0v7VzijOmn2SOWsWkH0twvVa X-Proofpoint-GUID: Oww_vzHS0v7VzijOmn2SOWsWkH0twvVa X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph RVU PF FUNC of the CPT LF need not be set as the hardware would determine that. Instead SSO PF FUNC need to be set as inline device so that critical errors would reach inline device. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 2 +- drivers/crypto/cnxk/cnxk_cryptodev.h | 2 +- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 673220977c..2bdf4d29c2 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -1423,7 +1423,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, fc_addr = vf->rx_inj_lmtline.fc_addr; ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); - pf_func = vf->rx_inj_pf_func; + pf_func = vf->rx_inj_sso_pf_func; const uint32_t fc_thresh = vf->rx_inj_lmtline.fc_thresh; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h index fffc4a47b4..4000e84a7e 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev.h @@ -22,7 +22,7 @@ */ struct cnxk_cpt_vf { struct roc_cpt_lmtline rx_inj_lmtline; - uint16_t rx_inj_pf_func; + uint16_t rx_inj_sso_pf_func; uint16_t *rx_chan_base; struct roc_cpt cpt; struct rte_cryptodev_capabilities crypto_caps[CNXK_CPT_MAX_CAPS]; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index d7f5780637..51369309c5 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -483,7 +483,7 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, goto exit; } - vf->rx_inj_pf_func = qp->lf.pf_func; + vf->rx_inj_sso_pf_func = roc_idev_nix_inl_dev_pffunc_get(); /* Block the queue for other submissions */ qp->pend_q.pq_mask = 0; From patchwork Wed Jun 26 10:55:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141766 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CAB7A45500; Wed, 26 Jun 2024 13:00:58 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E5E3142F21; Wed, 26 Jun 2024 13:00:26 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id DC29840B95 for ; Wed, 26 Jun 2024 12:55:59 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q9jhp0020687 for ; Wed, 26 Jun 2024 03:55:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=x Xcjv4pTY95RUMuPLNIsfebeT+7lmh9ShPiGuuma3q0=; b=bX3/JfngsI7VWhrSV yqC1SZ8/E+A78yP30fA12tspWJhxWZA9w9Urg3sNFfcNtmYfagFnWRqJepvfhNjp HuNVuFPxAJRp9xfgNhCFF2eEzUskH7PopC2pHcruRcMKwrkuxWOyDiHUWwv2MnwE FXVz/Vgu0dldfri7wY25ovSfgEMcB8GO8MWJaFm6hkSCLwBr96ZMNw/hg+BQTCTC aqBIW2aEQNX3fnDD6ZFacnRy/7LaMxjhbIgPppe3pJ1edHowlod3dCe7YNzHffWt +WtPaW7dVZRCW6lKTRX3FSsC2v0w1fciNYH6ab8IkXbeEJiK5FoMQouR8qCgsYiw QJzGQ== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 400ftngbwe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:55:58 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:55:57 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:55:57 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 20BC23F7079; Wed, 26 Jun 2024 03:55:54 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v3 06/12] crypto/cnxk: use NEON for Rx inject inst preparation Date: Wed, 26 Jun 2024 16:25:28 +0530 Message-ID: <20240626105534.1386528-7-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: AhL4xEjKh8_--bHmiS84pN9wLeVrQshT X-Proofpoint-GUID: AhL4xEjKh8_--bHmiS84pN9wLeVrQshT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Use NEON instructions for Rx inject instruction preparation. Signed-off-by: Anoob Joseph Signed-off-by: Aakash Sasidharan --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 62 +++++++++++++++++------ 1 file changed, 46 insertions(+), 16 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 2bdf4d29c2..c489f0884e 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -7,6 +7,7 @@ #include #include #include +#include #include @@ -1395,15 +1396,17 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return i; } +#if defined(RTE_ARCH_ARM64) uint16_t __rte_hot cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, struct rte_security_session **sess, uint16_t nb_pkts) { - uint16_t l2_len, pf_func, lmt_id, count = 0; - uint64_t lmt_base, lmt_arg, io_addr; + uint64_t lmt_base, lmt_arg, io_addr, u64_0, u64_1, l2_len, pf_func; + uint64x2_t inst_01, inst_23, inst_45, inst_67; struct cn10k_sec_session *sec_sess; struct rte_cryptodev *cdev = dev; union cpt_res_s *hw_res = NULL; + uint16_t lmt_id, count = 0; struct cpt_inst_s *inst; union cpt_fc_write_s fc; struct cnxk_cpt_vf *vf; @@ -1461,26 +1464,41 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, hw_res = RTE_PTR_ALIGN_CEIL(hw_res, 16); /* Prepare CPT instruction */ - inst->w0.u64 = 0; - inst->w2.u64 = 0; - inst->w2.s.rvu_pf_func = pf_func; - inst->w3.u64 = (((uint64_t)m + sizeof(struct rte_mbuf)) >> 3) << 3 | 1; - inst->w4.u64 = sec_sess->inst.w4 | (rte_pktmbuf_pkt_len(m)); + /* Word 0 and 1 */ + inst_01 = vdupq_n_u64(0); + u64_0 = pf_func << 48 | *(vf->rx_chan_base + m->port) << 4 | (l2_len - 2) << 24 | + l2_len << 16; + inst_01 = vsetq_lane_u64(u64_0, inst_01, 0); + inst_01 = vsetq_lane_u64((uint64_t)hw_res, inst_01, 1); + vst1q_u64(&inst->w0.u64, inst_01); + + /* Word 2 and 3 */ + inst_23 = vdupq_n_u64(0); + u64_1 = (((uint64_t)m + sizeof(struct rte_mbuf)) >> 3) << 3 | 1; + inst_23 = vsetq_lane_u64(u64_1, inst_23, 1); + vst1q_u64(&inst->w2.u64, inst_23); + + /* Word 4 and 5 */ + inst_45 = vdupq_n_u64(0); + u64_0 = sec_sess->inst.w4 | (rte_pktmbuf_pkt_len(m)); + inst_45 = vsetq_lane_u64(u64_0, inst_45, 0); dptr = (uint64_t)rte_pktmbuf_iova(m); - inst->dptr = dptr; - inst->rptr = dptr; - - inst->w0.hw_s.chan = *(vf->rx_chan_base + m->port); - inst->w0.hw_s.l2_len = l2_len; - inst->w0.hw_s.et_offset = l2_len - 2; + u64_1 = dptr; + inst_45 = vsetq_lane_u64(u64_1, inst_45, 1); + vst1q_u64(&inst->w4.u64, inst_45); + + /* Word 6 and 7 */ + inst_67 = vdupq_n_u64(0); + u64_0 = dptr; + u64_1 = sec_sess->inst.w7; + inst_67 = vsetq_lane_u64(u64_0, inst_67, 0); + inst_67 = vsetq_lane_u64(u64_1, inst_67, 1); + vst1q_u64(&inst->w6.u64, inst_67); - inst->res_addr = (uint64_t)hw_res; rte_atomic_store_explicit((unsigned long __rte_atomic *)&hw_res->u64[0], res.u64[0], rte_memory_order_relaxed); - inst->w7.u64 = sec_sess->inst.w7; - inst += 2; } @@ -1508,6 +1526,18 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, exit: return count + i; } +#else +uint16_t __rte_hot +cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, + struct rte_security_session **sess, uint16_t nb_pkts) +{ + RTE_SET_USED(dev); + RTE_SET_USED(pkts); + RTE_SET_USED(sess); + RTE_SET_USED(nb_pkts); + return 0; +} +#endif void cn10k_cpt_set_enqdeq_fns(struct rte_cryptodev *dev, struct cnxk_cpt_vf *vf) From patchwork Wed Jun 26 10:55:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141767 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18DF045500; Wed, 26 Jun 2024 13:01:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ECBF342F37; Wed, 26 Jun 2024 13:00:27 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 7B27940B95 for ; Wed, 26 Jun 2024 12:56:03 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q5Saib006220 for ; Wed, 26 Jun 2024 03:56:03 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=5 kjDVYSGD7bm7iApQeFuyayx2CnU2mjkVbMXtJPmEZI=; b=KTZAkNJKSYiPQ4N9X WjNtSMVpwNZeEVxmfSfXaruB3/rdyLlqj0xCZVciWB2IzyjooUFQtZuz7ZaaBXVQ 8yWKpll6wY243sjC5gOs+cvTVQkaJzHBobAOop4bz+ejmDKRT9pyO5fTeni1BePN DsPZJ6LxCP/iDRKwJRXshbt6ao+HXi6217pSgPqCvBp7tkcL9eFfI2VA0ACpt3jP La7reuPIRL29TX5ni9nX2GPaQdRPpmykz6J3Rs3JgeNwiw4raqTwQxzkdsFMIAU1 ugmfkiM+AxfD74ui/SEiBGTgZ6Pve7EiD5wLUuXsAOHgqQm+KSmzKuyimdYMLW5p 28FtA== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 400cur8y3a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:56:02 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:56:00 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:56:00 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 38A993F7079; Wed, 26 Jun 2024 03:55:57 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v3 07/12] crypto/cnxk: remove init of CPT result field in packet Date: Wed, 26 Jun 2024 16:25:29 +0530 Message-ID: <20240626105534.1386528-8-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: POMPpNHXO0gnXdGgXMtJ24JVOWHbFszV X-Proofpoint-GUID: POMPpNHXO0gnXdGgXMtJ24JVOWHbFszV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph The packet would be posted to CPT only when there is a valid result. Skip setting of the same. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index c489f0884e..b808ea2946 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -1415,10 +1415,6 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, uint64_t dptr; int i; - const union cpt_res_s res = { - .cn10k.compcode = CPT_COMP_NOT_DONE, - }; - vf = cdev->data->dev_private; lmt_base = vf->rx_inj_lmtline.lmt_base; @@ -1496,9 +1492,6 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, inst_67 = vsetq_lane_u64(u64_1, inst_67, 1); vst1q_u64(&inst->w6.u64, inst_67); - rte_atomic_store_explicit((unsigned long __rte_atomic *)&hw_res->u64[0], res.u64[0], - rte_memory_order_relaxed); - inst += 2; } From patchwork Wed Jun 26 10:55:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141768 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6473645500; Wed, 26 Jun 2024 13:01:11 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DEF5E42F51; Wed, 26 Jun 2024 13:00:28 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id D60D840B95 for ; Wed, 26 Jun 2024 12:56:06 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q5RZRS005244 for ; Wed, 26 Jun 2024 03:56:06 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=C YicOSNNycaJYK+ipCrxsJSOe1K8VCFtYYlFCsZWFbI=; b=DyOA73SggE6+HPR86 AbwZKxjSfEL8QKBsO0+sS50D4n+b4L/aCSr9E21eftvgVosUvYRBGBEWHbu8tZxA TN4YN6ZtjBXDhpTs49j7r1go5Oo4SUMzIno54fFjBxZ2BJ/r6TmXe+QSMXIW9j5k QpxnHx/DAOiF3C8QAnCUQTRExquy/fU7VyATDMYHXoPLX4Qr/pmmbAyLSZQ05u4+ mk5Ru14pG8cxlK/IG4iSyElbjLDKSnYJMNnmdMQFypEIBvKPU7GqLjmTiiv1cZ6M njehF1EpTcR6miBU5oOY4UkPCwdBNqeLEiaIaVzQotfzNES/hSbp52bwGehPVQ2E idsHw== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 400cur8y3n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:56:06 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:56:05 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:56:05 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 56C743F707B; Wed, 26 Jun 2024 03:56:01 -0700 (PDT) From: Aakash Sasidharan To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v3 08/12] crypto/cnxk: add dual submission in Rx inject Date: Wed, 26 Jun 2024 16:25:30 +0530 Message-ID: <20240626105534.1386528-9-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: nQPq6W19vSnEBQ5rQLtk2keBlKhfjrxE X-Proofpoint-GUID: nQPq6W19vSnEBQ5rQLtk2keBlKhfjrxE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Add dual submission to CPT in Rx inject path. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri Signed-off-by: Aakash Sasidharan --- drivers/common/cnxk/roc_cpt.h | 43 +++++++++----- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 72 +++++++++++++++++------ drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 9 +++ 3 files changed, 92 insertions(+), 32 deletions(-) diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h index 3721fa08c0..8ef9062ae0 100644 --- a/drivers/common/cnxk/roc_cpt.h +++ b/drivers/common/cnxk/roc_cpt.h @@ -30,23 +30,36 @@ /* Vector of sizes in the burst of 16 CPT inst except first in 63:19 of * APT_LMT_ARG_S */ -#define ROC_CN10K_CPT_LMT_ARG \ - (ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 0) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 1) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 2) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 3) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 4) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 5) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 6) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 7) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 8) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 9) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 10) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 11) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 12) | \ - ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 13) | \ +#define ROC_CN10K_CPT_LMT_ARG \ + (ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 0) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 1) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 2) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 3) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 4) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 5) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 6) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 7) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 8) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 9) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 10) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 11) | \ + ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 12) | ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 13) | \ ROC_CN10K_CPT_INST_DW_M1 << (19 + 3 * 14)) +/* Vector of sizes in the burst of 2 * 16 CPT inst except first in 63:19 of + * APT_LMT_ARG_S + */ +#define ROC_CN10K_DUAL_CPT_LMT_ARG \ + (ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 0) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 1) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 2) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 3) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 4) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 5) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 6) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 7) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 8) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 9) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 10) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 11) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 12) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 13) | \ + ROC_CN10K_TWO_CPT_INST_DW_M1 << (19 + 3 * 14)) + /* CPT helper macros */ #define ROC_CPT_AH_HDR_LEN 12 #define ROC_CPT_AES_GCM_IV_LEN 8 diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index b808ea2946..e42a3d2ea6 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -55,6 +55,56 @@ struct vec_request { uint64_t w2; }; +#if defined(RTE_ARCH_ARM64) +static __rte_always_inline void __rte_hot +cn10k_cpt_lmtst_dual_submit(uint64_t *io_addr, const uint16_t lmt_id, int *i) +{ + uint64_t lmt_arg; + + /* Check if the total number of instructions is odd or even. */ + const int flag_odd = *i & 0x1; + + /* Reduce i by 1 when odd number of instructions.*/ + *i -= flag_odd; + + if (*i > 2 * CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, *io_addr); + lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - CN10K_PKTS_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + if (flag_odd) { + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_CPT_INST_DW_M1 << 4); + lmt_arg = (uint64_t)(lmt_id + *i / 2); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); + *i += 1; + } + } else { + if (*i != 0) { + lmt_arg = + ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, *io_addr); + } + + if (flag_odd) { + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_CPT_INST_DW_M1 << 4); + lmt_arg = (uint64_t)(lmt_id + *i / 2); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); + *i += 1; + } + } + + rte_io_wmb(); +} +#endif + static inline struct cnxk_se_sess * cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op) { @@ -1401,7 +1451,7 @@ uint16_t __rte_hot cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, struct rte_security_session **sess, uint16_t nb_pkts) { - uint64_t lmt_base, lmt_arg, io_addr, u64_0, u64_1, l2_len, pf_func; + uint64_t lmt_base, io_addr, u64_0, u64_1, l2_len, pf_func; uint64x2_t inst_01, inst_23, inst_45, inst_67; struct cn10k_sec_session *sec_sess; struct rte_cryptodev *cdev = dev; @@ -1436,7 +1486,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, if (unlikely(fc.s.qsize > fc_thresh)) goto exit; - for (; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_pkts); i++) { + for (; i < RTE_MIN(2 * CN10K_PKTS_PER_LOOP, nb_pkts); i++) { m = pkts[i]; sec_sess = (struct cn10k_sec_session *)sess[i]; @@ -1492,24 +1542,12 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, inst_67 = vsetq_lane_u64(u64_1, inst_67, 1); vst1q_u64(&inst->w6.u64, inst_67); - inst += 2; + inst++; } - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_pkts - i > 0 && i == CN10K_PKTS_PER_LOOP) { + if (nb_pkts - i > 0 && i == 2 * CN10K_PKTS_PER_LOOP) { nb_pkts -= i; pkts += i; count += i; diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 51369309c5..6acaa4413b 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -431,6 +431,7 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, struct rte_pci_device *pci_dev; struct cnxk_cpt_qp *qp; uint32_t nb_desc; + uint64_t io_addr; int ret; if (dev->data->queue_pairs[qp_id] != NULL) @@ -485,6 +486,14 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, vf->rx_inj_sso_pf_func = roc_idev_nix_inl_dev_pffunc_get(); + /* Update IO addr to enable dual submission */ + io_addr = vf->rx_inj_lmtline.io_addr; + io_addr = (io_addr & ~(uint64_t)(0x7 << 4)) | ROC_CN10K_TWO_CPT_INST_DW_M1 << 4; + vf->rx_inj_lmtline.io_addr = io_addr; + + /* Update FC threshold to reflect dual submission */ + vf->rx_inj_lmtline.fc_thresh -= 32; + /* Block the queue for other submissions */ qp->pend_q.pq_mask = 0; } From patchwork Wed Jun 26 10:55:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141769 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 21A4645500; Wed, 26 Jun 2024 13:01:18 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CFA4042F5D; Wed, 26 Jun 2024 13:00:29 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 863F140B95 for ; Wed, 26 Jun 2024 12:56:10 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q9jkfu020753 for ; Wed, 26 Jun 2024 03:56:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=h 7udc8Qr1Npbsw03x1Vt5a8t8gc+dnjCyZac6UWrbnk=; b=DqtYDTJDKnYjcqyLc 9pkEGP+tu6hIazHgY+hYX+uAZ+H5pwNkJZsrtdbGvl9rKfkhiFZE2nEpJNbvlNkf LBsOr6EkgbDK5rASkMmY7LjtQNzl9/5v2ivVwXC4QBvEMWAXCepyp5+pbLhOHmVG Uyj56MgCcwv1AQYXxC+4ySAzS2Dv1Q0xQifITPmcpnUwvSM8oOhWugZB30F1bFi6 ZJDO5c8i6LxoJbOLW8FFBWuO+F8u/n/YIpq1EQlkWeTgGD2tSPmnRtfFwZO/+Y+D cRBoTvfk8B7xlEz6/Yq+UuB4wSdJNpvh9efGeqtZjlbIzLi5fJEfr2zIjJAwiJZM ALWzw== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 400ftngbwr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:56:09 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:56:08 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:56:08 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id A9CFA3F7079; Wed, 26 Jun 2024 03:56:05 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v3 09/12] crypto/cnxk: update sess pointer for next iteration Date: Wed, 26 Jun 2024 16:25:31 +0530 Message-ID: <20240626105534.1386528-10-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: ixuts5BHzdTfY5kAnvn5EmZ0-NhcvAyl X-Proofpoint-GUID: ixuts5BHzdTfY5kAnvn5EmZ0-NhcvAyl X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Update sess pointer while working on next set of packets. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index e42a3d2ea6..ed964d4d01 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -1467,6 +1467,8 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, vf = cdev->data->dev_private; + const int nb_pkts_per_loop = 2 * CN10K_PKTS_PER_LOOP; + lmt_base = vf->rx_inj_lmtline.lmt_base; io_addr = vf->rx_inj_lmtline.io_addr; fc_addr = vf->rx_inj_lmtline.fc_addr; @@ -1486,7 +1488,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, if (unlikely(fc.s.qsize > fc_thresh)) goto exit; - for (; i < RTE_MIN(2 * CN10K_PKTS_PER_LOOP, nb_pkts); i++) { + for (; i < RTE_MIN(nb_pkts_per_loop, nb_pkts); i++) { m = pkts[i]; sec_sess = (struct cn10k_sec_session *)sess[i]; @@ -1547,10 +1549,11 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_pkts - i > 0 && i == 2 * CN10K_PKTS_PER_LOOP) { - nb_pkts -= i; - pkts += i; - count += i; + if (nb_pkts - i > 0 && i == nb_pkts_per_loop) { + nb_pkts -= nb_pkts_per_loop; + pkts += nb_pkts_per_loop; + count += nb_pkts_per_loop; + sess += nb_pkts_per_loop; goto again; } From patchwork Wed Jun 26 10:55:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141770 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 627D445500; Wed, 26 Jun 2024 13:01:24 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BE7AB42F76; Wed, 26 Jun 2024 13:00:30 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 4EB6440B95; Wed, 26 Jun 2024 12:56:13 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q8nVrN022914; Wed, 26 Jun 2024 03:56:12 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=X mKXF6Py/eAjnXD3KD0RmIuSdcKYBE6y7+b5VU4kP7s=; b=U+XkNgYxwblPyWzTE XzFpswurek5SNJBjdl2mV24Gi7oI4sDWiZmx9HA2ks92upkOnaAV1HvgMmhpNmwy mwvqENPGXiUako5/2hknLkFhy7FSP994iKLpW6ZC1piCUoNl8M5X4B0FPIqYxtPJ uXU/2hTsXphvAof6l3pqo89NCmEqOnMdWHT0m1iYbapA8HUjtvE9DEN++gdu47iQ vLOPb4JQY732a8deZXwRIEfZWp/7qlPtYphw15NYWo+wtXiUXM6UtODe9T+Z7fuX InidYs+eTLlK4fTmtgarSSjBrMtpbKVJt+AYkyLcM/sEeBmSi12JDWyCfze5niqx Axn3Q== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 400ftngbww-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 26 Jun 2024 03:56:12 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:56:11 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:56:11 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id BCE283F7079; Wed, 26 Jun 2024 03:56:08 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Akhil Goyal CC: , , , , Subject: [PATCH v3 10/12] crypto/cnxk: fix aes-gcm zero len input cases Date: Wed, 26 Jun 2024 16:25:32 +0530 Message-ID: <20240626105534.1386528-11-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: rkb8gko_fMRuRAKohmtmtgeRkDjxxgSL X-Proofpoint-GUID: rkb8gko_fMRuRAKohmtmtgeRkDjxxgSL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For aes-gcm (AEAD) zero length input, sg code path is taken unlike the digest only cases as AAD is treated as a separate input component. Fix the zero len case in SG path by avoiding the gather component only when it is a non AEAD algorithm. Also add sg version check as the fix only applies to specific model. Fixes: 4d8166d64988 ("crypto/cnxk: enable digest for zero length input") Cc: stable@dpdk.org Signed-off-by: Aakash Sasidharan Acked-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 6374718a82..63dbef4411 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -2468,13 +2468,14 @@ fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess) } static __rte_always_inline uint32_t -prepare_iov_from_pkt(struct rte_mbuf *pkt, struct roc_se_iov_ptr *iovec, uint32_t start_offset) +prepare_iov_from_pkt(struct rte_mbuf *pkt, struct roc_se_iov_ptr *iovec, uint32_t start_offset, + const bool is_aead, const bool is_sg_ver2) { uint16_t index = 0; void *seg_data = NULL; int32_t seg_size = 0; - if (!pkt || pkt->data_len == 0) { + if (!pkt || (is_sg_ver2 && (pkt->data_len == 0) && !is_aead)) { iovec->buf_cnt = 0; return 0; } @@ -2619,13 +2620,13 @@ fill_sm_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.dst_iov = (void *)dst; /* Store SG I/O in the api for reuse */ - if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0)) { + if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0, false, is_sg_ver2)) { plt_dp_err("Prepare src iov failed"); ret = -EINVAL; goto err_exit; } - if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0)) { + if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0, false, is_sg_ver2)) { plt_dp_err("Prepare dst iov failed for m_dst %p", m_dst); ret = -EINVAL; goto err_exit; @@ -2816,14 +2817,15 @@ fill_fc_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.dst_iov = (void *)dst; /* Store SG I/O in the api for reuse */ - if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0)) { + if (prepare_iov_from_pkt(m_src, fc_params.src_iov, 0, is_aead, is_sg_ver2)) { plt_dp_err("Prepare src iov failed"); ret = -EINVAL; goto err_exit; } if (unlikely(m_dst != NULL)) { - if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0)) { + if (prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0, is_aead, + is_sg_ver2)) { plt_dp_err("Prepare dst iov failed for " "m_dst %p", m_dst); @@ -2957,13 +2959,15 @@ fill_pdcp_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.dst_iov = (void *)dst; /* Store SG I/O in the api for reuse */ - if (unlikely(prepare_iov_from_pkt(m_src, fc_params.src_iov, 0))) { + if (unlikely( + prepare_iov_from_pkt(m_src, fc_params.src_iov, 0, false, is_sg_ver2))) { plt_dp_err("Prepare src iov failed"); ret = -EINVAL; goto err_exit; } - if (unlikely(prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0))) { + if (unlikely( + prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0, false, is_sg_ver2))) { plt_dp_err("Prepare dst iov failed for m_dst %p", m_dst); ret = -EINVAL; goto err_exit; @@ -3080,14 +3084,16 @@ fill_pdcp_chain_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, fc_params.dst_iov = (void *)dst; /* Store SG I/O in the api for reuse */ - if (unlikely(prepare_iov_from_pkt(m_src, fc_params.src_iov, 0))) { + if (unlikely( + prepare_iov_from_pkt(m_src, fc_params.src_iov, 0, false, is_sg_ver2))) { plt_dp_err("Could not prepare src iov"); ret = -EINVAL; goto err_exit; } if (unlikely(m_dst != NULL)) { - if (unlikely(prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0))) { + if (unlikely(prepare_iov_from_pkt(m_dst, fc_params.dst_iov, 0, false, + is_sg_ver2))) { plt_dp_err("Could not prepare m_dst iov %p", m_dst); ret = -EINVAL; goto err_exit; @@ -3306,7 +3312,7 @@ fill_digest_params(struct rte_crypto_op *cop, struct cnxk_se_sess *sess, params.src_iov = (void *)src; /*Store SG I/O in the api for reuse */ - if (prepare_iov_from_pkt(m_src, params.src_iov, auth_range_off)) { + if (prepare_iov_from_pkt(m_src, params.src_iov, auth_range_off, false, is_sg_ver2)) { plt_dp_err("Prepare src iov failed"); ret = -EINVAL; goto free_mdata_and_exit; From patchwork Wed Jun 26 10:55:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141771 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 132DC45500; Wed, 26 Jun 2024 13:01:31 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BCC0D42FAF; Wed, 26 Jun 2024 13:00:31 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A60D340B95 for ; Wed, 26 Jun 2024 12:56:16 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q5SCga005544 for ; Wed, 26 Jun 2024 03:56:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=j cif+WxeU4jjzNUCkGATxOk+hLTsckyNLKZBmXGs9xw=; b=M08chzTTg6cXG/QJS annCNFPbkr9z63WQZ2o66DHYQUSqfsxWDLh0o7AhSzKx9Uv8jYxtPmvUaa5FB+3+ dBGx5uxtVK5j60RlCPwD7560lEUJp3sMS4SA70hkXJ5Chs5V5+4CR9ozs+cn93Nl crxgbu5NeUTHb9LZP+UaYl7FVjDtfr/LUn1jHQd5dBAc5nn1wBrQj43M5Q1o7PKh dYaqmGwN6piwg9LhVW+8+PvgbMuPbzU16aX4UVMBzUwKkzS7FvEq3UCldmtSO9rz GaTkVYMqLlJRK3Zlj0vkzR0/Gq0H4HoqiwcM5d5IAMeUr4xX3JaJbE7sM+HclLIV nFd3Q== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 400cur8y3x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:56:15 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:56:14 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:56:14 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 174B13F7079; Wed, 26 Jun 2024 03:56:11 -0700 (PDT) From: Aakash Sasidharan To: Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj CC: , , , , Subject: [PATCH v3 11/12] crypto/cnxk: make pack IV variable as const Date: Wed, 26 Jun 2024 16:25:33 +0530 Message-ID: <20240626105534.1386528-12-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: UFRVgfWZ2KGUg8A7ciz8MxzcM4u4oBnz X-Proofpoint-GUID: UFRVgfWZ2KGUg8A7ciz8MxzcM4u4oBnz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Make 'pack_iv' variable as const to avoid multiple checks. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cnxk_se.h | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index 63dbef4411..dbd36a8a54 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -105,7 +105,7 @@ cpt_pack_iv(uint8_t *iv_src, uint8_t *iv_dst) } static inline void -pdcp_iv_copy(uint8_t *iv_d, const uint8_t *iv_s, const uint8_t pdcp_alg_type, uint8_t pack_iv) +pdcp_iv_copy(uint8_t *iv_d, const uint8_t *iv_s, const uint8_t pdcp_alg_type, const bool pack_iv) { const uint32_t *iv_s_temp; uint32_t iv_temp[4]; @@ -261,7 +261,7 @@ cpt_mac_len_verify(struct rte_crypto_auth_xform *auth) static __rte_always_inline int sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t offset_ctrl, - const uint8_t *iv_s, int iv_len, uint8_t pack_iv, uint8_t pdcp_alg_type, + const uint8_t *iv_s, int iv_len, const bool pack_iv, uint8_t pdcp_alg_type, int32_t inputlen, int32_t outputlen, uint32_t passthrough_len, uint32_t req_flags, int pdcp_flag, int decrypt) { @@ -457,7 +457,7 @@ sg_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t static __rte_always_inline int sg2_inst_prep(struct roc_se_fc_params *params, struct cpt_inst_s *inst, uint64_t offset_ctrl, - const uint8_t *iv_s, int iv_len, uint8_t pack_iv, uint8_t pdcp_alg_type, + const uint8_t *iv_s, int iv_len, const bool pack_iv, uint8_t pdcp_alg_type, int32_t inputlen, int32_t outputlen, uint32_t passthrough_len, uint32_t req_flags, int pdcp_flag, int decrypt) { @@ -882,7 +882,7 @@ static inline int pdcp_chain_sg1_prep(struct roc_se_fc_params *params, struct roc_se_ctx *cpt_ctx, struct cpt_inst_s *inst, union cpt_inst_w4 w4, int32_t inputlen, uint8_t hdr_len, uint64_t offset_ctrl, uint32_t req_flags, - const uint8_t *cipher_iv, const uint8_t *auth_iv, const int pack_iv, + const uint8_t *cipher_iv, const uint8_t *auth_iv, const bool pack_iv, const uint8_t pdcp_ci_alg, const uint8_t pdcp_auth_alg) { struct roc_sglist_comp *scatter_comp, *gather_comp; @@ -991,7 +991,7 @@ static inline int pdcp_chain_sg2_prep(struct roc_se_fc_params *params, struct roc_se_ctx *cpt_ctx, struct cpt_inst_s *inst, union cpt_inst_w4 w4, int32_t inputlen, uint8_t hdr_len, uint64_t offset_ctrl, uint32_t req_flags, - const uint8_t *cipher_iv, const uint8_t *auth_iv, const int pack_iv, + const uint8_t *cipher_iv, const uint8_t *auth_iv, const bool pack_iv, const uint8_t pdcp_ci_alg, const uint8_t pdcp_auth_alg) { struct roc_sg2list_comp *gather_comp, *scatter_comp; @@ -1528,7 +1528,6 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, struct roc_se_ctx *se_ctx; uint64_t *offset_vaddr; uint64_t offset_ctrl; - uint8_t pack_iv = 0; int32_t inputlen; void *dm_vaddr; uint8_t *iv_d; @@ -1606,10 +1605,10 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, cpt_inst_w4.s.dlen = inputlen + ROC_SE_OFF_CTRL_LEN; iv_d = ((uint8_t *)offset_vaddr + ROC_SE_OFF_CTRL_LEN); - pdcp_iv_copy(iv_d, cipher_iv, pdcp_ci_alg, pack_iv); + pdcp_iv_copy(iv_d, cipher_iv, pdcp_ci_alg, false); iv_d = ((uint8_t *)offset_vaddr + ROC_SE_OFF_CTRL_LEN + pdcp_iv_off); - pdcp_iv_copy(iv_d, auth_iv, pdcp_auth_alg, pack_iv); + pdcp_iv_copy(iv_d, auth_iv, pdcp_auth_alg, false); inst->w4.u64 = cpt_inst_w4.u64; return 0; @@ -1618,11 +1617,11 @@ cpt_pdcp_chain_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, if (is_sg_ver2) return pdcp_chain_sg2_prep(params, se_ctx, inst, cpt_inst_w4, inputlen, hdr_len, offset_ctrl, req_flags, cipher_iv, - auth_iv, pack_iv, pdcp_ci_alg, pdcp_auth_alg); + auth_iv, false, pdcp_ci_alg, pdcp_auth_alg); else return pdcp_chain_sg1_prep(params, se_ctx, inst, cpt_inst_w4, inputlen, hdr_len, offset_ctrl, req_flags, cipher_iv, - auth_iv, pack_iv, pdcp_ci_alg, pdcp_auth_alg); + auth_iv, false, pdcp_ci_alg, pdcp_auth_alg); } } @@ -1647,9 +1646,9 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, uint64_t *offset_vaddr; uint8_t pdcp_alg_type; uint32_t mac_len = 0; - const uint8_t *iv_s; - uint8_t pack_iv = 0; uint64_t offset_ctrl; + bool pack_iv = false; + const uint8_t *iv_s; int ret; mac_len = se_ctx->mac_len; @@ -1671,7 +1670,7 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, if (pdcp_alg_type != ROC_SE_PDCP_ALG_TYPE_AES_CMAC) { if (params->auth_iv_len == 25) - pack_iv = 1; + pack_iv = true; auth_offset = auth_offset / 8; auth_data_len = RTE_ALIGN(auth_data_len, 8) / 8; @@ -1694,7 +1693,7 @@ cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens, pdcp_alg_type = se_ctx->pdcp_ci_alg; if (params->cipher_iv_len == 25) - pack_iv = 1; + pack_iv = true; /* * Microcode expects offsets in bytes From patchwork Wed Jun 26 10:55:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aakash Sasidharan X-Patchwork-Id: 141772 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0E14345500; Wed, 26 Jun 2024 13:01:37 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AB41442FB7; Wed, 26 Jun 2024 13:00:32 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id E8D2940B95 for ; Wed, 26 Jun 2024 12:56:21 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45Q9jmqK020777 for ; Wed, 26 Jun 2024 03:56:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=T mN31JDjwyFd5PzJryPYOhsUS/wmbyyU/ckNHB5VRFg=; b=GN5AZZDRMA6W9Umn5 rv1IUluGhHt0tluYFsfguar5RXgk7TACG8Pat/SowhYyW/8jZXGjdEM779GWIwAQ X55WXXzxXryV/sb7Dpzs5BK3UMQdYdS6vh8A4wxR2rD7WDiohHFdy59Qc8HpfKIA mylWgYodnPa0rwGonJLomucFsGr3kP3ZFYvRXmNo/37TuV6xJmYBITzXdjlMaMNj PJlDbLvRHQB6fXMLBJHaGqny7/Puo5mDA9ypYfhsv4z19c43YGrDrUNbTJG0astT fymiHvw9x0CZlvfh3M/cNHKuzfUdX3IzkTkGeGDhKaHudAyOPWPSuzuYSKP2qBzH 7gVyg== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 400ftngbx9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 26 Jun 2024 03:56:20 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 26 Jun 2024 03:56:19 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 26 Jun 2024 03:56:19 -0700 Received: from localhost.localdomain (unknown [10.28.36.177]) by maili.marvell.com (Postfix) with ESMTP id 30DF93F707B; Wed, 26 Jun 2024 03:56:14 -0700 (PDT) From: Aakash Sasidharan To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Pavan Nikhilesh , "Shijith Thotton" CC: , , , , Subject: [PATCH v3 12/12] crypto/cnxk: enable dual submission to CPT Date: Wed, 26 Jun 2024 16:25:34 +0530 Message-ID: <20240626105534.1386528-13-asasidharan@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240626105534.1386528-1-asasidharan@marvell.com> References: <20240624062401.4143606-1-asasidharan@marvell.com> <20240626105534.1386528-1-asasidharan@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: VmxH_CXoEvecgIjU4Xckr-1awj2iLek- X-Proofpoint-GUID: VmxH_CXoEvecgIjU4Xckr-1awj2iLek- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-26_05,2024-06-25_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anoob Joseph Submit two instructions in one LMTLINE. Signed-off-by: Anoob Joseph Signed-off-by: Aakash Sasidharan --- drivers/common/cnxk/roc_cpt.c | 17 +- drivers/common/cnxk/roc_cpt.h | 8 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 184 +++++----------------- drivers/crypto/cnxk/cn10k_cryptodev_ops.h | 60 ++++++- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 47 ++---- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 2 + drivers/event/cnxk/cnxk_eventdev_adptr.c | 4 +- 7 files changed, 124 insertions(+), 198 deletions(-) diff --git a/drivers/common/cnxk/roc_cpt.c b/drivers/common/cnxk/roc_cpt.c index 9f283ceb2e..aba2a49d19 100644 --- a/drivers/common/cnxk/roc_cpt.c +++ b/drivers/common/cnxk/roc_cpt.c @@ -1135,8 +1135,8 @@ roc_cpt_iq_enable(struct roc_cpt_lf *lf) } int -roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, struct roc_cpt_lmtline *lmtline, - int lf_id) +roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, struct roc_cpt_lmtline *lmtline, int lf_id, + bool is_dual) { struct roc_cpt_lf *lf; @@ -1145,12 +1145,19 @@ roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, struct roc_cpt_lmtline *lmtline, return -ENOTSUP; lmtline->io_addr = lf->io_addr; - if (roc_model_is_cn10k()) - lmtline->io_addr |= ROC_CN10K_CPT_INST_DW_M1 << 4; + lmtline->fc_thresh = lf->nb_desc - CPT_LF_FC_MIN_THRESHOLD; + + if (roc_model_is_cn10k()) { + if (is_dual) { + lmtline->io_addr |= ROC_CN10K_TWO_CPT_INST_DW_M1 << 4; + lmtline->fc_thresh = lf->nb_desc - 2 * CPT_LF_FC_MIN_THRESHOLD; + } else { + lmtline->io_addr |= ROC_CN10K_CPT_INST_DW_M1 << 4; + } + } lmtline->fc_addr = lf->fc_addr; lmtline->lmt_base = lf->lmt_base; - lmtline->fc_thresh = lf->nb_desc - CPT_LF_FC_MIN_THRESHOLD; return 0; } diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h index 8ef9062ae0..e2e919f80f 100644 --- a/drivers/common/cnxk/roc_cpt.h +++ b/drivers/common/cnxk/roc_cpt.h @@ -200,12 +200,12 @@ int __roc_api roc_cpt_afs_print(struct roc_cpt *roc_cpt); int __roc_api roc_cpt_lfs_print(struct roc_cpt *roc_cpt); void __roc_api roc_cpt_iq_disable(struct roc_cpt_lf *lf); void __roc_api roc_cpt_iq_enable(struct roc_cpt_lf *lf); -int __roc_api roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, - struct roc_cpt_lmtline *lmtline, int lf_id); +int __roc_api roc_cpt_lmtline_init(struct roc_cpt *roc_cpt, struct roc_cpt_lmtline *lmtline, + int lf_id, bool is_dual); void __roc_api roc_cpt_parse_hdr_dump(FILE *file, const struct cpt_parse_hdr_s *cpth); -int __roc_api roc_cpt_ctx_write(struct roc_cpt_lf *lf, void *sa_dptr, - void *sa_cptr, uint16_t sa_len); +int __roc_api roc_cpt_ctx_write(struct roc_cpt_lf *lf, void *sa_dptr, void *sa_cptr, + uint16_t sa_len); void __roc_api roc_cpt_int_misc_cb_register(roc_cpt_int_misc_cb_t cb, void *args); int __roc_api roc_cpt_int_misc_cb_unregister(roc_cpt_int_misc_cb_t cb, void *args); diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index ed964d4d01..780785d656 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -12,11 +12,6 @@ #include #include "roc_cpt.h" -#if defined(__aarch64__) -#include "roc_io.h" -#else -#include "roc_io_generic.h" -#endif #include "roc_idev.h" #include "roc_sso.h" #include "roc_sso_dp.h" @@ -40,8 +35,8 @@ /* Holds information required to send crypto operations in one burst */ struct ops_burst { - struct rte_crypto_op *op[CN10K_PKTS_PER_LOOP]; - uint64_t w2[CN10K_PKTS_PER_LOOP]; + struct rte_crypto_op *op[CN10K_CPT_PKTS_PER_LOOP]; + uint64_t w2[CN10K_CPT_PKTS_PER_LOOP]; struct cn10k_sso_hws *ws; struct cnxk_cpt_qp *qp; uint16_t nb_ops; @@ -55,56 +50,6 @@ struct vec_request { uint64_t w2; }; -#if defined(RTE_ARCH_ARM64) -static __rte_always_inline void __rte_hot -cn10k_cpt_lmtst_dual_submit(uint64_t *io_addr, const uint16_t lmt_id, int *i) -{ - uint64_t lmt_arg; - - /* Check if the total number of instructions is odd or even. */ - const int flag_odd = *i & 0x1; - - /* Reduce i by 1 when odd number of instructions.*/ - *i -= flag_odd; - - if (*i > 2 * CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, *io_addr); - lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, *io_addr); - if (flag_odd) { - *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | - (ROC_CN10K_CPT_INST_DW_M1 << 4); - lmt_arg = (uint64_t)(lmt_id + *i / 2); - roc_lmt_submit_steorl(lmt_arg, *io_addr); - *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | - (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); - *i += 1; - } - } else { - if (*i != 0) { - lmt_arg = - ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, *io_addr); - } - - if (flag_odd) { - *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | - (ROC_CN10K_CPT_INST_DW_M1 << 4); - lmt_arg = (uint64_t)(lmt_id + *i / 2); - roc_lmt_submit_steorl(lmt_arg, *io_addr); - *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | - (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); - *i += 1; - } - } - - rte_io_wmb(); -} -#endif - static inline struct cnxk_se_sess * cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op) { @@ -387,8 +332,8 @@ static uint16_t cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, const bool is_sg_ver2) { - uint64_t lmt_base, lmt_arg, io_addr; struct cpt_inflight_req *infl_req; + uint64_t head, lmt_base, io_addr; uint16_t nb_allowed, count = 0; struct cnxk_cpt_qp *qp = qptr; struct pending_queue *pend_q; @@ -396,7 +341,6 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, union cpt_fc_write_s fc; uint64_t *fc_addr; uint16_t lmt_id; - uint64_t head; int ret, i; pend_q = &qp->pend_q; @@ -426,11 +370,11 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, goto pend_q_commit; } - for (i = 0; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_ops); i++) { + for (i = 0; i < RTE_MIN(CN10K_CPT_PKTS_PER_LOOP, nb_ops); i++) { infl_req = &pend_q->req_queue[head]; infl_req->op_flags = 0; - ret = cn10k_cpt_fill_inst(qp, ops + i, &inst[2 * i], infl_req, is_sg_ver2); + ret = cn10k_cpt_fill_inst(qp, ops + i, &inst[i], infl_req, is_sg_ver2); if (unlikely(ret != 1)) { plt_dp_err("Could not process op: %p", ops + i); if (i == 0) @@ -441,24 +385,12 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops, pending_queue_advance(&head, pq_mask); } - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_ops - i > 0 && i == CN10K_PKTS_PER_LOOP) { - nb_ops -= i; - ops += i; - count += i; + if (nb_ops - i > 0 && i == CN10K_CPT_PKTS_PER_LOOP) { + nb_ops -= CN10K_CPT_PKTS_PER_LOOP; + ops += CN10K_CPT_PKTS_PER_LOOP; + count += CN10K_CPT_PKTS_PER_LOOP; goto again; } @@ -633,7 +565,7 @@ cn10k_cpt_vec_pkt_submission_timeout_handle(void) static inline void cn10k_cpt_vec_submit(struct vec_request vec_tbl[], uint16_t vec_tbl_len, struct cnxk_cpt_qp *qp) { - uint64_t lmt_base, lmt_arg, lmt_id, io_addr; + uint64_t lmt_base, lmt_id, io_addr; union cpt_fc_write_s fc; struct cpt_inst_s *inst; uint16_t burst_size; @@ -661,7 +593,7 @@ cn10k_cpt_vec_submit(struct vec_request vec_tbl[], uint16_t vec_tbl_len, struct again: burst_size = RTE_MIN(CN10K_PKTS_PER_STEORL, vec_tbl_len); for (i = 0; i < burst_size; i++) - cn10k_cpt_vec_inst_fill(&vec_tbl[i], &inst[i * 2], qp, vec_tbl[0].w7); + cn10k_cpt_vec_inst_fill(&vec_tbl[i], &inst[i], qp, vec_tbl[0].w7); do { fc.u64[0] = __atomic_load_n(fc_addr, __ATOMIC_RELAXED); @@ -671,10 +603,7 @@ cn10k_cpt_vec_submit(struct vec_request vec_tbl[], uint16_t vec_tbl_len, struct cn10k_cpt_vec_pkt_submission_timeout_handle(); } while (true); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); vec_tbl_len -= i; @@ -688,12 +617,12 @@ static inline int ca_lmtst_vec_submit(struct ops_burst *burst, struct vec_request vec_tbl[], uint16_t *vec_tbl_len, const bool is_sg_ver2) { - struct cpt_inflight_req *infl_reqs[CN10K_PKTS_PER_LOOP]; - uint64_t lmt_base, lmt_arg, io_addr; + struct cpt_inflight_req *infl_reqs[CN10K_CPT_PKTS_PER_LOOP]; uint16_t lmt_id, len = *vec_tbl_len; struct cpt_inst_s *inst, *inst_base; struct cpt_inflight_req *infl_req; struct rte_event_vector *vec; + uint64_t lmt_base, io_addr; union cpt_fc_write_s fc; struct cnxk_cpt_qp *qp; uint64_t *fc_addr; @@ -730,7 +659,7 @@ ca_lmtst_vec_submit(struct ops_burst *burst, struct vec_request vec_tbl[], uint1 } for (i = 0; i < burst->nb_ops; i++) { - inst = &inst_base[2 * i]; + inst = &inst_base[i]; infl_req = infl_reqs[i]; infl_req->op_flags = 0; @@ -790,24 +719,12 @@ next_op:; if (CNXK_TT_FROM_TAG(burst->ws->gw_rdata) == SSO_TT_ORDERED) roc_sso_hws_head_wait(burst->ws->base); - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); /* Store w7 of last successfully filled instruction */ inst = &inst_base[2 * (i - 1)]; vec_tbl[0].w7 = inst->w7; - rte_io_wmb(); - put: if (i != burst->nb_ops) rte_mempool_put_bulk(qp->ca.req_mp, (void *)&infl_reqs[i], burst->nb_ops - i); @@ -820,10 +737,10 @@ next_op:; static inline uint16_t ca_lmtst_burst_submit(struct ops_burst *burst, const bool is_sg_ver2) { - struct cpt_inflight_req *infl_reqs[CN10K_PKTS_PER_LOOP]; - uint64_t lmt_base, lmt_arg, io_addr; + struct cpt_inflight_req *infl_reqs[CN10K_CPT_PKTS_PER_LOOP]; struct cpt_inst_s *inst, *inst_base; struct cpt_inflight_req *infl_req; + uint64_t lmt_base, io_addr; union cpt_fc_write_s fc; struct cnxk_cpt_qp *qp; uint64_t *fc_addr; @@ -854,7 +771,7 @@ ca_lmtst_burst_submit(struct ops_burst *burst, const bool is_sg_ver2) } for (i = 0; i < burst->nb_ops; i++) { - inst = &inst_base[2 * i]; + inst = &inst_base[i]; infl_req = infl_reqs[i]; infl_req->op_flags = 0; @@ -891,19 +808,7 @@ ca_lmtst_burst_submit(struct ops_burst *burst, const bool is_sg_ver2) if (CNXK_TT_FROM_TAG(burst->ws->gw_rdata) == SSO_TT_ORDERED) roc_sso_hws_head_wait(burst->ws->base); - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); put: if (unlikely(i != burst->nb_ops)) @@ -965,7 +870,7 @@ cn10k_cpt_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_ev burst.op[burst.nb_ops] = op; /* Max nb_ops per burst check */ - if (++burst.nb_ops == CN10K_PKTS_PER_LOOP) { + if (++burst.nb_ops == CN10K_CPT_PKTS_PER_LOOP) { if (is_vector) submitted = ca_lmtst_vec_submit(&burst, vec_tbl, &vec_tbl_len, is_sg_ver2); @@ -1467,8 +1372,6 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, vf = cdev->data->dev_private; - const int nb_pkts_per_loop = 2 * CN10K_PKTS_PER_LOOP; - lmt_base = vf->rx_inj_lmtline.lmt_base; io_addr = vf->rx_inj_lmtline.io_addr; fc_addr = vf->rx_inj_lmtline.fc_addr; @@ -1488,7 +1391,7 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, if (unlikely(fc.s.qsize > fc_thresh)) goto exit; - for (; i < RTE_MIN(nb_pkts_per_loop, nb_pkts); i++) { + for (; i < RTE_MIN(CN10K_CPT_PKTS_PER_LOOP, nb_pkts); i++) { m = pkts[i]; sec_sess = (struct cn10k_sec_session *)sess[i]; @@ -1549,11 +1452,11 @@ cn10k_cryptodev_sec_inb_rx_inject(void *dev, struct rte_mbuf **pkts, cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_pkts - i > 0 && i == nb_pkts_per_loop) { - nb_pkts -= nb_pkts_per_loop; - pkts += nb_pkts_per_loop; - count += nb_pkts_per_loop; - sess += nb_pkts_per_loop; + if (nb_pkts - i > 0 && i == CN10K_CPT_PKTS_PER_LOOP) { + nb_pkts -= CN10K_CPT_PKTS_PER_LOOP; + pkts += CN10K_CPT_PKTS_PER_LOOP; + count += CN10K_CPT_PKTS_PER_LOOP; + sess += CN10K_CPT_PKTS_PER_LOOP; goto again; } @@ -1652,8 +1555,8 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym const bool is_sgv2) { uint16_t lmt_id, nb_allowed, nb_ops = vec->num; - uint64_t lmt_base, lmt_arg, io_addr, head; struct cpt_inflight_req *infl_req; + uint64_t lmt_base, io_addr, head; struct cnxk_cpt_qp *qp = qpair; struct cnxk_sym_dp_ctx *dp_ctx; struct pending_queue *pend_q; @@ -1690,7 +1593,7 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym goto pend_q_commit; } - for (i = 0; i < RTE_MIN(CN10K_PKTS_PER_LOOP, nb_ops); i++) { + for (i = 0; i < RTE_MIN(CN10K_CPT_PKTS_PER_LOOP, nb_ops); i++) { struct cnxk_iov iov; index = count + i; @@ -1698,7 +1601,7 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym infl_req->op_flags = 0; cnxk_raw_burst_to_iov(vec, &ofs, index, &iov); - ret = cn10k_cpt_raw_fill_inst(&iov, qp, dp_ctx, &inst[2 * i], infl_req, + ret = cn10k_cpt_raw_fill_inst(&iov, qp, dp_ctx, &inst[i], infl_req, user_data[index], is_sgv2); if (unlikely(ret != 1)) { plt_dp_err("Could not process vec: %d", index); @@ -1712,21 +1615,9 @@ cn10k_cpt_raw_enqueue_burst(void *qpair, uint8_t *drv_ctx, struct rte_crypto_sym pending_queue_advance(&head, pq_mask); } - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); - if (nb_ops - i > 0 && i == CN10K_PKTS_PER_LOOP) { + if (nb_ops - i > 0 && i == CN10K_CPT_PKTS_PER_LOOP) { nb_ops -= i; count += i; goto again; @@ -1767,8 +1658,8 @@ cn10k_cpt_raw_enqueue(void *qpair, uint8_t *drv_ctx, struct rte_crypto_vec *data struct rte_crypto_va_iova_ptr *aad_or_auth_iv, void *user_data, const bool is_sgv2) { - uint64_t lmt_base, lmt_arg, io_addr, head; struct cpt_inflight_req *infl_req; + uint64_t lmt_base, io_addr, head; struct cnxk_cpt_qp *qp = qpair; struct cnxk_sym_dp_ctx *dp_ctx; uint16_t lmt_id, nb_allowed; @@ -1776,7 +1667,7 @@ cn10k_cpt_raw_enqueue(void *qpair, uint8_t *drv_ctx, struct rte_crypto_vec *data union cpt_fc_write_s fc; struct cnxk_iov iov; uint64_t *fc_addr; - int ret; + int ret, i = 1; struct pending_queue *pend_q = &qp->pend_q; const uint64_t pq_mask = pend_q->pq_mask; @@ -1813,10 +1704,7 @@ cn10k_cpt_raw_enqueue(void *qpair, uint8_t *drv_ctx, struct rte_crypto_vec *data pending_queue_advance(&head, pq_mask); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); pend_q->head = head; pend_q->time_out = rte_get_timer_cycles() + DEFAULT_COMMAND_TIMEOUT * rte_get_timer_hz(); diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h index 406c4abc7f..be76c49a65 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.h @@ -5,15 +5,21 @@ #ifndef _CN10K_CRYPTODEV_OPS_H_ #define _CN10K_CRYPTODEV_OPS_H_ -#include #include +#include #include #include +#if defined(__aarch64__) +#include "roc_io.h" +#else +#include "roc_io_generic.h" +#endif + #include "cnxk_cryptodev.h" -#define CN10K_PKTS_PER_LOOP 32 -#define CN10K_PKTS_PER_STEORL 16 +#define CN10K_PKTS_PER_STEORL 32 +#define CN10K_LMTLINES_PER_STEORL 16 extern struct rte_cryptodev_ops cn10k_cpt_ops; @@ -34,4 +40,52 @@ __rte_internal uint16_t __rte_hot cn10k_cpt_sg_ver2_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16_t nb_events); +static __rte_always_inline void __rte_hot +cn10k_cpt_lmtst_dual_submit(uint64_t *io_addr, const uint16_t lmt_id, int *i) +{ + uint64_t lmt_arg; + + /* Check if the total number of instructions is odd or even. */ + const int flag_odd = *i & 0x1; + + /* Reduce i by 1 when odd number of instructions.*/ + *i -= flag_odd; + + if (*i > CN10K_PKTS_PER_STEORL) { + lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | (CN10K_LMTLINES_PER_STEORL - 1) << 12 | + (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, *io_addr); + lmt_arg = ROC_CN10K_DUAL_CPT_LMT_ARG | + (*i / 2 - CN10K_LMTLINES_PER_STEORL - 1) << 12 | + (uint64_t)(lmt_id + CN10K_LMTLINES_PER_STEORL); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + if (flag_odd) { + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_CPT_INST_DW_M1 << 4); + lmt_arg = (uint64_t)(lmt_id + *i / 2); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); + *i += 1; + } + } else { + if (*i != 0) { + lmt_arg = + ROC_CN10K_DUAL_CPT_LMT_ARG | (*i / 2 - 1) << 12 | (uint64_t)lmt_id; + roc_lmt_submit_steorl(lmt_arg, *io_addr); + } + + if (flag_odd) { + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_CPT_INST_DW_M1 << 4); + lmt_arg = (uint64_t)(lmt_id + *i / 2); + roc_lmt_submit_steorl(lmt_arg, *io_addr); + *io_addr = (*io_addr & ~(uint64_t)(0x7 << 4)) | + (ROC_CN10K_TWO_CPT_INST_DW_M1 << 4); + *i += 1; + } + } + + rte_io_wmb(); +} #endif /* _CN10K_CRYPTODEV_OPS_H_ */ diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index 6acaa4413b..cfcfa79fdf 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -431,7 +431,6 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, struct rte_pci_device *pci_dev; struct cnxk_cpt_qp *qp; uint32_t nb_desc; - uint64_t io_addr; int ret; if (dev->data->queue_pairs[qp_id] != NULL) @@ -467,7 +466,7 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, roc_cpt->lf[qp_id] = &qp->lf; - ret = roc_cpt_lmtline_init(roc_cpt, &qp->lmtline, qp_id); + ret = roc_cpt_lmtline_init(roc_cpt, &qp->lmtline, qp_id, true); if (ret < 0) { roc_cpt->lf[qp_id] = NULL; plt_err("Could not init lmtline for queue pair %d", qp_id); @@ -478,7 +477,7 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, dev->data->queue_pairs[qp_id] = qp; if (qp_id == vf->rx_inject_qp) { - ret = roc_cpt_lmtline_init(roc_cpt, &vf->rx_inj_lmtline, vf->rx_inject_qp); + ret = roc_cpt_lmtline_init(roc_cpt, &vf->rx_inj_lmtline, vf->rx_inject_qp, true); if (ret) { plt_err("Could not init lmtline Rx inject"); goto exit; @@ -486,14 +485,6 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, vf->rx_inj_sso_pf_func = roc_idev_nix_inl_dev_pffunc_get(); - /* Update IO addr to enable dual submission */ - io_addr = vf->rx_inj_lmtline.io_addr; - io_addr = (io_addr & ~(uint64_t)(0x7 << 4)) | ROC_CN10K_TWO_CPT_INST_DW_M1 << 4; - vf->rx_inj_lmtline.io_addr = io_addr; - - /* Update FC threshold to reflect dual submission */ - vf->rx_inj_lmtline.fc_thresh -= 32; - /* Block the queue for other submissions */ qp->pend_q.pq_mask = 0; } @@ -969,44 +960,28 @@ rte_pmd_cnxk_crypto_qptr_get(uint8_t dev_id, uint16_t qp_id) static inline void cnxk_crypto_cn10k_submit(void *qptr, void *inst, uint16_t nb_inst) { - uint64_t lmt_base, lmt_arg, io_addr; struct cnxk_cpt_qp *qp = qptr; - uint16_t i, j, lmt_id; + uint64_t lmt_base, io_addr; + uint16_t lmt_id; void *lmt_dst; + int i; lmt_base = qp->lmtline.lmt_base; io_addr = qp->lmtline.io_addr; ROC_LMT_BASE_ID_GET(lmt_base, lmt_id); -again: - i = RTE_MIN(nb_inst, CN10K_PKTS_PER_LOOP); lmt_dst = PLT_PTR_CAST(lmt_base); +again: + i = RTE_MIN(nb_inst, CN10K_CPT_PKTS_PER_LOOP); - for (j = 0; j < i; j++) { - rte_memcpy(lmt_dst, inst, sizeof(struct cpt_inst_s)); - inst = RTE_PTR_ADD(inst, sizeof(struct cpt_inst_s)); - lmt_dst = RTE_PTR_ADD(lmt_dst, 2 * sizeof(struct cpt_inst_s)); - } - - rte_io_wmb(); - - if (i > CN10K_PKTS_PER_STEORL) { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - CN10K_PKTS_PER_STEORL - 1) << 12 | - (uint64_t)(lmt_id + CN10K_PKTS_PER_STEORL); - roc_lmt_submit_steorl(lmt_arg, io_addr); - } else { - lmt_arg = ROC_CN10K_CPT_LMT_ARG | (i - 1) << 12 | (uint64_t)lmt_id; - roc_lmt_submit_steorl(lmt_arg, io_addr); - } + memcpy(lmt_dst, inst, i * sizeof(struct cpt_inst_s)); - rte_io_wmb(); + cn10k_cpt_lmtst_dual_submit(&io_addr, lmt_id, &i); if (nb_inst - i > 0) { - nb_inst -= i; + nb_inst -= CN10K_CPT_PKTS_PER_LOOP; + inst = RTE_PTR_ADD(inst, CN10K_CPT_PKTS_PER_LOOP * sizeof(struct cpt_inst_s)); goto again; } } diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h index 9de7e432e4..caf6ac35e5 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h @@ -25,6 +25,8 @@ #define MOD_INC(i, l) ((i) == (l - 1) ? (i) = 0 : (i)++) +#define CN10K_CPT_PKTS_PER_LOOP 64 + /* Macros to form words in CPT instruction */ #define CNXK_CPT_INST_W2(tag, tt, grp, rvu_pf_func) \ ((tag) | ((uint64_t)(tt) << 32) | ((uint64_t)(grp) << 34) | \ diff --git a/drivers/event/cnxk/cnxk_eventdev_adptr.c b/drivers/event/cnxk/cnxk_eventdev_adptr.c index 98db11ad61..2c049e7041 100644 --- a/drivers/event/cnxk/cnxk_eventdev_adptr.c +++ b/drivers/event/cnxk/cnxk_eventdev_adptr.c @@ -632,7 +632,7 @@ crypto_adapter_qp_setup(const struct rte_cryptodev *cdev, struct cnxk_cpt_qp *qp * simultaneous enqueue from all available cores. */ if (roc_model_is_cn10k()) - nb_desc_min = rte_lcore_count() * 32; + nb_desc_min = rte_lcore_count() * CN10K_CPT_PKTS_PER_LOOP; else nb_desc_min = rte_lcore_count() * 2; @@ -707,7 +707,7 @@ crypto_adapter_qp_free(struct cnxk_cpt_qp *qp) rte_mempool_free(qp->ca.req_mp); qp->ca.enabled = false; - ret = roc_cpt_lmtline_init(qp->lf.roc_cpt, &qp->lmtline, qp->lf.lf_id); + ret = roc_cpt_lmtline_init(qp->lf.roc_cpt, &qp->lmtline, qp->lf.lf_id, true); if (ret < 0) { plt_err("Could not reset lmtline for queue pair %d", qp->lf.lf_id); return ret;