From patchwork Sun May 1 19:24:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 110552 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E5913A0093; Sun, 1 May 2022 21:25:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 955294113D; Sun, 1 May 2022 21:25:23 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 152CB42821 for ; Sun, 1 May 2022 21:25:21 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2419iarm011533; Sun, 1 May 2022 12:25:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=KLelthhYYEPGvDBXnUKSbNFaE58nYBw8QJ4jTXFBfBY=; b=UQslGUl1PXJsSNR59lTAk4PSIZreHXvvK71DNI+N3suMwRoFFTI+9gxfdSlwaUnaD98Z FrzfxcoFlZE/lRKupmWpiHxaAEGXnV2FAfoTuQlNvoaxRPynsWnsoWqOis3KaDOauGHF SmQ9j+51O/deYQ6eurNTVG8k/gv/2aBb3EJcrty7wopC6hNbCnQS/tlX7DQB+YiL/mma aqOcStPLxxSoZwKg5G86+4DIUQTFM66dAvzBFKSyZLFr22K7qT9FZewFjSh3pl3swNJF 1oaJJ1qiMElbGrObvS2ux1so6sNjZRvrOa9NRvKPL9A2p4B/e0njseY7liSHVRq3x5JF XA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3fs4mkk7ah-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 01 May 2022 12:25:21 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 1 May 2022 12:25:19 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Sun, 1 May 2022 12:25:18 -0700 Received: from localhost.localdomain (unknown [10.28.48.55]) by maili.marvell.com (Postfix) with ESMTP id 03BBE3F70A5; Sun, 1 May 2022 12:25:15 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , Akhil Goyal , Fan Zhang , Abhinandan Gujjar Subject: [PATCH v4 2/7] crypto/cnxk: add event metadata set operation Date: Mon, 2 May 2022 00:54:52 +0530 Message-ID: <20220501192457.3670278-3-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220501192457.3670278-1-gakhil@marvell.com> References: <20220421143720.1583062-1-gakhil@marvell.com> <20220501192457.3670278-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 9H97EOpwy6wPV0fErsAdoHbeEDZR53Cx X-Proofpoint-ORIG-GUID: 9H97EOpwy6wPV0fErsAdoHbeEDZR53Cx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-01_07,2022-04-28_01,2022-02-23_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Added cryptodev operation for setting event crypto metadata for all supported sessions - sym/asym/security. Signed-off-by: Volodymyr Fialko Signed-off-by: Akhil Goyal Acked-by: Fan Zhang Acked-by: Abhinandan Gujjar Acked-by: Abhinandan Gujjar --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 144 +++++++++++++++++++--- drivers/crypto/cnxk/cn10k_ipsec.h | 2 + drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 138 ++++++++++++++++++--- drivers/crypto/cnxk/cn9k_ipsec.h | 2 + drivers/crypto/cnxk/cnxk_ae.h | 2 + drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 18 --- drivers/crypto/cnxk/cnxk_se.h | 2 + 7 files changed, 255 insertions(+), 53 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index c4d5d039ec..01aa0d6870 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -264,30 +264,136 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return count + i; } -uint16_t -cn10k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) +static int +cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, + void *sess, + enum rte_crypto_op_type op_type, + enum rte_crypto_op_sess_type sess_type, + void *mdata) { - union rte_event_crypto_metadata *ec_mdata; - struct cpt_inflight_req *infl_req; + union rte_event_crypto_metadata *ec_mdata = mdata; struct rte_event *rsp_info; - uint64_t lmt_base, lmt_arg; - struct cpt_inst_s *inst; struct cnxk_cpt_qp *qp; uint8_t cdev_id; - uint16_t lmt_id; - uint16_t qp_id; - int ret; - - ec_mdata = cnxk_event_crypto_mdata_get(op); - if (!ec_mdata) { - rte_errno = EINVAL; - return 0; - } + int16_t qp_id; + uint64_t w2; + /* Get queue pair */ cdev_id = ec_mdata->request_info.cdev_id; qp_id = ec_mdata->request_info.queue_pair_id; qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id]; + + /* Prepare w2 */ rsp_info = &ec_mdata->response_info; + w2 = CNXK_CPT_INST_W2( + (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, + rsp_info->sched_type, rsp_info->queue_id, 0); + + /* Set meta according to session type */ + if (op_type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + if (sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct cn10k_sec_session *priv; + struct cn10k_ipsec_sa *sa; + + priv = get_sec_session_private_data(sess); + sa = &priv->sa; + sa->qp = qp; + sa->inst.w2 = w2; + } else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct cnxk_se_sess *priv; + + priv = get_sym_session_private_data( + sess, cn10k_cryptodev_driver_id); + priv->qp = qp; + priv->cpt_inst_w2 = w2; + } else + return -EINVAL; + } else if (op_type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct rte_cryptodev_asym_session *asym_sess = sess; + struct cnxk_ae_sess *priv; + + priv = (struct cnxk_ae_sess *)asym_sess->sess_private_data; + priv->qp = qp; + priv->cpt_inst_w2 = w2; + } else + return -EINVAL; + } else + return -EINVAL; + + return 0; +} + +static inline int +cn10k_ca_meta_info_extract(struct rte_crypto_op *op, + struct cnxk_cpt_qp **qp, uint64_t *w2) +{ + if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct cn10k_sec_session *priv; + struct cn10k_ipsec_sa *sa; + + priv = get_sec_session_private_data(op->sym->sec_session); + sa = &priv->sa; + *qp = sa->qp; + *w2 = sa->inst.w2; + } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct cnxk_se_sess *priv; + + priv = get_sym_session_private_data( + op->sym->session, cn10k_cryptodev_driver_id); + *qp = priv->qp; + *w2 = priv->cpt_inst_w2; + } else { + union rte_event_crypto_metadata *ec_mdata; + struct rte_event *rsp_info; + uint8_t cdev_id; + uint16_t qp_id; + + ec_mdata = (union rte_event_crypto_metadata *) + ((uint8_t *)op + op->private_data_offset); + if (!ec_mdata) + return -EINVAL; + rsp_info = &ec_mdata->response_info; + cdev_id = ec_mdata->request_info.cdev_id; + qp_id = ec_mdata->request_info.queue_pair_id; + *qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id]; + *w2 = CNXK_CPT_INST_W2( + (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, + rsp_info->sched_type, rsp_info->queue_id, 0); + } + } else if (op->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct rte_cryptodev_asym_session *asym_sess; + struct cnxk_ae_sess *priv; + + asym_sess = op->asym->session; + priv = (struct cnxk_ae_sess *)asym_sess->sess_private_data; + *qp = priv->qp; + *w2 = priv->cpt_inst_w2; + } else + return -EINVAL; + } else + return -EINVAL; + + return 0; +} + +uint16_t +cn10k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) +{ + struct cpt_inflight_req *infl_req; + uint64_t lmt_base, lmt_arg, w2; + struct cpt_inst_s *inst; + struct cnxk_cpt_qp *qp; + uint16_t lmt_id; + int ret; + + ret = cn10k_ca_meta_info_extract(op, &qp, &w2); + if (unlikely(ret)) { + rte_errno = EINVAL; + return 0; + } if (unlikely(!qp->ca.enabled)) { rte_errno = EINVAL; @@ -316,9 +422,7 @@ cn10k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) infl_req->qp = qp; inst->w0.u64 = 0; inst->res_addr = (uint64_t)&infl_req->res; - inst->w2.u64 = CNXK_CPT_INST_W2( - (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, - rsp_info->sched_type, rsp_info->queue_id, 0); + inst->w2.u64 = w2; inst->w3.u64 = CNXK_CPT_INST_W3(1, infl_req); if (roc_cpt_is_iq_full(&qp->lf)) { @@ -327,7 +431,7 @@ cn10k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) return 0; } - if (!rsp_info->sched_type) + if (inst->w2.s.tt == RTE_SCHED_TYPE_ORDERED) roc_sso_hws_head_wait(tag_op); lmt_arg = ROC_CN10K_CPT_LMT_ARG | (uint64_t)lmt_id; @@ -592,4 +696,6 @@ struct rte_cryptodev_ops cn10k_cpt_ops = { .asym_session_configure = cnxk_ae_session_cfg, .asym_session_clear = cnxk_ae_session_clear, + /* Event crypto ops */ + .session_ev_mdata_set = cn10k_cpt_crypto_adapter_ev_mdata_set, }; diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h index 647a71cdd5..1c1d904799 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.h +++ b/drivers/crypto/cnxk/cn10k_ipsec.h @@ -20,6 +20,8 @@ struct cn10k_ipsec_sa { uint16_t iv_offset; uint8_t iv_length; bool is_outbound; + /** Queue pair */ + struct cnxk_cpt_qp *qp; /** * End of SW mutable area diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index d3d441cb24..98fa97ef01 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -316,28 +316,134 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return count; } -uint16_t -cn9k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) +static int +cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, + void *sess, + enum rte_crypto_op_type op_type, + enum rte_crypto_op_sess_type sess_type, + void *mdata) { - union rte_event_crypto_metadata *ec_mdata; - struct cpt_inflight_req *infl_req; + union rte_event_crypto_metadata *ec_mdata = mdata; struct rte_event *rsp_info; struct cnxk_cpt_qp *qp; - struct cpt_inst_s inst; uint8_t cdev_id; uint16_t qp_id; - int ret; - - ec_mdata = cnxk_event_crypto_mdata_get(op); - if (!ec_mdata) { - rte_errno = EINVAL; - return 0; - } + uint64_t w2; + /* Get queue pair */ cdev_id = ec_mdata->request_info.cdev_id; qp_id = ec_mdata->request_info.queue_pair_id; qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id]; + + /* Prepare w2 */ rsp_info = &ec_mdata->response_info; + w2 = CNXK_CPT_INST_W2( + (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, + rsp_info->sched_type, rsp_info->queue_id, 0); + + /* Set meta according to session type */ + if (op_type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + if (sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct cn9k_sec_session *priv; + struct cn9k_ipsec_sa *sa; + + priv = get_sec_session_private_data(sess); + sa = &priv->sa; + sa->qp = qp; + sa->inst.w2 = w2; + } else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct cnxk_se_sess *priv; + + priv = get_sym_session_private_data( + sess, cn9k_cryptodev_driver_id); + priv->qp = qp; + priv->cpt_inst_w2 = w2; + } else + return -EINVAL; + } else if (op_type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct rte_cryptodev_asym_session *asym_sess = sess; + struct cnxk_ae_sess *priv; + + priv = (struct cnxk_ae_sess *)asym_sess->sess_private_data; + priv->qp = qp; + priv->cpt_inst_w2 = w2; + } else + return -EINVAL; + } else + return -EINVAL; + + return 0; +} + +static inline int +cn9k_ca_meta_info_extract(struct rte_crypto_op *op, + struct cnxk_cpt_qp **qp, struct cpt_inst_s *inst) +{ + if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct cn9k_sec_session *priv; + struct cn9k_ipsec_sa *sa; + + priv = get_sec_session_private_data(op->sym->sec_session); + sa = &priv->sa; + *qp = sa->qp; + inst->w2.u64 = sa->inst.w2; + } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct cnxk_se_sess *priv; + + priv = get_sym_session_private_data( + op->sym->session, cn9k_cryptodev_driver_id); + *qp = priv->qp; + inst->w2.u64 = priv->cpt_inst_w2; + } else { + union rte_event_crypto_metadata *ec_mdata; + struct rte_event *rsp_info; + uint8_t cdev_id; + uint16_t qp_id; + + ec_mdata = (union rte_event_crypto_metadata *) + ((uint8_t *)op + op->private_data_offset); + if (!ec_mdata) + return -EINVAL; + rsp_info = &ec_mdata->response_info; + cdev_id = ec_mdata->request_info.cdev_id; + qp_id = ec_mdata->request_info.queue_pair_id; + *qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id]; + inst->w2.u64 = CNXK_CPT_INST_W2( + (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, + rsp_info->sched_type, rsp_info->queue_id, 0); + } + } else if (op->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct rte_cryptodev_asym_session *asym_sess; + struct cnxk_ae_sess *priv; + + asym_sess = op->asym->session; + priv = (struct cnxk_ae_sess *)asym_sess->sess_private_data; + *qp = priv->qp; + inst->w2.u64 = priv->cpt_inst_w2; + } else + return -EINVAL; + } else + return -EINVAL; + + return 0; +} + +uint16_t +cn9k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) +{ + struct cpt_inflight_req *infl_req; + struct cnxk_cpt_qp *qp; + struct cpt_inst_s inst; + int ret; + + ret = cn9k_ca_meta_info_extract(op, &qp, &inst); + if (unlikely(ret)) { + rte_errno = EINVAL; + return 0; + } if (unlikely(!qp->ca.enabled)) { rte_errno = EINVAL; @@ -362,9 +468,6 @@ cn9k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) infl_req->qp = qp; inst.w0.u64 = 0; inst.res_addr = (uint64_t)&infl_req->res; - inst.w2.u64 = CNXK_CPT_INST_W2( - (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, - rsp_info->sched_type, rsp_info->queue_id, 0); inst.w3.u64 = CNXK_CPT_INST_W3(1, infl_req); if (roc_cpt_is_iq_full(&qp->lf)) { @@ -373,7 +476,7 @@ cn9k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) return 0; } - if (!rsp_info->sched_type) + if (inst.w2.s.tt == RTE_SCHED_TYPE_ORDERED) roc_sso_hws_head_wait(tag_op); cn9k_cpt_inst_submit(&inst, qp->lmtline.lmt_base, qp->lmtline.io_addr); @@ -613,4 +716,7 @@ struct rte_cryptodev_ops cn9k_cpt_ops = { .asym_session_configure = cnxk_ae_session_cfg, .asym_session_clear = cnxk_ae_session_clear, + /* Event crypto ops */ + .session_ev_mdata_set = cn9k_cpt_crypto_adapter_ev_mdata_set, + }; diff --git a/drivers/crypto/cnxk/cn9k_ipsec.h b/drivers/crypto/cnxk/cn9k_ipsec.h index f3acad561b..499dbc2782 100644 --- a/drivers/crypto/cnxk/cn9k_ipsec.h +++ b/drivers/crypto/cnxk/cn9k_ipsec.h @@ -42,6 +42,8 @@ struct cn9k_ipsec_sa { struct cnxk_on_ipsec_ar ar; /** Anti replay window size */ uint32_t replay_win_sz; + /** Queue pair */ + struct cnxk_cpt_qp *qp; }; struct cn9k_sec_session { diff --git a/drivers/crypto/cnxk/cnxk_ae.h b/drivers/crypto/cnxk/cnxk_ae.h index 01ccfcd334..10854c79c8 100644 --- a/drivers/crypto/cnxk/cnxk_ae.h +++ b/drivers/crypto/cnxk/cnxk_ae.h @@ -22,6 +22,8 @@ struct cnxk_ae_sess { uint64_t *cnxk_fpm_iova; struct roc_ae_ec_group **ec_grp; uint64_t cpt_inst_w7; + uint64_t cpt_inst_w2; + struct cnxk_cpt_qp *qp; }; static __rte_always_inline void diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h index ab0f00ee7c..7ece0214dc 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h @@ -125,24 +125,6 @@ int cnxk_ae_session_cfg(struct rte_cryptodev *dev, struct rte_cryptodev_asym_session *sess); void cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp); -static inline union rte_event_crypto_metadata * -cnxk_event_crypto_mdata_get(struct rte_crypto_op *op) -{ - union rte_event_crypto_metadata *ec_mdata; - - if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) - ec_mdata = rte_cryptodev_sym_session_get_user_data( - op->sym->session); - else if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS && - op->private_data_offset) - ec_mdata = (union rte_event_crypto_metadata - *)((uint8_t *)op + op->private_data_offset); - else - return NULL; - - return ec_mdata; -} - static __rte_always_inline void pending_queue_advance(uint64_t *index, const uint64_t mask) { diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index ce7ca2eda9..a339b80a87 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -33,6 +33,8 @@ struct cnxk_se_sess { uint16_t auth_iv_offset; uint32_t salt; uint64_t cpt_inst_w7; + uint64_t cpt_inst_w2; + struct cnxk_cpt_qp *qp; struct roc_se_ctx roc_se_ctx; } __rte_cache_aligned;