From patchwork Tue Jun 23 12:12:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejasree Kondoj X-Patchwork-Id: 72031 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E3A0BA0350; Tue, 23 Jun 2020 13:20:06 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 43F6C1D67E; Tue, 23 Jun 2020 13:19:32 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id EBD3C1D67A for ; Tue, 23 Jun 2020 13:19:30 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 05NBEvWa024290; Tue, 23 Jun 2020 04:19:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=WMH5nSUcbmonHQ5djw6XRI2Rqz9lYOlb3wvcIh1zpQ0=; b=ZpJ0YGk4EboHjRGnS+KK9axzF2IJHVPKCsb0vJMfzY6SfYKE0JbKP7P7n0JpJ81AOeo+ Wp9FkrDmSF9t3UbxprBj0gpWxKQhAbwxTuHVJmUqtIZkbsMY344ROQEbltR9pjVFCvsg OEsFBveIuCQMWUrYRulLnC9Ze2aVA7hhmXepdATYEK1uZtqOCfinRlfuy6fm/iHsmyHx VWcNtB4yX1v2BiZ2J4eTmOuqG+cO6bj6Do5AfzckPdIc1/ud8q19MBVKU7C8iWw1ykFY h9Kjdb2WGkxYLmGtHd0z+fgq8sjWZbnxtT9fNiaDF9zMxkQ89BFZEyfFn7i0iPQNepfq 0g== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 31sftpnn2w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 23 Jun 2020 04:19:30 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 23 Jun 2020 04:19:29 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 23 Jun 2020 04:19:28 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 23 Jun 2020 04:19:28 -0700 Received: from hyd1554T5810.caveonetworks.com.com (unknown [10.29.57.11]) by maili.marvell.com (Postfix) with ESMTP id 8A0623F7040; Tue, 23 Jun 2020 04:19:26 -0700 (PDT) From: Tejasree Kondoj To: Akhil Goyal , Radu Nicolau CC: Vamsi Attunuru , Narayana Prasad , Anoob Joseph , Tejasree Kondoj , Date: Tue, 23 Jun 2020 17:42:27 +0530 Message-ID: <20200623121228.10355-8-ktejasree@marvell.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200623121228.10355-1-ktejasree@marvell.com> References: <20200623121228.10355-1-ktejasree@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-23_06:2020-06-23, 2020-06-23 signatures=0 Subject: [dpdk-dev] [PATCH 7/8] crypto/octeontx2: add cryptodev sec enqueue routine X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru Signed-off-by: Vamsi Attunuru Signed-off-by: Tejasree Kondoj --- drivers/crypto/octeontx2/otx2_cryptodev.h | 8 + drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 36 +++- drivers/crypto/octeontx2/otx2_ipsec_po.h | 6 + drivers/crypto/octeontx2/otx2_ipsec_po_ops.h | 179 ++++++++++++++++++ 4 files changed, 228 insertions(+), 1 deletion(-) create mode 100644 drivers/crypto/octeontx2/otx2_ipsec_po_ops.h diff --git a/drivers/crypto/octeontx2/otx2_cryptodev.h b/drivers/crypto/octeontx2/otx2_cryptodev.h index e7a1730b22..f329741b38 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev.h +++ b/drivers/crypto/octeontx2/otx2_cryptodev.h @@ -6,6 +6,7 @@ #define _OTX2_CRYPTODEV_H_ #include "cpt_common.h" +#include "cpt_hw_types.h" #include "otx2_dev.h" @@ -33,6 +34,13 @@ struct otx2_cpt_vf { /**< CPT device capabilities */ }; +struct cpt_meta_info { + uint64_t deq_op_info[4]; + uint64_t comp_code_sz; + union cpt_res_s cpt_res __rte_aligned(16); + struct cpt_request_info cpt_req __rte_aligned(8); +}; + #define CPT_LOGTYPE otx2_cpt_logtype extern int otx2_cpt_logtype; diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c index 08254062e9..d796488def 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c @@ -13,8 +13,10 @@ #include "otx2_cryptodev_hw_access.h" #include "otx2_cryptodev_mbox.h" #include "otx2_cryptodev_ops.h" +#include "otx2_ipsec_po_ops.h" #include "otx2_mbox.h" #include "otx2_sec_idev.h" +#include "otx2_security.h" #include "cpt_hw_types.h" #include "cpt_pmd_logs.h" @@ -603,6 +605,36 @@ otx2_cpt_enqueue_sym(struct otx2_cpt_qp *qp, struct rte_crypto_op *op, return ret; } +static __rte_always_inline int __rte_hot +otx2_cpt_enqueue_sec(struct otx2_cpt_qp *qp, struct rte_crypto_op *op, + struct pending_queue *pend_q) +{ + struct otx2_sec_session_ipsec_lp *sess; + struct otx2_ipsec_po_sa_ctl *ctl_wrd; + struct otx2_sec_session *priv; + struct cpt_request_info *req; + int ret; + + priv = get_sec_session_private_data(op->sym->sec_session); + sess = &priv->ipsec.lp; + + ctl_wrd = &sess->in_sa.ctl; + + if (ctl_wrd->direction == OTX2_IPSEC_PO_SA_DIRECTION_OUTBOUND) + ret = process_outb_sa(op, sess, &qp->meta_info, (void **)&req); + else + ret = process_inb_sa(op, sess, &qp->meta_info, (void **)&req); + + if (unlikely(ret)) { + otx2_err("Crypto req : op %p, ret 0x%x", op, ret); + return ret; + } + + ret = otx2_cpt_enqueue_req(qp, pend_q, req); + + return ret; +} + static __rte_always_inline int __rte_hot otx2_cpt_enqueue_sym_sessless(struct otx2_cpt_qp *qp, struct rte_crypto_op *op, struct pending_queue *pend_q) @@ -656,7 +688,9 @@ otx2_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) for (count = 0; count < nb_ops; count++) { op = ops[count]; if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { - if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) + if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) + ret = otx2_cpt_enqueue_sec(qp, op, pend_q); + else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) ret = otx2_cpt_enqueue_sym(qp, op, pend_q); else ret = otx2_cpt_enqueue_sym_sessless(qp, op, diff --git a/drivers/crypto/octeontx2/otx2_ipsec_po.h b/drivers/crypto/octeontx2/otx2_ipsec_po.h index 5d9f1393e2..d127f310b7 100644 --- a/drivers/crypto/octeontx2/otx2_ipsec_po.h +++ b/drivers/crypto/octeontx2/otx2_ipsec_po.h @@ -86,6 +86,12 @@ enum { OTX2_IPSEC_PO_SA_ENCAP_UDP = 1, }; +struct otx2_ipsec_po_out_hdr { + uint32_t ip_id; + uint32_t seq; + uint8_t iv[16]; +}; + union bit_perfect_iv { uint8_t aes_iv[16]; uint8_t des_iv[8]; diff --git a/drivers/crypto/octeontx2/otx2_ipsec_po_ops.h b/drivers/crypto/octeontx2/otx2_ipsec_po_ops.h new file mode 100644 index 0000000000..1bfcbbe75b --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_ipsec_po_ops.h @@ -0,0 +1,179 @@ + +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#ifndef __OTX2_IPSEC_PO_OPS_H__ +#define __OTX2_IPSEC_PO_OPS_H__ + +#include +#include + +#include "otx2_cryptodev.h" +#include "otx2_security.h" + +static __rte_always_inline int32_t +otx2_ipsec_po_out_rlen_get(struct otx2_sec_session_ipsec_lp *sess, + uint32_t plen) +{ + uint32_t enc_payload_len; + + enc_payload_len = RTE_ALIGN_CEIL(plen + sess->roundup_len, + sess->roundup_byte); + + return sess->partial_len + enc_payload_len; +} + +static __rte_always_inline struct cpt_request_info * +alloc_request_struct(char *maddr, void *cop, int mdata_len) +{ + struct cpt_request_info *req; + struct cpt_meta_info *meta; + uint8_t *resp_addr; + uintptr_t *op; + + meta = (void *)RTE_PTR_ALIGN((uint8_t *)maddr, 16); + + op = meta->deq_op_info; + req = &meta->cpt_req; + resp_addr = (uint8_t *)&meta->cpt_res; + + req->completion_addr = (uint64_t *)((uint8_t *)resp_addr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = rte_mem_virt2iova(resp_addr); + req->op = op; + + op[0] = (uintptr_t)((uint64_t)meta | 1ull); + op[1] = (uintptr_t)cop; + op[2] = (uintptr_t)req; + op[3] = mdata_len; + + return req; +} + +static __rte_always_inline int +process_outb_sa(struct rte_crypto_op *cop, + struct otx2_sec_session_ipsec_lp *sess, + struct cpt_qp_meta_info *m_info, void **prep_req) +{ + uint32_t dlen, rlen, extend_head, extend_tail; + struct rte_crypto_sym_op *sym_op = cop->sym; + struct rte_mbuf *m_src = sym_op->m_src; + struct otx2_ipsec_po_sa_ctl *ctl_wrd; + struct cpt_request_info *req = NULL; + struct otx2_ipsec_po_out_hdr *hdr; + struct otx2_ipsec_po_out_sa *sa; + int hdr_len, mdata_len, ret = 0; + vq_cmd_word0_t word0; + char *mdata, *data; + + sa = &sess->out_sa; + ctl_wrd = &sa->ctl; + hdr_len = sizeof(*hdr); + + dlen = rte_pktmbuf_pkt_len(m_src) + hdr_len; + rlen = otx2_ipsec_po_out_rlen_get(sess, dlen - hdr_len); + + extend_head = hdr_len + RTE_ETHER_HDR_LEN; + extend_tail = rlen - dlen; + mdata_len = m_info->lb_mlen + 8; + + mdata = rte_pktmbuf_append(m_src, extend_tail + mdata_len); + if (unlikely(mdata == NULL)) { + otx2_err("Not enough tail room\n"); + ret = -ENOMEM; + goto exit; + } + + mdata += extend_tail; /* mdata follows encrypted data */ + req = alloc_request_struct(mdata, (void *)cop, mdata_len); + + data = rte_pktmbuf_prepend(m_src, extend_head); + if (unlikely(data == NULL)) { + otx2_err("Not enough head room\n"); + ret = -ENOMEM; + goto exit; + } + + /* + * Move the Ethernet header, to insert otx2_ipsec_po_out_hdr prior + * to the IP header + */ + memcpy(data, data + hdr_len, RTE_ETHER_HDR_LEN); + + hdr = (struct otx2_ipsec_po_out_hdr *)rte_pktmbuf_adj(m_src, + RTE_ETHER_HDR_LEN); + + if (ctl_wrd->enc_type == OTX2_IPSEC_FP_SA_ENC_AES_GCM) { + memcpy(&hdr->iv[0], &sa->iv.misc.nonce, 4); + memcpy(&hdr->iv[4], rte_crypto_op_ctod_offset(cop, uint8_t *, + sess->iv_offset), sess->iv_length); + } else if (ctl_wrd->auth_type == OTX2_IPSEC_FP_SA_ENC_AES_CBC) { + memcpy(&hdr->iv[0], rte_crypto_op_ctod_offset(cop, uint8_t *, + sess->iv_offset), sess->iv_length); + } + + /* Prepare CPT instruction */ + word0.s.opcode = sess->ucmd_opcode; + word0.s.param1 = sess->ucmd_param1; + word0.s.param2 = sess->ucmd_param2; + word0.s.dlen = dlen; + + req->ist.ei0 = word0.u64; + req->ist.ei1 = rte_pktmbuf_iova(m_src); + req->ist.ei2 = req->ist.ei1; + req->ist.ei3 = sess->ucmd_w3; + + hdr->seq = rte_cpu_to_be_32(sess->seq_lo); + hdr->ip_id = rte_cpu_to_be_32(sess->ip_id); + + sess->ip_id++; + sess->esn++; + +exit: + *prep_req = req; + + return ret; +} + +static __rte_always_inline int +process_inb_sa(struct rte_crypto_op *cop, + struct otx2_sec_session_ipsec_lp *sess, + struct cpt_qp_meta_info *m_info, void **prep_req) +{ + struct rte_crypto_sym_op *sym_op = cop->sym; + struct rte_mbuf *m_src = sym_op->m_src; + struct cpt_request_info *req = NULL; + int mdata_len, ret = 0; + vq_cmd_word0_t word0; + uint32_t dlen; + char *mdata; + + dlen = rte_pktmbuf_pkt_len(m_src); + mdata_len = m_info->lb_mlen + 8; + + mdata = rte_pktmbuf_append(m_src, mdata_len); + if (unlikely(mdata == NULL)) { + otx2_err("Not enough tail room\n"); + ret = -ENOMEM; + goto exit; + } + + req = alloc_request_struct(mdata, (void *)cop, mdata_len); + + /* Prepare CPT instruction */ + word0.s.opcode = sess->ucmd_opcode; + word0.s.param1 = sess->ucmd_param1; + word0.s.param2 = sess->ucmd_param2; + word0.s.dlen = dlen; + + req->ist.ei0 = word0.u64; + req->ist.ei1 = rte_pktmbuf_iova(m_src); + req->ist.ei2 = req->ist.ei1; + req->ist.ei3 = sess->ucmd_w3; + +exit: + *prep_req = req; + return ret; +} +#endif /* __OTX2_IPSEC_PO_OPS_H__ */