From patchwork Tue Dec 28 09:10:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gagandeep Singh X-Patchwork-Id: 105455 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BEADCA034F; Tue, 28 Dec 2021 10:11:42 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DD9FB4114A; Tue, 28 Dec 2021 10:11:33 +0100 (CET) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10069.outbound.protection.outlook.com [40.107.1.69]) by mails.dpdk.org (Postfix) with ESMTP id 9BE374113D for ; Tue, 28 Dec 2021 10:11:30 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QcRH2S1BX0CZIREQrSl+njwvhYkbCMxUUMBIYa+SrPm8u5KudzeXHbet27i8m5nXDwMgKb/ZsTuLXMVSIQzhpN2ooOqZoEK7Kt214Yy4Jep4i1YNTJcgswR9FzFQVGSJiTBBU38Wxn1nMaTkzWbpgB03FjVikIkNm6V1MJbEmrJkRXp8U+IoRvE5KcxNVVI+2EHeBP4tAqf1InWywmGHGgLjt0bOR9eSu5YZBnTcwf5WCvetfv/3hxmDHoZmDUg3W9WMItuEZwzYXiqL8PGDHImlhS9nQfG3TiEE/rsMXcr+ePtUJZqJuqO7x3HFHojzcZKbygUr3dwssXQVLn192g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lDjWAy47x5cY8icoRld4jLPDQey9Vg9RQMCZ5uSIl+M=; b=LDRmATfNOHiaCtkiJiiAaFslF4cMG5yRwIAAR+alJUqD/AAVdHKgfs913slcrudfjxFp/NXzMmUx9TecSINrmok8+5CYobSWzN9UBsaZN32880u5yB0LVm+sGxOxAKtxfjKqsBqkxcmIRmWMNS+4Cbkajeqsz03MB5wwinPZ69/YkTihzl79rrS2lAzn0K/VuTLSsm/XKd0Wryz0CTTH50tB8Pm6pM15zKDOwALlPjwmIAibJvG5ngzruB3edNfXDZ30BHh2zijEWB5b1QhukfUR5DhZf9whIVl4UG3D7XbOFSUSVM9Tko4F3p2UQFtGYU+wujm9WeUFtx9lrJ1uGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lDjWAy47x5cY8icoRld4jLPDQey9Vg9RQMCZ5uSIl+M=; b=fxNN6/+h2pIl9r1xBTok9baDulmcfIb8Ly3D5tY34b9z0CoyFff3ZBjGSagcZl2drEaZwjnW3OuO7ebo1Xhm25h69bBRPcjUHi8ugU4yBoUkivWnNJjy1l8QylQ+giYLa7o9GnviLdeA8V5ylPnM5ubR+mV9TiNLEeLaILUm0Wc= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AS8PR04MB8198.eurprd04.prod.outlook.com (2603:10a6:20b:3b0::14) by AM6PR04MB6135.eurprd04.prod.outlook.com (2603:10a6:20b:bf::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4823.21; Tue, 28 Dec 2021 09:11:29 +0000 Received: from AS8PR04MB8198.eurprd04.prod.outlook.com ([fe80::4c23:c171:682e:9085]) by AS8PR04MB8198.eurprd04.prod.outlook.com ([fe80::4c23:c171:682e:9085%4]) with mapi id 15.20.4823.023; Tue, 28 Dec 2021 09:11:29 +0000 From: Gagandeep Singh To: gakhil@marvell.com, dev@dpdk.org Cc: Nipun Gupta Subject: [PATCH v2 3/8] crypto/dpaa2_sec: ordered queue support Date: Tue, 28 Dec 2021 14:40:53 +0530 Message-Id: <20211228091058.3620177-4-g.singh@nxp.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211228091058.3620177-1-g.singh@nxp.com> References: <20211220102710.3083370-1-g.singh@nxp.com> <20211228091058.3620177-1-g.singh@nxp.com> X-ClientProxiedBy: SG2PR06CA0118.apcprd06.prod.outlook.com (2603:1096:1:1d::20) To AS8PR04MB8198.eurprd04.prod.outlook.com (2603:10a6:20b:3b0::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2be1b502-1288-4c9b-0587-08d9c9e20858 X-MS-TrafficTypeDiagnostic: AM6PR04MB6135:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WgP7mzxbt97L6Ac1y214zuCfY7gxJL7MPZKBlzm/M69TOff/purYi8jNRLzlABG/nkoqPrcuoQmCpXyN7MQj/VQdyvi8ARc+u556ha2H0hUJHJy7ab/aZQjE/oRf5FVDAVuVPhifRQsfLONpskbYZP8YL7FT9YrJ4qw1qjpeyXczfge8BUmfuH4EEnWTS7fn7NQOSZYoDd0Y1D86K80RZDMfRn74LDbW1GliBGo+BpqydxJnZgVbxWtUvehratJNIalSfrjPpY228WrWb40rJXm5svwBA3CPEWFiLRJB5YWlX34BUuSj9KY5wQ+uEp6IDbiiUNw2YGPuMTQVNutJVszBLSmLOaEs+K8HJ2dttL47898c7wT/XzZNUIyXWmlzq1wydzxD/UhE7ueFGxEeKPtX2CXjmAq8K/0sS8Ky+59IirwF40uryfoYm2r2iJ1iedEJut15DfvfQSa0buVgrNoNADzRq21Q7SG/PqBgJ4/muBxe1XFnd8/xrsiiKih8J7FTGIDlTcuTgYHu1Lz9BFbcb9L0BT27CU9oUC3w4p0kXLTZwCT8Z8wh1u+SQx+wNydOWVPiedAD8tdsrx1CMqokbGo54R128Qm/CrvFrGRKpV0UuwSRWuAjzZYlg2NwtaynAtfyC0ZwAdaE9XSGLjzvU2I/zaVEb2puh3gTWSQAl0xJoj0iGgslifR/H2zIEFwnV4wm8XGFKtGAR/MEoQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AS8PR04MB8198.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(316002)(8676002)(2616005)(6486002)(4326008)(6506007)(30864003)(66556008)(66476007)(6512007)(6666004)(55236004)(38100700002)(38350700002)(36756003)(52116002)(83380400001)(2906002)(1076003)(66946007)(86362001)(26005)(5660300002)(8936002)(186003)(508600001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: m95oT5OHu9S2KiGaLHGryCXnGwfQVVWEVH5MxSdqbxJeRDVrWdeTtigdLeW0KpKwcJGDHEgBFhIWjL5KgVyJaDJZgt4Q+m3vJ+oFVu4Wi1YmuNrTzg+JZRG87kwRjrN56lcDcM5Cf57IbsfcyRSqQ/ueKdCkRYVxUwbtGrxHvrfVBn3N/UXFIAJdf8UYCG1VMXagYR0iwOfYLWm+0H2SHx9FgHTPlRJ9GebWCQXbRnGO4eWQ7K+VG1rH7OUiO3ZkFFvhRRSoU9pvNGZHtgRU8VRQ6JkOZbA20GTGqUSoXvS4HEzAzKp/RWmtY+qEcp44abO8Jnyi7NTwGnoT+98APmwJ5DmhCwURwk/OZw808sgKH4EBo5cZm3s2rZff2KCR7siHFMuHbuzAgYYKFqDj12S3yBt1on0AhqJGnymXsLY05e9YyyqbV5fS+zls/ZKIC54I9tk/s+XXoWUDOS1aKF6FSL1BbLPRb8bZTvl/ZaMUlOM+k7Dv5A+Ws1nv+u/No0BQu0rrVtxi+/3T8OiUTVLc5EBtPMH/EQoqZCxmb53LUeJ1U+Ow3pZEjC00G5kuaijZoJuSzLxh2Q41mGxs2iP9hW84t36SzeNwbjU7bOessMgXTO27T/LV3/GzKdyVQDSt4d6IylxP44Sf7aIS7WmkWDWR/ap9c7TyxjZMmddYK2kc7suroOY4+tLxWd3xS0IVXdpdpM2kb2p8nsztgsM53MRA3nzWVTO3MLx1Oe8Xopkkgs5S3Uz829TBT4jf9/vJeLGG1wBu9NxuyLOWDzFnTBbNhOC+GxF7TOLRNm+DGQK9EBRYNQm770Hpn8/vsGcsL9h4XLJiLwvrz5iBUrurE5UhPf5YLBcPV2oIx3rUEcupPfxwLXhtBsms+tyrGLNQ/2jAeOnIyRvrNxdVE4c/jtO/bSY6ZsooS6bPwDHo2hQ1UhEACx6skDGW3pcJkx8t5zPMyCph0aZp2ELxjMmvzC5/QZ5RFOwcw6uFX1YKUoCKZZ/pup+ec61xE9nvUV74cedexBXpbfx4Ld2MFrpisiqLwPYL81hkaZVxJAOrQ5OcWYAJzO+V4wnRbgVgOWFoDXiXf4RVEG87Qz1Bli2fbJO549DNZpTnJ+4xgotqrJacl55XVYWT2u3l4pz4mJd4gq6z8dcMN2dcnE9KrwKvZTUyUjJfMJlTHC/Y4BszsJy4nnrdeiGdCMV1L8s09C/JRpg/05NKppfj20iujpGbpBJV+pf0Bz0Jc4c9iSriy8HKMeXLo3P9YuYbRHUNf6ds6sgmP8gN2jgt+eO6vHRbkSk9T7Hi+AKGExSPWoF4lNHRlsr1RWgpY+OZFZYeOCly238HwYoqHuRd8dxxL9kll9LIG+eZzPbT1lbmiyuEdr9gpADIL87pUSgKq+K726TGgAmtynlTsCka7n65gDASbHt/Z+GiQg/0Aln0G6F1ON1sjT87rVlB5v8Vc1Wn7vEQKBdNMQGO2lItaUC08HUBecU0tdCMzpkaF2C8yaKSCgCh/WTAW29Qho4meueVNMFUbk5y3EjbrLDuzvSVUgE6IxgHAv1ENMRybB+vxAo= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2be1b502-1288-4c9b-0587-08d9c9e20858 X-MS-Exchange-CrossTenant-AuthSource: AS8PR04MB8198.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Dec 2021 09:11:29.3465 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8rJwtLC1Myo/FNxOozJy+aY/zw1+82iJhpsDjQux5fTx7IPdow3Vg7epBPZ6N6wX X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB6135 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Nipun Gupta This patch supports ordered queue for DPAA2 platform. Signed-off-by: Nipun Gupta --- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 255 +++++++++++++++++++- drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 8 +- drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h | 14 +- 3 files changed, 263 insertions(+), 14 deletions(-) diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index 1e6b3e548a..a9fda67ac3 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -1466,14 +1466,14 @@ dpaa2_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops, for (loop = 0; loop < frames_to_send; loop++) { if (*dpaa2_seqn((*ops)->sym->m_src)) { - uint8_t dqrr_index = - *dpaa2_seqn((*ops)->sym->m_src) - 1; - - flags[loop] = QBMAN_ENQUEUE_FLAG_DCA | dqrr_index; - DPAA2_PER_LCORE_DQRR_SIZE--; - DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << dqrr_index); - *dpaa2_seqn((*ops)->sym->m_src) = - DPAA2_INVALID_MBUF_SEQN; + if (*dpaa2_seqn((*ops)->sym->m_src) & QBMAN_ENQUEUE_FLAG_DCA) { + DPAA2_PER_LCORE_DQRR_SIZE--; + DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << + *dpaa2_seqn((*ops)->sym->m_src) & + QBMAN_EQCR_DCA_IDXMASK); + } + flags[loop] = *dpaa2_seqn((*ops)->sym->m_src); + *dpaa2_seqn((*ops)->sym->m_src) = DPAA2_INVALID_MBUF_SEQN; } /*Clear the unused FD fields before sending*/ @@ -1621,6 +1621,169 @@ sec_fd_to_mbuf(const struct qbman_fd *fd) return op; } +static void +dpaa2_sec_free_eqresp_buf(uint16_t eqresp_ci) +{ + struct dpaa2_dpio_dev *dpio_dev = DPAA2_PER_LCORE_DPIO; + struct rte_crypto_op *op; + struct qbman_fd *fd; + + fd = qbman_result_eqresp_fd(&dpio_dev->eqresp[eqresp_ci]); + op = sec_fd_to_mbuf(fd); + /* Instead of freeing, enqueue it to the sec tx queue (sec->core) + * after setting an error in FD. But this will have performance impact. + */ + rte_pktmbuf_free(op->sym->m_src); +} + +static void +dpaa2_sec_set_enqueue_descriptor(struct dpaa2_queue *dpaa2_q, + struct rte_mbuf *m, + struct qbman_eq_desc *eqdesc) +{ + struct dpaa2_dpio_dev *dpio_dev = DPAA2_PER_LCORE_DPIO; + struct eqresp_metadata *eqresp_meta; + struct dpaa2_sec_dev_private *priv = dpaa2_q->crypto_data->dev_private; + uint16_t orpid, seqnum; + uint8_t dq_idx; + + if (*dpaa2_seqn(m) & DPAA2_ENQUEUE_FLAG_ORP) { + orpid = (*dpaa2_seqn(m) & DPAA2_EQCR_OPRID_MASK) >> + DPAA2_EQCR_OPRID_SHIFT; + seqnum = (*dpaa2_seqn(m) & DPAA2_EQCR_SEQNUM_MASK) >> + DPAA2_EQCR_SEQNUM_SHIFT; + + + if (!priv->en_loose_ordered) { + qbman_eq_desc_set_orp(eqdesc, 1, orpid, seqnum, 0); + qbman_eq_desc_set_response(eqdesc, (uint64_t) + DPAA2_VADDR_TO_IOVA(&dpio_dev->eqresp[ + dpio_dev->eqresp_pi]), 1); + qbman_eq_desc_set_token(eqdesc, 1); + + eqresp_meta = &dpio_dev->eqresp_meta[dpio_dev->eqresp_pi]; + eqresp_meta->dpaa2_q = dpaa2_q; + eqresp_meta->mp = m->pool; + + dpio_dev->eqresp_pi + 1 < MAX_EQ_RESP_ENTRIES ? + dpio_dev->eqresp_pi++ : (dpio_dev->eqresp_pi = 0); + } else { + qbman_eq_desc_set_orp(eqdesc, 0, orpid, seqnum, 0); + } + } else { + dq_idx = *dpaa2_seqn(m) - 1; + qbman_eq_desc_set_dca(eqdesc, 1, dq_idx, 0); + DPAA2_PER_LCORE_DQRR_SIZE--; + DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << dq_idx); + } + *dpaa2_seqn(m) = DPAA2_INVALID_MBUF_SEQN; +} + + +static uint16_t +dpaa2_sec_enqueue_burst_ordered(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function to transmit the frames to given device and VQ*/ + uint32_t loop; + int32_t ret; + struct qbman_fd fd_arr[MAX_TX_RING_SLOTS]; + uint32_t frames_to_send, num_free_eq_desc, retry_count; + struct qbman_eq_desc eqdesc[MAX_TX_RING_SLOTS]; + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp; + struct qbman_swp *swp; + uint16_t num_tx = 0; + /*todo - need to support multiple buffer pools */ + uint16_t bpid; + struct rte_mempool *mb_pool; + struct dpaa2_sec_dev_private *priv = + dpaa2_qp->tx_vq.crypto_data->dev_private; + + if (unlikely(nb_ops == 0)) + return 0; + + if (ops[0]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { + DPAA2_SEC_ERR("sessionless crypto op not supported"); + return 0; + } + + if (!DPAA2_PER_LCORE_DPIO) { + ret = dpaa2_affine_qbman_swp(); + if (ret) { + DPAA2_SEC_ERR("Failure in affining portal"); + return 0; + } + } + swp = DPAA2_PER_LCORE_PORTAL; + + while (nb_ops) { + frames_to_send = (nb_ops > dpaa2_eqcr_size) ? + dpaa2_eqcr_size : nb_ops; + + if (!priv->en_loose_ordered) { + if (*dpaa2_seqn((*ops)->sym->m_src)) { + num_free_eq_desc = dpaa2_free_eq_descriptors(); + if (num_free_eq_desc < frames_to_send) + frames_to_send = num_free_eq_desc; + } + } + + for (loop = 0; loop < frames_to_send; loop++) { + /*Prepare enqueue descriptor*/ + qbman_eq_desc_clear(&eqdesc[loop]); + qbman_eq_desc_set_fq(&eqdesc[loop], dpaa2_qp->tx_vq.fqid); + + if (*dpaa2_seqn((*ops)->sym->m_src)) + dpaa2_sec_set_enqueue_descriptor( + &dpaa2_qp->tx_vq, + (*ops)->sym->m_src, + &eqdesc[loop]); + else + qbman_eq_desc_set_no_orp(&eqdesc[loop], + DPAA2_EQ_RESP_ERR_FQ); + + /*Clear the unused FD fields before sending*/ + memset(&fd_arr[loop], 0, sizeof(struct qbman_fd)); + mb_pool = (*ops)->sym->m_src->pool; + bpid = mempool_to_bpid(mb_pool); + ret = build_sec_fd(*ops, &fd_arr[loop], bpid); + if (ret) { + DPAA2_SEC_ERR("error: Improper packet contents" + " for crypto operation"); + goto skip_tx; + } + ops++; + } + + loop = 0; + retry_count = 0; + while (loop < frames_to_send) { + ret = qbman_swp_enqueue_multiple_desc(swp, + &eqdesc[loop], &fd_arr[loop], + frames_to_send - loop); + if (unlikely(ret < 0)) { + retry_count++; + if (retry_count > DPAA2_MAX_TX_RETRY_COUNT) { + num_tx += loop; + nb_ops -= loop; + goto skip_tx; + } + } else { + loop += ret; + retry_count = 0; + } + } + + num_tx += loop; + nb_ops -= loop; + } + +skip_tx: + dpaa2_qp->tx_vq.tx_pkts += num_tx; + dpaa2_qp->tx_vq.err_pkts += nb_ops; + return num_tx; +} + static uint16_t dpaa2_sec_dequeue_burst(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) @@ -3527,6 +3690,10 @@ dpaa2_sec_dev_start(struct rte_cryptodev *dev) PMD_INIT_FUNC_TRACE(); + /* Change the tx burst function if ordered queues are used */ + if (priv->en_ordered) + dev->enqueue_burst = dpaa2_sec_enqueue_burst_ordered; + memset(&attr, 0, sizeof(struct dpseci_attr)); ret = dpseci_enable(dpseci, CMD_PRI_LOW, priv->token); @@ -3739,12 +3906,46 @@ dpaa2_sec_process_atomic_event(struct qbman_swp *swp __rte_unused, ev->event_ptr = sec_fd_to_mbuf(fd); dqrr_index = qbman_get_dqrr_idx(dq); - *dpaa2_seqn(crypto_op->sym->m_src) = dqrr_index + 1; + *dpaa2_seqn(crypto_op->sym->m_src) = QBMAN_ENQUEUE_FLAG_DCA | dqrr_index; DPAA2_PER_LCORE_DQRR_SIZE++; DPAA2_PER_LCORE_DQRR_HELD |= 1 << dqrr_index; DPAA2_PER_LCORE_DQRR_MBUF(dqrr_index) = crypto_op->sym->m_src; } +static void __attribute__((hot)) +dpaa2_sec_process_ordered_event(struct qbman_swp *swp, + const struct qbman_fd *fd, + const struct qbman_result *dq, + struct dpaa2_queue *rxq, + struct rte_event *ev) +{ + struct rte_crypto_op *crypto_op = (struct rte_crypto_op *)ev->event_ptr; + + /* Prefetching mbuf */ + rte_prefetch0((void *)(size_t)(DPAA2_GET_FD_ADDR(fd)- + rte_dpaa2_bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size)); + + /* Prefetching ipsec crypto_op stored in priv data of mbuf */ + rte_prefetch0((void *)(size_t)(DPAA2_GET_FD_ADDR(fd)-64)); + + ev->flow_id = rxq->ev.flow_id; + ev->sub_event_type = rxq->ev.sub_event_type; + ev->event_type = RTE_EVENT_TYPE_CRYPTODEV; + ev->op = RTE_EVENT_OP_NEW; + ev->sched_type = rxq->ev.sched_type; + ev->queue_id = rxq->ev.queue_id; + ev->priority = rxq->ev.priority; + ev->event_ptr = sec_fd_to_mbuf(fd); + + *dpaa2_seqn(crypto_op->sym->m_src) = DPAA2_ENQUEUE_FLAG_ORP; + *dpaa2_seqn(crypto_op->sym->m_src) |= qbman_result_DQ_odpid(dq) << + DPAA2_EQCR_OPRID_SHIFT; + *dpaa2_seqn(crypto_op->sym->m_src) |= qbman_result_DQ_seqnum(dq) << + DPAA2_EQCR_SEQNUM_SHIFT; + + qbman_swp_dqrr_consume(swp, dq); +} + int dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, int qp_id, @@ -3762,6 +3963,8 @@ dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, qp->rx_vq.cb = dpaa2_sec_process_parallel_event; else if (event->sched_type == RTE_SCHED_TYPE_ATOMIC) qp->rx_vq.cb = dpaa2_sec_process_atomic_event; + else if (event->sched_type == RTE_SCHED_TYPE_ORDERED) + qp->rx_vq.cb = dpaa2_sec_process_ordered_event; else return -EINVAL; @@ -3780,6 +3983,40 @@ dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, cfg.options |= DPSECI_QUEUE_OPT_ORDER_PRESERVATION; cfg.order_preservation_en = 1; } + + if (event->sched_type == RTE_SCHED_TYPE_ORDERED) { + struct opr_cfg ocfg; + + /* Restoration window size = 256 frames */ + ocfg.oprrws = 3; + /* Restoration window size = 512 frames for LX2 */ + if (dpaa2_svr_family == SVR_LX2160A) + ocfg.oprrws = 4; + /* Auto advance NESN window enabled */ + ocfg.oa = 1; + /* Late arrival window size disabled */ + ocfg.olws = 0; + /* ORL resource exhaustaion advance NESN disabled */ + ocfg.oeane = 0; + /* Loose ordering enabled */ + ocfg.oloe = 1; + priv->en_loose_ordered = 1; + /* Strict ordering enabled if explicitly set */ + if (getenv("DPAA2_STRICT_ORDERING_ENABLE")) { + ocfg.oloe = 0; + priv->en_loose_ordered = 0; + } + + ret = dpseci_set_opr(dpseci, CMD_PRI_LOW, priv->token, + qp_id, OPR_OPT_CREATE, &ocfg); + if (ret) { + RTE_LOG(ERR, PMD, "Error setting opr: ret: %d\n", ret); + return ret; + } + qp->tx_vq.cb_eqresp_free = dpaa2_sec_free_eqresp_buf; + priv->en_ordered = 1; + } + ret = dpseci_set_rx_queue(dpseci, CMD_PRI_LOW, priv->token, qp_id, &cfg); if (ret) { diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h index 05bd7c0736..1756d917dd 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h @@ -1,8 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause - * - * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. - * Copyright 2016,2020-2021 NXP - * + * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. + * Copyright 2016,2019-2021 NXP */ #ifndef _DPAA2_SEC_PMD_PRIVATE_H_ @@ -37,6 +35,8 @@ struct dpaa2_sec_dev_private { uint16_t token; /**< Token required by DPxxx objects */ unsigned int max_nb_queue_pairs; /**< Max number of queue pairs supported by device */ + uint8_t en_ordered; + uint8_t en_loose_ordered; }; struct dpaa2_sec_qp { diff --git a/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h b/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h index 279e8f4d4a..c295c04f24 100644 --- a/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h +++ b/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0) * * Copyright 2013-2016 Freescale Semiconductor Inc. - * Copyright 2016-2017 NXP + * Copyright 2016-2020 NXP * */ #ifndef __FSL_DPSECI_H @@ -11,6 +11,8 @@ * Contains initialization APIs and runtime control APIs for DPSECI */ +#include + struct fsl_mc_io; /** @@ -41,6 +43,16 @@ int dpseci_close(struct fsl_mc_io *mc_io, */ #define DPSECI_OPT_HAS_CG 0x000020 +/** + * Enable the Order Restoration support + */ +#define DPSECI_OPT_HAS_OPR 0x000040 + +/** + * Order Point Records are shared for the entire DPSECI + */ +#define DPSECI_OPT_OPR_SHARED 0x000080 + /** * struct dpseci_cfg - Structure representing DPSECI configuration * @options: Any combination of the following options: