From patchwork Wed Oct 20 11:27:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 102423 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7DD85A0C43; Wed, 20 Oct 2021 13:28:38 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0F778411FA; Wed, 20 Oct 2021 13:28:38 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 62F3240142 for ; Wed, 20 Oct 2021 13:28:35 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19K8K7tE021014; Wed, 20 Oct 2021 04:28:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=gxmrBeKVkyjnE3mcKazp1VxeCORA3pkxjPBBTRJTnB8=; b=kMSOy6KwfPRXtuwLVJiUWGN/2FGNkOZPpkEwHzNj/tbiZ+KLBP0TpBh9xMcWlexHYaEA SZdHQhZQ9dOcFk1HvPpGKTZ+5TyUlx95OGyyKxCMy2CnckTpzyS5x04cAsE8cARkzDXN AvpqwIIHOo4pust3DzhNJ3gza7V0gxEWbnQvmFpJCIcwaFr0JY7vgfx73ZLUY0mTAYEt ssWKnBSYe3JaPZjj+QWa8Y36YFazgiIWD5ArlZ+T7ttrmBJra+HqdW34fMavct0IY/6L UYFQxDHNlFPBFmG0NsriDkUNXqhWLuvxeKx7y/ExSbJjCISgKUxvE7m0DxV1jB70286B KQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 3bt05g4pwg-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 04:28:29 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 04:28:28 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 20 Oct 2021 04:28:28 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id A2A773F707F; Wed, 20 Oct 2021 04:28:21 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal , Rebecca Troy Date: Wed, 20 Oct 2021 16:57:49 +0530 Message-ID: <20211020112754.1270163-4-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211020112754.1270163-1-gakhil@marvell.com> References: <20211018144201.2028022-1-gakhil@marvell.com> <20211020112754.1270163-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: wrsxH6RTvXH_vFzXWJtvEjLDJXs5orst X-Proofpoint-ORIG-GUID: wrsxH6RTvXH_vFzXWJtvEjLDJXs5orst X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-20_04,2021-10-20_02,2020-04-07_01 Subject: [dpdk-dev] [PATCH v4 3/8] cryptodev: move inline APIs into separate structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Move fastpath inline function pointers from rte_cryptodev into a separate structure accessed via a flat array. The intension is to make rte_cryptodev and related structures private to avoid future API/ABI breakages. Signed-off-by: Akhil Goyal Tested-by: Rebecca Troy Acked-by: Fan Zhang Acked-by: Konstantin Ananyev --- lib/cryptodev/cryptodev_pmd.c | 53 +++++++++++++++++++++++++++++- lib/cryptodev/cryptodev_pmd.h | 11 +++++++ lib/cryptodev/rte_cryptodev.c | 19 +++++++++++ lib/cryptodev/rte_cryptodev_core.h | 29 ++++++++++++++++ lib/cryptodev/version.map | 5 +++ 5 files changed, 116 insertions(+), 1 deletion(-) diff --git a/lib/cryptodev/cryptodev_pmd.c b/lib/cryptodev/cryptodev_pmd.c index 44a70ecb35..fd74543682 100644 --- a/lib/cryptodev/cryptodev_pmd.c +++ b/lib/cryptodev/cryptodev_pmd.c @@ -3,7 +3,7 @@ */ #include - +#include #include #include @@ -160,3 +160,54 @@ rte_cryptodev_pmd_destroy(struct rte_cryptodev *cryptodev) return 0; } + +static uint16_t +dummy_crypto_enqueue_burst(__rte_unused void *qp, + __rte_unused struct rte_crypto_op **ops, + __rte_unused uint16_t nb_ops) +{ + CDEV_LOG_ERR( + "crypto enqueue burst requested for unconfigured device"); + rte_errno = ENOTSUP; + return 0; +} + +static uint16_t +dummy_crypto_dequeue_burst(__rte_unused void *qp, + __rte_unused struct rte_crypto_op **ops, + __rte_unused uint16_t nb_ops) +{ + CDEV_LOG_ERR( + "crypto dequeue burst requested for unconfigured device"); + rte_errno = ENOTSUP; + return 0; +} + +void +cryptodev_fp_ops_reset(struct rte_crypto_fp_ops *fp_ops) +{ + static struct rte_cryptodev_cb_rcu dummy_cb[RTE_MAX_QUEUES_PER_PORT]; + static void *dummy_data[RTE_MAX_QUEUES_PER_PORT]; + static const struct rte_crypto_fp_ops dummy = { + .enqueue_burst = dummy_crypto_enqueue_burst, + .dequeue_burst = dummy_crypto_dequeue_burst, + .qp = { + .data = dummy_data, + .enq_cb = dummy_cb, + .deq_cb = dummy_cb, + }, + }; + + *fp_ops = dummy; +} + +void +cryptodev_fp_ops_set(struct rte_crypto_fp_ops *fp_ops, + const struct rte_cryptodev *dev) +{ + fp_ops->enqueue_burst = dev->enqueue_burst; + fp_ops->dequeue_burst = dev->dequeue_burst; + fp_ops->qp.data = dev->data->queue_pairs; + fp_ops->qp.enq_cb = dev->enq_cbs; + fp_ops->qp.deq_cb = dev->deq_cbs; +} diff --git a/lib/cryptodev/cryptodev_pmd.h b/lib/cryptodev/cryptodev_pmd.h index 36606dd10b..a71edbb991 100644 --- a/lib/cryptodev/cryptodev_pmd.h +++ b/lib/cryptodev/cryptodev_pmd.h @@ -516,6 +516,17 @@ RTE_INIT(init_ ##driver_id)\ driver_id = rte_cryptodev_allocate_driver(&crypto_drv, &(drv));\ } +/* Reset crypto device fastpath APIs to dummy values. */ +__rte_internal +void +cryptodev_fp_ops_reset(struct rte_crypto_fp_ops *fp_ops); + +/* Setup crypto device fastpath APIs. */ +__rte_internal +void +cryptodev_fp_ops_set(struct rte_crypto_fp_ops *fp_ops, + const struct rte_cryptodev *dev); + static inline void * get_sym_session_private_data(const struct rte_cryptodev_sym_session *sess, uint8_t driver_id) { diff --git a/lib/cryptodev/rte_cryptodev.c b/lib/cryptodev/rte_cryptodev.c index eb86e629aa..305e013ebb 100644 --- a/lib/cryptodev/rte_cryptodev.c +++ b/lib/cryptodev/rte_cryptodev.c @@ -53,6 +53,9 @@ static struct rte_cryptodev_global cryptodev_globals = { .nb_devs = 0 }; +/* Public fastpath APIs. */ +struct rte_crypto_fp_ops rte_crypto_fp_ops[RTE_CRYPTO_MAX_DEVS]; + /* spinlock for crypto device callbacks */ static rte_spinlock_t rte_cryptodev_cb_lock = RTE_SPINLOCK_INITIALIZER; @@ -917,6 +920,8 @@ rte_cryptodev_pmd_release_device(struct rte_cryptodev *cryptodev) dev_id = cryptodev->data->dev_id; + cryptodev_fp_ops_reset(rte_crypto_fp_ops + dev_id); + /* Close device only if device operations have been set */ if (cryptodev->dev_ops) { ret = rte_cryptodev_close(dev_id); @@ -1080,6 +1085,9 @@ rte_cryptodev_start(uint8_t dev_id) } diag = (*dev->dev_ops->dev_start)(dev); + /* expose selection of PMD fast-path functions */ + cryptodev_fp_ops_set(rte_crypto_fp_ops + dev_id, dev); + rte_cryptodev_trace_start(dev_id, diag); if (diag == 0) dev->data->dev_started = 1; @@ -1109,6 +1117,9 @@ rte_cryptodev_stop(uint8_t dev_id) return; } + /* point fast-path functions to dummy ones */ + cryptodev_fp_ops_reset(rte_crypto_fp_ops + dev_id); + (*dev->dev_ops->dev_stop)(dev); rte_cryptodev_trace_stop(dev_id); dev->data->dev_started = 0; @@ -2411,3 +2422,11 @@ rte_cryptodev_allocate_driver(struct cryptodev_driver *crypto_drv, return nb_drivers++; } + +RTE_INIT(cryptodev_init_fp_ops) +{ + uint32_t i; + + for (i = 0; i != RTE_DIM(rte_crypto_fp_ops); i++) + cryptodev_fp_ops_reset(rte_crypto_fp_ops + i); +} diff --git a/lib/cryptodev/rte_cryptodev_core.h b/lib/cryptodev/rte_cryptodev_core.h index 1633e55889..2bb9a228c1 100644 --- a/lib/cryptodev/rte_cryptodev_core.h +++ b/lib/cryptodev/rte_cryptodev_core.h @@ -25,6 +25,35 @@ typedef uint16_t (*enqueue_pkt_burst_t)(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops); /**< Enqueue packets for processing on queue pair of a device. */ +/** + * @internal + * Structure used to hold opaque pointers to internal ethdev Rx/Tx + * queues data. + * The main purpose to expose these pointers at all - allow compiler + * to fetch this data for fast-path cryptodev inline functions in advance. + */ +struct rte_cryptodev_qpdata { + /** points to array of internal queue pair data pointers. */ + void **data; + /** points to array of enqueue callback data pointers */ + struct rte_cryptodev_cb_rcu *enq_cb; + /** points to array of dequeue callback data pointers */ + struct rte_cryptodev_cb_rcu *deq_cb; +}; + +struct rte_crypto_fp_ops { + /** PMD enqueue burst function. */ + enqueue_pkt_burst_t enqueue_burst; + /** PMD dequeue burst function. */ + dequeue_pkt_burst_t dequeue_burst; + /** Internal queue pair data pointers. */ + struct rte_cryptodev_qpdata qp; + /** Reserved for future ops. */ + uintptr_t reserved[3]; +} __rte_cache_aligned; + +extern struct rte_crypto_fp_ops rte_crypto_fp_ops[RTE_CRYPTO_MAX_DEVS]; + /** * @internal * The data part, with no function pointers, associated with each device. diff --git a/lib/cryptodev/version.map b/lib/cryptodev/version.map index 43cf937e40..ed62ced221 100644 --- a/lib/cryptodev/version.map +++ b/lib/cryptodev/version.map @@ -45,6 +45,9 @@ DPDK_22 { rte_cryptodev_sym_session_init; rte_cryptodevs; + #added in 21.11 + rte_crypto_fp_ops; + local: *; }; @@ -109,6 +112,8 @@ EXPERIMENTAL { INTERNAL { global: + cryptodev_fp_ops_reset; + cryptodev_fp_ops_set; rte_cryptodev_allocate_driver; rte_cryptodev_pmd_allocate; rte_cryptodev_pmd_callback_process;