From patchwork Wed Oct 20 11:27:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 102421 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EFA62A0C43; Wed, 20 Oct 2021 13:28:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DB3E5411F2; Wed, 20 Oct 2021 13:28:26 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id C8F864118F for ; Wed, 20 Oct 2021 13:28:25 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19K2OelQ015203; Wed, 20 Oct 2021 04:28:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=M5wjOj1/3E4QVfZz9taOLponqpNAAPK2ps/KEuCsEbc=; b=QjwZtssqYRv4Jz3yuRcMNiCX72MGNmsV69t5I7WHd7Botk0qMrb5zy4ZueeRr2KnkIl3 +ujgLui84+tDoE2f35ebIe5rL7t+129TYN5CiUZUtwJcOHHRzBEWk5cJmqYEe6VN9Atx GIs+fx+M4O4tVDyMIWhFYGz2zqANoSg0iAk8LoOdDuLlJ2KHZ7uRPJA1PHCDCscNeF/s 59HRmyL7EPxoKPAHc26g1xq8MqF5/SWFEi3plwwwXWbT0n5dt0oyMyUKG1V9tS/43ESV w9Uhm8Ocpm9ScftTuZgk6648eT6y4MC6uVCRiKis8mnk4d7Txg8FrjkQYFTM/5x4xiN6 rA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3btaa7a4qw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 04:28:16 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 04:28:14 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 20 Oct 2021 04:28:14 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id 4662B3F707F; Wed, 20 Oct 2021 04:28:08 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal , Rebecca Troy Date: Wed, 20 Oct 2021 16:57:47 +0530 Message-ID: <20211020112754.1270163-2-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211020112754.1270163-1-gakhil@marvell.com> References: <20211018144201.2028022-1-gakhil@marvell.com> <20211020112754.1270163-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: g1Cl7pq9IYiqFVF-zwZGCwnFbxiX2pUg X-Proofpoint-GUID: g1Cl7pq9IYiqFVF-zwZGCwnFbxiX2pUg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-20_04,2021-10-20_02,2020-04-07_01 Subject: [dpdk-dev] [PATCH v4 1/8] cryptodev: separate out internal structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" A new header file rte_cryptodev_core.h is added and all internal data structures which need not be exposed directly to application are moved to this file. These structures are mostly used by drivers, but they need to be in the public header file as they are accessed by datapath inline functions for performance reasons. Signed-off-by: Akhil Goyal Tested-by: Rebecca Troy Acked-by: Fan Zhang Acked-by: Konstantin Ananyev --- lib/cryptodev/cryptodev_pmd.h | 6 - lib/cryptodev/meson.build | 4 +- lib/cryptodev/rte_cryptodev.h | 360 ++++++++++++----------------- lib/cryptodev/rte_cryptodev_core.h | 100 ++++++++ 4 files changed, 245 insertions(+), 225 deletions(-) create mode 100644 lib/cryptodev/rte_cryptodev_core.h diff --git a/lib/cryptodev/cryptodev_pmd.h b/lib/cryptodev/cryptodev_pmd.h index 8cc9051e09..36606dd10b 100644 --- a/lib/cryptodev/cryptodev_pmd.h +++ b/lib/cryptodev/cryptodev_pmd.h @@ -92,12 +92,6 @@ __rte_internal struct rte_cryptodev * rte_cryptodev_pmd_get_named_dev(const char *name); -/** - * The pool of rte_cryptodev structures. - */ -extern struct rte_cryptodev *rte_cryptodevs; - - /** * Definitions of all functions exported by a driver through the * the generic structure of type *crypto_dev_ops* supplied in the diff --git a/lib/cryptodev/meson.build b/lib/cryptodev/meson.build index 51371c3aa2..289b66ab76 100644 --- a/lib/cryptodev/meson.build +++ b/lib/cryptodev/meson.build @@ -14,7 +14,9 @@ headers = files( 'rte_crypto_sym.h', 'rte_crypto_asym.h', ) - +indirect_headers += files( + 'rte_cryptodev_core.h', +) driver_sdk_headers += files( 'cryptodev_pmd.h', ) diff --git a/lib/cryptodev/rte_cryptodev.h b/lib/cryptodev/rte_cryptodev.h index cdd7168fba..ce0dca72be 100644 --- a/lib/cryptodev/rte_cryptodev.h +++ b/lib/cryptodev/rte_cryptodev.h @@ -867,17 +867,6 @@ rte_cryptodev_callback_unregister(uint8_t dev_id, enum rte_cryptodev_event_type event, rte_cryptodev_cb_fn cb_fn, void *cb_arg); -typedef uint16_t (*dequeue_pkt_burst_t)(void *qp, - struct rte_crypto_op **ops, uint16_t nb_ops); -/**< Dequeue processed packets from queue pair of a device. */ - -typedef uint16_t (*enqueue_pkt_burst_t)(void *qp, - struct rte_crypto_op **ops, uint16_t nb_ops); -/**< Enqueue packets for processing on queue pair of a device. */ - - - - struct rte_cryptodev_callback; /** Structure to keep track of registered callbacks */ @@ -907,216 +896,9 @@ struct rte_cryptodev_cb_rcu { /**< RCU QSBR variable per queue pair */ }; -/** The data structure associated with each crypto device. */ -struct rte_cryptodev { - dequeue_pkt_burst_t dequeue_burst; - /**< Pointer to PMD receive function. */ - enqueue_pkt_burst_t enqueue_burst; - /**< Pointer to PMD transmit function. */ - - struct rte_cryptodev_data *data; - /**< Pointer to device data */ - struct rte_cryptodev_ops *dev_ops; - /**< Functions exported by PMD */ - uint64_t feature_flags; - /**< Feature flags exposes HW/SW features for the given device */ - struct rte_device *device; - /**< Backing device */ - - uint8_t driver_id; - /**< Crypto driver identifier*/ - - struct rte_cryptodev_cb_list link_intr_cbs; - /**< User application callback for interrupts if present */ - - void *security_ctx; - /**< Context for security ops */ - - __extension__ - uint8_t attached : 1; - /**< Flag indicating the device is attached */ - - struct rte_cryptodev_cb_rcu *enq_cbs; - /**< User application callback for pre enqueue processing */ - - struct rte_cryptodev_cb_rcu *deq_cbs; - /**< User application callback for post dequeue processing */ -} __rte_cache_aligned; - void * rte_cryptodev_get_sec_ctx(uint8_t dev_id); -/** - * - * The data part, with no function pointers, associated with each device. - * - * This structure is safe to place in shared memory to be common among - * different processes in a multi-process configuration. - */ -struct rte_cryptodev_data { - uint8_t dev_id; - /**< Device ID for this instance */ - uint8_t socket_id; - /**< Socket ID where memory is allocated */ - char name[RTE_CRYPTODEV_NAME_MAX_LEN]; - /**< Unique identifier name */ - - __extension__ - uint8_t dev_started : 1; - /**< Device state: STARTED(1)/STOPPED(0) */ - - struct rte_mempool *session_pool; - /**< Session memory pool */ - void **queue_pairs; - /**< Array of pointers to queue pairs. */ - uint16_t nb_queue_pairs; - /**< Number of device queue pairs. */ - - void *dev_private; - /**< PMD-specific private data */ -} __rte_cache_aligned; - -extern struct rte_cryptodev *rte_cryptodevs; -/** - * - * Dequeue a burst of processed crypto operations from a queue on the crypto - * device. The dequeued operation are stored in *rte_crypto_op* structures - * whose pointers are supplied in the *ops* array. - * - * The rte_cryptodev_dequeue_burst() function returns the number of ops - * actually dequeued, which is the number of *rte_crypto_op* data structures - * effectively supplied into the *ops* array. - * - * A return value equal to *nb_ops* indicates that the queue contained - * at least *nb_ops* operations, and this is likely to signify that other - * processed operations remain in the devices output queue. Applications - * implementing a "retrieve as many processed operations as possible" policy - * can check this specific case and keep invoking the - * rte_cryptodev_dequeue_burst() function until a value less than - * *nb_ops* is returned. - * - * The rte_cryptodev_dequeue_burst() function does not provide any error - * notification to avoid the corresponding overhead. - * - * @param dev_id The symmetric crypto device identifier - * @param qp_id The index of the queue pair from which to - * retrieve processed packets. The value must be - * in the range [0, nb_queue_pair - 1] previously - * supplied to rte_cryptodev_configure(). - * @param ops The address of an array of pointers to - * *rte_crypto_op* structures that must be - * large enough to store *nb_ops* pointers in it. - * @param nb_ops The maximum number of operations to dequeue. - * - * @return - * - The number of operations actually dequeued, which is the number - * of pointers to *rte_crypto_op* structures effectively supplied to the - * *ops* array. - */ -static inline uint16_t -rte_cryptodev_dequeue_burst(uint8_t dev_id, uint16_t qp_id, - struct rte_crypto_op **ops, uint16_t nb_ops) -{ - struct rte_cryptodev *dev = &rte_cryptodevs[dev_id]; - - rte_cryptodev_trace_dequeue_burst(dev_id, qp_id, (void **)ops, nb_ops); - nb_ops = (*dev->dequeue_burst) - (dev->data->queue_pairs[qp_id], ops, nb_ops); -#ifdef RTE_CRYPTO_CALLBACKS - if (unlikely(dev->deq_cbs != NULL)) { - struct rte_cryptodev_cb_rcu *list; - struct rte_cryptodev_cb *cb; - - /* __ATOMIC_RELEASE memory order was used when the - * call back was inserted into the list. - * Since there is a clear dependency between loading - * cb and cb->fn/cb->next, __ATOMIC_ACQUIRE memory order is - * not required. - */ - list = &dev->deq_cbs[qp_id]; - rte_rcu_qsbr_thread_online(list->qsbr, 0); - cb = __atomic_load_n(&list->next, __ATOMIC_RELAXED); - - while (cb != NULL) { - nb_ops = cb->fn(dev_id, qp_id, ops, nb_ops, - cb->arg); - cb = cb->next; - }; - - rte_rcu_qsbr_thread_offline(list->qsbr, 0); - } -#endif - return nb_ops; -} - -/** - * Enqueue a burst of operations for processing on a crypto device. - * - * The rte_cryptodev_enqueue_burst() function is invoked to place - * crypto operations on the queue *qp_id* of the device designated by - * its *dev_id*. - * - * The *nb_ops* parameter is the number of operations to process which are - * supplied in the *ops* array of *rte_crypto_op* structures. - * - * The rte_cryptodev_enqueue_burst() function returns the number of - * operations it actually enqueued for processing. A return value equal to - * *nb_ops* means that all packets have been enqueued. - * - * @param dev_id The identifier of the device. - * @param qp_id The index of the queue pair which packets are - * to be enqueued for processing. The value - * must be in the range [0, nb_queue_pairs - 1] - * previously supplied to - * *rte_cryptodev_configure*. - * @param ops The address of an array of *nb_ops* pointers - * to *rte_crypto_op* structures which contain - * the crypto operations to be processed. - * @param nb_ops The number of operations to process. - * - * @return - * The number of operations actually enqueued on the crypto device. The return - * value can be less than the value of the *nb_ops* parameter when the - * crypto devices queue is full or if invalid parameters are specified in - * a *rte_crypto_op*. - */ -static inline uint16_t -rte_cryptodev_enqueue_burst(uint8_t dev_id, uint16_t qp_id, - struct rte_crypto_op **ops, uint16_t nb_ops) -{ - struct rte_cryptodev *dev = &rte_cryptodevs[dev_id]; - -#ifdef RTE_CRYPTO_CALLBACKS - if (unlikely(dev->enq_cbs != NULL)) { - struct rte_cryptodev_cb_rcu *list; - struct rte_cryptodev_cb *cb; - - /* __ATOMIC_RELEASE memory order was used when the - * call back was inserted into the list. - * Since there is a clear dependency between loading - * cb and cb->fn/cb->next, __ATOMIC_ACQUIRE memory order is - * not required. - */ - list = &dev->enq_cbs[qp_id]; - rte_rcu_qsbr_thread_online(list->qsbr, 0); - cb = __atomic_load_n(&list->next, __ATOMIC_RELAXED); - - while (cb != NULL) { - nb_ops = cb->fn(dev_id, qp_id, ops, nb_ops, - cb->arg); - cb = cb->next; - }; - - rte_rcu_qsbr_thread_offline(list->qsbr, 0); - } -#endif - - rte_cryptodev_trace_enqueue_burst(dev_id, qp_id, (void **)ops, nb_ops); - return (*dev->enqueue_burst)( - dev->data->queue_pairs[qp_id], ops, nb_ops); -} - - /** Cryptodev symmetric crypto session * Each session is derived from a fixed xform chain. Therefore each session * has a fixed algo, key, op-type, digest_len etc. @@ -2009,6 +1791,148 @@ int rte_cryptodev_remove_deq_callback(uint8_t dev_id, uint16_t qp_id, struct rte_cryptodev_cb *cb); +#include +/** + * + * Dequeue a burst of processed crypto operations from a queue on the crypto + * device. The dequeued operation are stored in *rte_crypto_op* structures + * whose pointers are supplied in the *ops* array. + * + * The rte_cryptodev_dequeue_burst() function returns the number of ops + * actually dequeued, which is the number of *rte_crypto_op* data structures + * effectively supplied into the *ops* array. + * + * A return value equal to *nb_ops* indicates that the queue contained + * at least *nb_ops* operations, and this is likely to signify that other + * processed operations remain in the devices output queue. Applications + * implementing a "retrieve as many processed operations as possible" policy + * can check this specific case and keep invoking the + * rte_cryptodev_dequeue_burst() function until a value less than + * *nb_ops* is returned. + * + * The rte_cryptodev_dequeue_burst() function does not provide any error + * notification to avoid the corresponding overhead. + * + * @param dev_id The symmetric crypto device identifier + * @param qp_id The index of the queue pair from which to + * retrieve processed packets. The value must be + * in the range [0, nb_queue_pair - 1] previously + * supplied to rte_cryptodev_configure(). + * @param ops The address of an array of pointers to + * *rte_crypto_op* structures that must be + * large enough to store *nb_ops* pointers in it. + * @param nb_ops The maximum number of operations to dequeue. + * + * @return + * - The number of operations actually dequeued, which is the number + * of pointers to *rte_crypto_op* structures effectively supplied to the + * *ops* array. + */ +static inline uint16_t +rte_cryptodev_dequeue_burst(uint8_t dev_id, uint16_t qp_id, + struct rte_crypto_op **ops, uint16_t nb_ops) +{ + struct rte_cryptodev *dev = &rte_cryptodevs[dev_id]; + + rte_cryptodev_trace_dequeue_burst(dev_id, qp_id, (void **)ops, nb_ops); + nb_ops = (*dev->dequeue_burst) + (dev->data->queue_pairs[qp_id], ops, nb_ops); +#ifdef RTE_CRYPTO_CALLBACKS + if (unlikely(dev->deq_cbs != NULL)) { + struct rte_cryptodev_cb_rcu *list; + struct rte_cryptodev_cb *cb; + + /* __ATOMIC_RELEASE memory order was used when the + * call back was inserted into the list. + * Since there is a clear dependency between loading + * cb and cb->fn/cb->next, __ATOMIC_ACQUIRE memory order is + * not required. + */ + list = &dev->deq_cbs[qp_id]; + rte_rcu_qsbr_thread_online(list->qsbr, 0); + cb = __atomic_load_n(&list->next, __ATOMIC_RELAXED); + + while (cb != NULL) { + nb_ops = cb->fn(dev_id, qp_id, ops, nb_ops, + cb->arg); + cb = cb->next; + }; + + rte_rcu_qsbr_thread_offline(list->qsbr, 0); + } +#endif + return nb_ops; +} + +/** + * Enqueue a burst of operations for processing on a crypto device. + * + * The rte_cryptodev_enqueue_burst() function is invoked to place + * crypto operations on the queue *qp_id* of the device designated by + * its *dev_id*. + * + * The *nb_ops* parameter is the number of operations to process which are + * supplied in the *ops* array of *rte_crypto_op* structures. + * + * The rte_cryptodev_enqueue_burst() function returns the number of + * operations it actually enqueued for processing. A return value equal to + * *nb_ops* means that all packets have been enqueued. + * + * @param dev_id The identifier of the device. + * @param qp_id The index of the queue pair which packets are + * to be enqueued for processing. The value + * must be in the range [0, nb_queue_pairs - 1] + * previously supplied to + * *rte_cryptodev_configure*. + * @param ops The address of an array of *nb_ops* pointers + * to *rte_crypto_op* structures which contain + * the crypto operations to be processed. + * @param nb_ops The number of operations to process. + * + * @return + * The number of operations actually enqueued on the crypto device. The return + * value can be less than the value of the *nb_ops* parameter when the + * crypto devices queue is full or if invalid parameters are specified in + * a *rte_crypto_op*. + */ +static inline uint16_t +rte_cryptodev_enqueue_burst(uint8_t dev_id, uint16_t qp_id, + struct rte_crypto_op **ops, uint16_t nb_ops) +{ + struct rte_cryptodev *dev = &rte_cryptodevs[dev_id]; + +#ifdef RTE_CRYPTO_CALLBACKS + if (unlikely(dev->enq_cbs != NULL)) { + struct rte_cryptodev_cb_rcu *list; + struct rte_cryptodev_cb *cb; + + /* __ATOMIC_RELEASE memory order was used when the + * call back was inserted into the list. + * Since there is a clear dependency between loading + * cb and cb->fn/cb->next, __ATOMIC_ACQUIRE memory order is + * not required. + */ + list = &dev->enq_cbs[qp_id]; + rte_rcu_qsbr_thread_online(list->qsbr, 0); + cb = __atomic_load_n(&list->next, __ATOMIC_RELAXED); + + while (cb != NULL) { + nb_ops = cb->fn(dev_id, qp_id, ops, nb_ops, + cb->arg); + cb = cb->next; + }; + + rte_rcu_qsbr_thread_offline(list->qsbr, 0); + } +#endif + + rte_cryptodev_trace_enqueue_burst(dev_id, qp_id, (void **)ops, nb_ops); + return (*dev->enqueue_burst)( + dev->data->queue_pairs[qp_id], ops, nb_ops); +} + + + #ifdef __cplusplus } #endif diff --git a/lib/cryptodev/rte_cryptodev_core.h b/lib/cryptodev/rte_cryptodev_core.h new file mode 100644 index 0000000000..1633e55889 --- /dev/null +++ b/lib/cryptodev/rte_cryptodev_core.h @@ -0,0 +1,100 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef _RTE_CRYPTODEV_CORE_H_ +#define _RTE_CRYPTODEV_CORE_H_ + +/** + * @file + * + * RTE Crypto Device internal header. + * + * This header contains internal data types. But they are still part of the + * public API because they are used by inline functions in the published API. + * + * Applications should not use these directly. + * + */ + +typedef uint16_t (*dequeue_pkt_burst_t)(void *qp, + struct rte_crypto_op **ops, uint16_t nb_ops); +/**< Dequeue processed packets from queue pair of a device. */ + +typedef uint16_t (*enqueue_pkt_burst_t)(void *qp, + struct rte_crypto_op **ops, uint16_t nb_ops); +/**< Enqueue packets for processing on queue pair of a device. */ + +/** + * @internal + * The data part, with no function pointers, associated with each device. + * + * This structure is safe to place in shared memory to be common among + * different processes in a multi-process configuration. + */ +struct rte_cryptodev_data { + uint8_t dev_id; + /**< Device ID for this instance */ + uint8_t socket_id; + /**< Socket ID where memory is allocated */ + char name[RTE_CRYPTODEV_NAME_MAX_LEN]; + /**< Unique identifier name */ + + __extension__ + uint8_t dev_started : 1; + /**< Device state: STARTED(1)/STOPPED(0) */ + + struct rte_mempool *session_pool; + /**< Session memory pool */ + void **queue_pairs; + /**< Array of pointers to queue pairs. */ + uint16_t nb_queue_pairs; + /**< Number of device queue pairs. */ + + void *dev_private; + /**< PMD-specific private data */ +} __rte_cache_aligned; + + +/** @internal The data structure associated with each crypto device. */ +struct rte_cryptodev { + dequeue_pkt_burst_t dequeue_burst; + /**< Pointer to PMD receive function. */ + enqueue_pkt_burst_t enqueue_burst; + /**< Pointer to PMD transmit function. */ + + struct rte_cryptodev_data *data; + /**< Pointer to device data */ + struct rte_cryptodev_ops *dev_ops; + /**< Functions exported by PMD */ + uint64_t feature_flags; + /**< Feature flags exposes HW/SW features for the given device */ + struct rte_device *device; + /**< Backing device */ + + uint8_t driver_id; + /**< Crypto driver identifier*/ + + struct rte_cryptodev_cb_list link_intr_cbs; + /**< User application callback for interrupts if present */ + + void *security_ctx; + /**< Context for security ops */ + + __extension__ + uint8_t attached : 1; + /**< Flag indicating the device is attached */ + + struct rte_cryptodev_cb_rcu *enq_cbs; + /**< User application callback for pre enqueue processing */ + + struct rte_cryptodev_cb_rcu *deq_cbs; + /**< User application callback for post dequeue processing */ +} __rte_cache_aligned; + +/** + * The pool of rte_cryptodev structures. + */ +extern struct rte_cryptodev *rte_cryptodevs; + +#endif /* _RTE_CRYPTODEV_CORE_H_ */ From patchwork Wed Oct 20 11:27:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 102422 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 997D5A0C43; Wed, 20 Oct 2021 13:28:32 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0D1D94118F; Wed, 20 Oct 2021 13:28:31 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 26509411FD for ; Wed, 20 Oct 2021 13:28:30 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19K8kiWp020984; Wed, 20 Oct 2021 04:28:23 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=nzybg0JJejNCCop6BqF7461gKDp7Ql8YPonwXvLx9is=; b=XW228ngzjZfGaOpqGenl0pnWIvTehvnMVunRNfgn3ReeGQhFcQ4hso8NB3YpXcGYujRn WyldmsLUtfTBs1CWx3sPrix1VAVKelXagmzKfgs0s7ei/7300ABWOupa+YAMFxLRI7z1 n2779mlEgyiPRpm9nh69vs1ty7OIH9XsVUFLaTc1Kh5MNMrx6OAajjbsTPB2+eo7T06Z rMBtvu41l0tzmxc+VZ5q2JXWMAJ5g/rBR6qG83hb1e3IY1tFrhma/X047FnPPeMrKpOZ NZjUFtuDePGw7HoXdOhgkCQRO8K95fzmJiDNyVcKFmiG3CpaE+AEgR6I4XPTSqTOqrka 1A== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 3bt05g4pv6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 04:28:23 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 04:28:21 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 20 Oct 2021 04:28:21 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id 0D8953F7098; Wed, 20 Oct 2021 04:28:14 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal Date: Wed, 20 Oct 2021 16:57:48 +0530 Message-ID: <20211020112754.1270163-3-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211020112754.1270163-1-gakhil@marvell.com> References: <20211018144201.2028022-1-gakhil@marvell.com> <20211020112754.1270163-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: bijQ647aR5uECrNrlbiCn9BUZGHrO9Hy X-Proofpoint-ORIG-GUID: bijQ647aR5uECrNrlbiCn9BUZGHrO9Hy X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-20_04,2021-10-20_02,2020-04-07_01 Subject: [dpdk-dev] [PATCH v4 2/8] cryptodev: allocate max space for internal qp array X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" At queue_pair config stage, allocate memory for maximum number of queue pair pointers that a device can support. This will allow fast path APIs(enqueue_burst/dequeue_burst) to refer pointer to internal QP data without checking for currently configured QPs. This is required to hide the rte_cryptodev and rte_cryptodev_data structure from user. Signed-off-by: Akhil Goyal Acked-by: Fan Zhang Acked-by: Konstantin Ananyev --- lib/cryptodev/rte_cryptodev.c | 21 +++------------------ 1 file changed, 3 insertions(+), 18 deletions(-) diff --git a/lib/cryptodev/rte_cryptodev.c b/lib/cryptodev/rte_cryptodev.c index b913c434c5..eb86e629aa 100644 --- a/lib/cryptodev/rte_cryptodev.c +++ b/lib/cryptodev/rte_cryptodev.c @@ -978,7 +978,8 @@ rte_cryptodev_queue_pairs_config(struct rte_cryptodev *dev, uint16_t nb_qpairs, if (dev->data->queue_pairs == NULL) { /* first time configuration */ dev->data->queue_pairs = rte_zmalloc_socket( "cryptodev->queue_pairs", - sizeof(dev->data->queue_pairs[0]) * nb_qpairs, + sizeof(dev->data->queue_pairs[0]) * + dev_info.max_nb_queue_pairs, RTE_CACHE_LINE_SIZE, socket_id); if (dev->data->queue_pairs == NULL) { @@ -1001,25 +1002,9 @@ rte_cryptodev_queue_pairs_config(struct rte_cryptodev *dev, uint16_t nb_qpairs, ret = (*dev->dev_ops->queue_pair_release)(dev, i); if (ret < 0) return ret; + qp[i] = NULL; } - qp = rte_realloc(qp, sizeof(qp[0]) * nb_qpairs, - RTE_CACHE_LINE_SIZE); - if (qp == NULL) { - CDEV_LOG_ERR("failed to realloc qp meta data," - " nb_queues %u", nb_qpairs); - return -(ENOMEM); - } - - if (nb_qpairs > old_nb_queues) { - uint16_t new_qs = nb_qpairs - old_nb_queues; - - memset(qp + old_nb_queues, 0, - sizeof(qp[0]) * new_qs); - } - - dev->data->queue_pairs = qp; - } dev->data->nb_queue_pairs = nb_qpairs; return 0; From patchwork Wed Oct 20 11:27:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 102423 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7DD85A0C43; Wed, 20 Oct 2021 13:28:38 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0F778411FA; Wed, 20 Oct 2021 13:28:38 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 62F3240142 for ; Wed, 20 Oct 2021 13:28:35 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19K8K7tE021014; Wed, 20 Oct 2021 04:28:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=gxmrBeKVkyjnE3mcKazp1VxeCORA3pkxjPBBTRJTnB8=; b=kMSOy6KwfPRXtuwLVJiUWGN/2FGNkOZPpkEwHzNj/tbiZ+KLBP0TpBh9xMcWlexHYaEA SZdHQhZQ9dOcFk1HvPpGKTZ+5TyUlx95OGyyKxCMy2CnckTpzyS5x04cAsE8cARkzDXN AvpqwIIHOo4pust3DzhNJ3gza7V0gxEWbnQvmFpJCIcwaFr0JY7vgfx73ZLUY0mTAYEt ssWKnBSYe3JaPZjj+QWa8Y36YFazgiIWD5ArlZ+T7ttrmBJra+HqdW34fMavct0IY/6L UYFQxDHNlFPBFmG0NsriDkUNXqhWLuvxeKx7y/ExSbJjCISgKUxvE7m0DxV1jB70286B KQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 3bt05g4pwg-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 04:28:29 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 04:28:28 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 20 Oct 2021 04:28:28 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id A2A773F707F; Wed, 20 Oct 2021 04:28:21 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal , Rebecca Troy Date: Wed, 20 Oct 2021 16:57:49 +0530 Message-ID: <20211020112754.1270163-4-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211020112754.1270163-1-gakhil@marvell.com> References: <20211018144201.2028022-1-gakhil@marvell.com> <20211020112754.1270163-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: wrsxH6RTvXH_vFzXWJtvEjLDJXs5orst X-Proofpoint-ORIG-GUID: wrsxH6RTvXH_vFzXWJtvEjLDJXs5orst X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-20_04,2021-10-20_02,2020-04-07_01 Subject: [dpdk-dev] [PATCH v4 3/8] cryptodev: move inline APIs into separate structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Move fastpath inline function pointers from rte_cryptodev into a separate structure accessed via a flat array. The intension is to make rte_cryptodev and related structures private to avoid future API/ABI breakages. Signed-off-by: Akhil Goyal Tested-by: Rebecca Troy Acked-by: Fan Zhang Acked-by: Konstantin Ananyev --- lib/cryptodev/cryptodev_pmd.c | 53 +++++++++++++++++++++++++++++- lib/cryptodev/cryptodev_pmd.h | 11 +++++++ lib/cryptodev/rte_cryptodev.c | 19 +++++++++++ lib/cryptodev/rte_cryptodev_core.h | 29 ++++++++++++++++ lib/cryptodev/version.map | 5 +++ 5 files changed, 116 insertions(+), 1 deletion(-) diff --git a/lib/cryptodev/cryptodev_pmd.c b/lib/cryptodev/cryptodev_pmd.c index 44a70ecb35..fd74543682 100644 --- a/lib/cryptodev/cryptodev_pmd.c +++ b/lib/cryptodev/cryptodev_pmd.c @@ -3,7 +3,7 @@ */ #include - +#include #include #include @@ -160,3 +160,54 @@ rte_cryptodev_pmd_destroy(struct rte_cryptodev *cryptodev) return 0; } + +static uint16_t +dummy_crypto_enqueue_burst(__rte_unused void *qp, + __rte_unused struct rte_crypto_op **ops, + __rte_unused uint16_t nb_ops) +{ + CDEV_LOG_ERR( + "crypto enqueue burst requested for unconfigured device"); + rte_errno = ENOTSUP; + return 0; +} + +static uint16_t +dummy_crypto_dequeue_burst(__rte_unused void *qp, + __rte_unused struct rte_crypto_op **ops, + __rte_unused uint16_t nb_ops) +{ + CDEV_LOG_ERR( + "crypto dequeue burst requested for unconfigured device"); + rte_errno = ENOTSUP; + return 0; +} + +void +cryptodev_fp_ops_reset(struct rte_crypto_fp_ops *fp_ops) +{ + static struct rte_cryptodev_cb_rcu dummy_cb[RTE_MAX_QUEUES_PER_PORT]; + static void *dummy_data[RTE_MAX_QUEUES_PER_PORT]; + static const struct rte_crypto_fp_ops dummy = { + .enqueue_burst = dummy_crypto_enqueue_burst, + .dequeue_burst = dummy_crypto_dequeue_burst, + .qp = { + .data = dummy_data, + .enq_cb = dummy_cb, + .deq_cb = dummy_cb, + }, + }; + + *fp_ops = dummy; +} + +void +cryptodev_fp_ops_set(struct rte_crypto_fp_ops *fp_ops, + const struct rte_cryptodev *dev) +{ + fp_ops->enqueue_burst = dev->enqueue_burst; + fp_ops->dequeue_burst = dev->dequeue_burst; + fp_ops->qp.data = dev->data->queue_pairs; + fp_ops->qp.enq_cb = dev->enq_cbs; + fp_ops->qp.deq_cb = dev->deq_cbs; +} diff --git a/lib/cryptodev/cryptodev_pmd.h b/lib/cryptodev/cryptodev_pmd.h index 36606dd10b..a71edbb991 100644 --- a/lib/cryptodev/cryptodev_pmd.h +++ b/lib/cryptodev/cryptodev_pmd.h @@ -516,6 +516,17 @@ RTE_INIT(init_ ##driver_id)\ driver_id = rte_cryptodev_allocate_driver(&crypto_drv, &(drv));\ } +/* Reset crypto device fastpath APIs to dummy values. */ +__rte_internal +void +cryptodev_fp_ops_reset(struct rte_crypto_fp_ops *fp_ops); + +/* Setup crypto device fastpath APIs. */ +__rte_internal +void +cryptodev_fp_ops_set(struct rte_crypto_fp_ops *fp_ops, + const struct rte_cryptodev *dev); + static inline void * get_sym_session_private_data(const struct rte_cryptodev_sym_session *sess, uint8_t driver_id) { diff --git a/lib/cryptodev/rte_cryptodev.c b/lib/cryptodev/rte_cryptodev.c index eb86e629aa..305e013ebb 100644 --- a/lib/cryptodev/rte_cryptodev.c +++ b/lib/cryptodev/rte_cryptodev.c @@ -53,6 +53,9 @@ static struct rte_cryptodev_global cryptodev_globals = { .nb_devs = 0 }; +/* Public fastpath APIs. */ +struct rte_crypto_fp_ops rte_crypto_fp_ops[RTE_CRYPTO_MAX_DEVS]; + /* spinlock for crypto device callbacks */ static rte_spinlock_t rte_cryptodev_cb_lock = RTE_SPINLOCK_INITIALIZER; @@ -917,6 +920,8 @@ rte_cryptodev_pmd_release_device(struct rte_cryptodev *cryptodev) dev_id = cryptodev->data->dev_id; + cryptodev_fp_ops_reset(rte_crypto_fp_ops + dev_id); + /* Close device only if device operations have been set */ if (cryptodev->dev_ops) { ret = rte_cryptodev_close(dev_id); @@ -1080,6 +1085,9 @@ rte_cryptodev_start(uint8_t dev_id) } diag = (*dev->dev_ops->dev_start)(dev); + /* expose selection of PMD fast-path functions */ + cryptodev_fp_ops_set(rte_crypto_fp_ops + dev_id, dev); + rte_cryptodev_trace_start(dev_id, diag); if (diag == 0) dev->data->dev_started = 1; @@ -1109,6 +1117,9 @@ rte_cryptodev_stop(uint8_t dev_id) return; } + /* point fast-path functions to dummy ones */ + cryptodev_fp_ops_reset(rte_crypto_fp_ops + dev_id); + (*dev->dev_ops->dev_stop)(dev); rte_cryptodev_trace_stop(dev_id); dev->data->dev_started = 0; @@ -2411,3 +2422,11 @@ rte_cryptodev_allocate_driver(struct cryptodev_driver *crypto_drv, return nb_drivers++; } + +RTE_INIT(cryptodev_init_fp_ops) +{ + uint32_t i; + + for (i = 0; i != RTE_DIM(rte_crypto_fp_ops); i++) + cryptodev_fp_ops_reset(rte_crypto_fp_ops + i); +} diff --git a/lib/cryptodev/rte_cryptodev_core.h b/lib/cryptodev/rte_cryptodev_core.h index 1633e55889..2bb9a228c1 100644 --- a/lib/cryptodev/rte_cryptodev_core.h +++ b/lib/cryptodev/rte_cryptodev_core.h @@ -25,6 +25,35 @@ typedef uint16_t (*enqueue_pkt_burst_t)(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops); /**< Enqueue packets for processing on queue pair of a device. */ +/** + * @internal + * Structure used to hold opaque pointers to internal ethdev Rx/Tx + * queues data. + * The main purpose to expose these pointers at all - allow compiler + * to fetch this data for fast-path cryptodev inline functions in advance. + */ +struct rte_cryptodev_qpdata { + /** points to array of internal queue pair data pointers. */ + void **data; + /** points to array of enqueue callback data pointers */ + struct rte_cryptodev_cb_rcu *enq_cb; + /** points to array of dequeue callback data pointers */ + struct rte_cryptodev_cb_rcu *deq_cb; +}; + +struct rte_crypto_fp_ops { + /** PMD enqueue burst function. */ + enqueue_pkt_burst_t enqueue_burst; + /** PMD dequeue burst function. */ + dequeue_pkt_burst_t dequeue_burst; + /** Internal queue pair data pointers. */ + struct rte_cryptodev_qpdata qp; + /** Reserved for future ops. */ + uintptr_t reserved[3]; +} __rte_cache_aligned; + +extern struct rte_crypto_fp_ops rte_crypto_fp_ops[RTE_CRYPTO_MAX_DEVS]; + /** * @internal * The data part, with no function pointers, associated with each device. diff --git a/lib/cryptodev/version.map b/lib/cryptodev/version.map index 43cf937e40..ed62ced221 100644 --- a/lib/cryptodev/version.map +++ b/lib/cryptodev/version.map @@ -45,6 +45,9 @@ DPDK_22 { rte_cryptodev_sym_session_init; rte_cryptodevs; + #added in 21.11 + rte_crypto_fp_ops; + local: *; }; @@ -109,6 +112,8 @@ EXPERIMENTAL { INTERNAL { global: + cryptodev_fp_ops_reset; + cryptodev_fp_ops_set; rte_cryptodev_allocate_driver; rte_cryptodev_pmd_allocate; rte_cryptodev_pmd_callback_process; From patchwork Wed Oct 20 11:27:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 102424 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 02BE5A0C43; Wed, 20 Oct 2021 13:28:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 598DE41203; Wed, 20 Oct 2021 13:28:45 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 4243C411F3 for ; Wed, 20 Oct 2021 13:28:43 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19KBCiPm015069; Wed, 20 Oct 2021 04:28:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=tbFnutcqeHU2IL7BrGrXM3mlivULTlS2a0V7Jz8JOwg=; b=GNFOGQHQAH7XHR0FSBrrmLeRE1b+ArLN+oE65xxnRSZ/gOqpa0TlEPn4lJQ3gLQTj5IJ RpOivhVo9oceAgWNd/ZMznRuCMAi1ZQs5DGa2dvub4CTj1BoyhXWv3orKfUxJqsgzCRh tX+NJ5VemwN8ja/L1zeL49TRnlPF4zZ5VBGu59Ko0ttLIbt/pCS92Ubp4RiAVnh7eKGL WwI6T+Q74nHkR42xIdeb7Me0WaozmGm2whPDVZpipwkYBwWy0tws7uzzeOtqeYeJYt+Q eD0Oo+KQ845dYw2FhLUsyQ29w2jKSe4hlvnh0o3hxzcIg1cHro/PzqL2r7A+ZOFvSeLh ww== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3btaa7a4sa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 04:28:36 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 04:28:34 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 20 Oct 2021 04:28:34 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id 7F4333F7080; Wed, 20 Oct 2021 04:28:28 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal Date: Wed, 20 Oct 2021 16:57:50 +0530 Message-ID: <20211020112754.1270163-5-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211020112754.1270163-1-gakhil@marvell.com> References: <20211018144201.2028022-1-gakhil@marvell.com> <20211020112754.1270163-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: KoZp_J-EoBOSgbLcEn9RgtNroJAjKjE3 X-Proofpoint-GUID: KoZp_J-EoBOSgbLcEn9RgtNroJAjKjE3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-20_04,2021-10-20_02,2020-04-07_01 Subject: [dpdk-dev] [PATCH v4 4/8] crypto/scheduler: use proper API for device start/stop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The worker PMDs were using direct device start/stop functions rather than rte_cryptodev_start(), so rte_crypto_fp_ops never get set. This patch calls the rte_cryptodev_start and stop APIs which start and stop devices properly and fp_ops get set. Reported-by: Ciara Power Signed-off-by: Akhil Goyal Acked-by: Fan Zhang Acked-by: Konstantin Ananyev --- drivers/crypto/scheduler/scheduler_pmd_ops.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/crypto/scheduler/scheduler_pmd_ops.c b/drivers/crypto/scheduler/scheduler_pmd_ops.c index 465b88ade8..f61640c9d6 100644 --- a/drivers/crypto/scheduler/scheduler_pmd_ops.c +++ b/drivers/crypto/scheduler/scheduler_pmd_ops.c @@ -181,10 +181,7 @@ scheduler_pmd_start(struct rte_cryptodev *dev) /* start all workers */ for (i = 0; i < sched_ctx->nb_workers; i++) { uint8_t worker_dev_id = sched_ctx->workers[i].dev_id; - struct rte_cryptodev *worker_dev = - rte_cryptodev_pmd_get_dev(worker_dev_id); - - ret = (*worker_dev->dev_ops->dev_start)(worker_dev); + ret = rte_cryptodev_start(worker_dev_id); if (ret < 0) { CR_SCHED_LOG(ERR, "Failed to start worker dev %u", worker_dev_id); @@ -208,10 +205,8 @@ scheduler_pmd_stop(struct rte_cryptodev *dev) /* stop all workers first */ for (i = 0; i < sched_ctx->nb_workers; i++) { uint8_t worker_dev_id = sched_ctx->workers[i].dev_id; - struct rte_cryptodev *worker_dev = - rte_cryptodev_pmd_get_dev(worker_dev_id); - (*worker_dev->dev_ops->dev_stop)(worker_dev); + rte_cryptodev_stop(worker_dev_id); } if (*sched_ctx->ops.scheduler_stop) From patchwork Wed Oct 20 11:27:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 102425 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CC678A0C43; Wed, 20 Oct 2021 13:28:51 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 71010411F3; Wed, 20 Oct 2021 13:28:51 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id B2E4440142 for ; Wed, 20 Oct 2021 13:28:49 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19KBQ9Vr015156; Wed, 20 Oct 2021 04:28:43 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=4NBtYcoGNTgehHffHIKWrtE8xdEBe+EZhs+5UACiAhc=; b=SpGSY6YkYyXu6FdsiGIgkpWlDYtFppT5ahXUDSR1Tyk9CG2em7f1HTfL4mJtjcmeQd5g Abz+S6YxN0U9OD/lIQEgIMwM/BGDa41yWU3UWHbhBaMk9JDhqQMkGKVk4lZHzNoOvObg I66dM+wA12lAY0uht311K3UJPKShUVwpoTLODBpQJ+Jwz/40BBXyZCisKgIFCucfuf5i 9Ma68G/jdQhHwT4ZP9ppwdUpHbE72/GJ58iP0nXQvAGfOfwTXhnk6DXkxNUVCPMAr2cM mEptOCNi/0Y2iGSsedv3pS5w3DTfBzf9rtRTNg+zKGKVNHhIvO4OEQiOGQz3t/jtL+Gy Xw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3btaa7a4sj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 04:28:42 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 04:28:41 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 20 Oct 2021 04:28:41 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id 412A13F707F; Wed, 20 Oct 2021 04:28:35 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal Date: Wed, 20 Oct 2021 16:57:51 +0530 Message-ID: <20211020112754.1270163-6-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211020112754.1270163-1-gakhil@marvell.com> References: <20211018144201.2028022-1-gakhil@marvell.com> <20211020112754.1270163-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 25AcxSewQqRE68SRQf8aqPrqFnA4E5Ni X-Proofpoint-GUID: 25AcxSewQqRE68SRQf8aqPrqFnA4E5Ni X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-20_04,2021-10-20_02,2020-04-07_01 Subject: [dpdk-dev] [PATCH v4 5/8] cryptodev: add PMD device probe finish API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Added a rte_cryptodev_pmd_probing_finish API which need to be called by the PMD after the device is initialized completely. This will set the fast path function pointers in the flat array for secondary process. For primary process, these are set in rte_cryptodev_start. Signed-off-by: Akhil Goyal Acked-by: Konstantin Ananyev Acked-by: Fan Zhang --- lib/cryptodev/cryptodev_pmd.c | 16 ++++++++++++++++ lib/cryptodev/cryptodev_pmd.h | 13 +++++++++++++ lib/cryptodev/version.map | 1 + 3 files changed, 30 insertions(+) diff --git a/lib/cryptodev/cryptodev_pmd.c b/lib/cryptodev/cryptodev_pmd.c index fd74543682..739a0b3f34 100644 --- a/lib/cryptodev/cryptodev_pmd.c +++ b/lib/cryptodev/cryptodev_pmd.c @@ -161,6 +161,22 @@ rte_cryptodev_pmd_destroy(struct rte_cryptodev *cryptodev) return 0; } +void +rte_cryptodev_pmd_probing_finish(struct rte_cryptodev *cryptodev) +{ + if (cryptodev == NULL) + return; + /* + * for secondary process, at that point we expect device + * to be already 'usable', so shared data and all function + * pointers for fast-path devops have to be setup properly + * inside rte_cryptodev. + */ + if (rte_eal_process_type() == RTE_PROC_SECONDARY) + cryptodev_fp_ops_set(rte_crypto_fp_ops + + cryptodev->data->dev_id, cryptodev); +} + static uint16_t dummy_crypto_enqueue_burst(__rte_unused void *qp, __rte_unused struct rte_crypto_op **ops, diff --git a/lib/cryptodev/cryptodev_pmd.h b/lib/cryptodev/cryptodev_pmd.h index a71edbb991..9bb1e47ae4 100644 --- a/lib/cryptodev/cryptodev_pmd.h +++ b/lib/cryptodev/cryptodev_pmd.h @@ -509,6 +509,19 @@ __rte_internal uint8_t rte_cryptodev_allocate_driver(struct cryptodev_driver *crypto_drv, const struct rte_driver *drv); +/** + * @internal + * This is the last step of device probing. It must be called after a + * cryptodev is allocated and initialized successfully. + * + * @param dev Pointer to cryptodev struct + * + * @return + * void + */ +__rte_internal +void +rte_cryptodev_pmd_probing_finish(struct rte_cryptodev *dev); #define RTE_PMD_REGISTER_CRYPTO_DRIVER(crypto_drv, drv, driver_id)\ RTE_INIT(init_ ##driver_id)\ diff --git a/lib/cryptodev/version.map b/lib/cryptodev/version.map index ed62ced221..157dac521d 100644 --- a/lib/cryptodev/version.map +++ b/lib/cryptodev/version.map @@ -123,5 +123,6 @@ INTERNAL { rte_cryptodev_pmd_get_dev; rte_cryptodev_pmd_get_named_dev; rte_cryptodev_pmd_parse_input_args; + rte_cryptodev_pmd_probing_finish; rte_cryptodev_pmd_release_device; }; From patchwork Wed Oct 20 11:27:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 102426 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8DD98A0C43; Wed, 20 Oct 2021 13:28:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7777D411F7; Wed, 20 Oct 2021 13:28:58 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id C242A40142 for ; Wed, 20 Oct 2021 13:28:56 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19KBCiPr015069; Wed, 20 Oct 2021 04:28:50 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=ri9+pmORzPv8yITvVwgusYotQduvkGpR/3OplXB9+Es=; b=g2Td1F5ivumRd30ya8S3NROfHc74Ph72K2n7xlWpGxXuwYb7CD72a9lhd4X8deCzDhN7 HZ1hke3CVgTBbVVkl2DXCXUzFmKGIThJ8/laYZq2Yj95bB0pyJGBkqzWrD1Rs9cvP8h3 RsbiRwJ3kluSBVsmp6ibnuP7FzsFPZpTpf0J17ncqjknTI/+JN8XW9AKZyNWVwQ0ixfG dninFj96gzPQ1zKUKeQGoy58t3ops+dLWQZcod75wJZL5a5HFDMmLBFsJfIh2sWmJZwk SQbApYsQ4Z7YhyRLpil60wiLaQFwlijwP8rP4LrZp+gesi6hdZBRdZdYMkCoKhRNVFbQ Aw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3btaa7a4t9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 04:28:50 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 04:28:48 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 20 Oct 2021 04:28:48 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id D19323F7082; Wed, 20 Oct 2021 04:28:41 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal Date: Wed, 20 Oct 2021 16:57:52 +0530 Message-ID: <20211020112754.1270163-7-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211020112754.1270163-1-gakhil@marvell.com> References: <20211018144201.2028022-1-gakhil@marvell.com> <20211020112754.1270163-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: eHj5SWapwsNU_jMw7lzc61KXqHVjLnD0 X-Proofpoint-GUID: eHj5SWapwsNU_jMw7lzc61KXqHVjLnD0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-20_04,2021-10-20_02,2020-04-07_01 Subject: [dpdk-dev] [PATCH v4 6/8] drivers/crypto: invoke probing finish function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Invoke event_dev_probing_finish() function at the end of probing, this function sets the function pointers in the fp_ops flat array in case of secondary process. For primary process, fp_ops is updated in rte_cryptodev_start(). Signed-off-by: Akhil Goyal Acked-by: Konstantin Ananyev Acked-by: Fan Zhang Acked-by: Matan Azrad --- drivers/crypto/armv8/rte_armv8_pmd.c | 2 ++ drivers/crypto/bcmfs/bcmfs_sym_pmd.c | 2 ++ drivers/crypto/caam_jr/caam_jr.c | 2 ++ drivers/crypto/ccp/rte_ccp_pmd.c | 2 ++ drivers/crypto/cnxk/cn10k_cryptodev.c | 2 ++ drivers/crypto/cnxk/cn9k_cryptodev.c | 2 ++ drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 4 +++- drivers/crypto/dpaa_sec/dpaa_sec.c | 4 +++- drivers/crypto/ipsec_mb/ipsec_mb_private.c | 2 ++ drivers/crypto/mlx5/mlx5_crypto.c | 3 +++ drivers/crypto/mvsam/rte_mrvl_pmd.c | 2 ++ drivers/crypto/nitrox/nitrox_sym.c | 3 +++ drivers/crypto/null/null_crypto_pmd.c | 2 ++ drivers/crypto/octeontx/otx_cryptodev.c | 4 +++- drivers/crypto/octeontx2/otx2_cryptodev.c | 2 ++ drivers/crypto/openssl/rte_openssl_pmd.c | 2 ++ drivers/crypto/qat/qat_asym_pmd.c | 3 +++ drivers/crypto/qat/qat_sym_pmd.c | 2 ++ drivers/crypto/scheduler/scheduler_pmd.c | 2 ++ drivers/crypto/virtio/virtio_cryptodev.c | 2 ++ 20 files changed, 46 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/armv8/rte_armv8_pmd.c b/drivers/crypto/armv8/rte_armv8_pmd.c index 36a1a9bb4f..32127a874c 100644 --- a/drivers/crypto/armv8/rte_armv8_pmd.c +++ b/drivers/crypto/armv8/rte_armv8_pmd.c @@ -802,6 +802,8 @@ cryptodev_armv8_crypto_create(const char *name, internals->max_nb_qpairs = init_params->max_nb_queue_pairs; + rte_cryptodev_pmd_probing_finish(dev); + return 0; init_error: diff --git a/drivers/crypto/bcmfs/bcmfs_sym_pmd.c b/drivers/crypto/bcmfs/bcmfs_sym_pmd.c index d1dd22823e..bc7fb67218 100644 --- a/drivers/crypto/bcmfs/bcmfs_sym_pmd.c +++ b/drivers/crypto/bcmfs/bcmfs_sym_pmd.c @@ -394,6 +394,8 @@ bcmfs_sym_dev_create(struct bcmfs_device *fsdev) internals->sym_dev_id = cryptodev->data->dev_id; internals->fsdev_capabilities = bcmfs_sym_get_capabilities(); + rte_cryptodev_pmd_probing_finish(cryptodev); + BCMFS_LOG(DEBUG, "Created bcmfs-sym device %s as cryptodev instance %d", cryptodev->data->name, internals->sym_dev_id); return 0; diff --git a/drivers/crypto/caam_jr/caam_jr.c b/drivers/crypto/caam_jr/caam_jr.c index 258750afe7..8c56610ac8 100644 --- a/drivers/crypto/caam_jr/caam_jr.c +++ b/drivers/crypto/caam_jr/caam_jr.c @@ -2368,6 +2368,8 @@ caam_jr_dev_init(const char *name, security_instance->sess_cnt = 0; dev->security_ctx = security_instance; + rte_cryptodev_pmd_probing_finish(dev); + RTE_LOG(INFO, PMD, "%s cryptodev init\n", dev->data->name); return 0; diff --git a/drivers/crypto/ccp/rte_ccp_pmd.c b/drivers/crypto/ccp/rte_ccp_pmd.c index a54d81de46..a35a8cd775 100644 --- a/drivers/crypto/ccp/rte_ccp_pmd.c +++ b/drivers/crypto/ccp/rte_ccp_pmd.c @@ -271,6 +271,8 @@ cryptodev_ccp_create(const char *name, internals->auth_opt = init_params->auth_opt; internals->crypto_num_dev = cryptodev_cnt; + rte_cryptodev_pmd_probing_finish(dev); + return 0; init_error: diff --git a/drivers/crypto/cnxk/cn10k_cryptodev.c b/drivers/crypto/cnxk/cn10k_cryptodev.c index 869d322d9b..1e0a7b91c9 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev.c @@ -102,6 +102,8 @@ cn10k_cpt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, cn10k_cpt_set_enqdeq_fns(dev); cn10k_sec_ops_override(); + rte_cryptodev_pmd_probing_finish(dev); + return 0; dev_fini: diff --git a/drivers/crypto/cnxk/cn9k_cryptodev.c b/drivers/crypto/cnxk/cn9k_cryptodev.c index 54df06eec0..54cd77e575 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev.c @@ -102,6 +102,8 @@ cn9k_cpt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, cn9k_cpt_set_enqdeq_fns(dev); cn9k_sec_ops_override(); + rte_cryptodev_pmd_probing_finish(dev); + return 0; dev_fini: diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index 4eb3615250..cb2ad435bf 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -4032,8 +4032,10 @@ cryptodev_dpaa2_sec_probe(struct rte_dpaa2_driver *dpaa2_drv __rte_unused, /* Invoke PMD device initialization function */ retval = dpaa2_sec_dev_init(cryptodev); - if (retval == 0) + if (retval == 0) { + rte_cryptodev_pmd_probing_finish(cryptodev); return 0; + } if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_free(cryptodev->data->dev_private); diff --git a/drivers/crypto/dpaa_sec/dpaa_sec.c b/drivers/crypto/dpaa_sec/dpaa_sec.c index c7ef1c7b0f..454b9c4785 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec.c +++ b/drivers/crypto/dpaa_sec/dpaa_sec.c @@ -3594,8 +3594,10 @@ cryptodev_dpaa_sec_probe(struct rte_dpaa_driver *dpaa_drv __rte_unused, /* Invoke PMD device initialization function */ retval = dpaa_sec_dev_init(cryptodev); - if (retval == 0) + if (retval == 0) { + rte_cryptodev_pmd_probing_finish(cryptodev); return 0; + } retval = -ENXIO; out: diff --git a/drivers/crypto/ipsec_mb/ipsec_mb_private.c b/drivers/crypto/ipsec_mb/ipsec_mb_private.c index d896fa20b4..aab42c360c 100644 --- a/drivers/crypto/ipsec_mb/ipsec_mb_private.c +++ b/drivers/crypto/ipsec_mb/ipsec_mb_private.c @@ -134,6 +134,8 @@ ipsec_mb_create(struct rte_vdev_device *vdev, break; } + rte_cryptodev_pmd_probing_finish(dev); + IPSEC_MB_LOG(INFO, "IPSec Multi-buffer library version used: %s\n", imb_get_version_str()); diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 6a2f8b6ac6..14b6783e13 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -1061,6 +1061,9 @@ mlx5_crypto_dev_probe(struct rte_device *dev) pthread_mutex_lock(&priv_list_lock); TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next); pthread_mutex_unlock(&priv_list_lock); + + rte_cryptodev_pmd_probing_finish(crypto_dev); + return 0; } diff --git a/drivers/crypto/mvsam/rte_mrvl_pmd.c b/drivers/crypto/mvsam/rte_mrvl_pmd.c index a72642a772..04efd9aaa8 100644 --- a/drivers/crypto/mvsam/rte_mrvl_pmd.c +++ b/drivers/crypto/mvsam/rte_mrvl_pmd.c @@ -1114,6 +1114,8 @@ cryptodev_mrvl_crypto_create(const char *name, if (ret) goto init_error; + rte_cryptodev_pmd_probing_finish(dev); + return 0; init_error: diff --git a/drivers/crypto/nitrox/nitrox_sym.c b/drivers/crypto/nitrox/nitrox_sym.c index f8b7edcd69..cb5393d2f1 100644 --- a/drivers/crypto/nitrox/nitrox_sym.c +++ b/drivers/crypto/nitrox/nitrox_sym.c @@ -783,6 +783,9 @@ nitrox_sym_pmd_create(struct nitrox_device *ndev) ndev->sym_dev = cdev->data->dev_private; ndev->sym_dev->cdev = cdev; ndev->sym_dev->ndev = ndev; + + rte_cryptodev_pmd_probing_finish(cdev); + NITROX_LOG(DEBUG, "Created cryptodev '%s', dev_id %d, drv_id %d\n", cdev->data->name, cdev->data->dev_id, nitrox_sym_drv_id); return 0; diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c index f9935d52cc..9ecb434fd0 100644 --- a/drivers/crypto/null/null_crypto_pmd.c +++ b/drivers/crypto/null/null_crypto_pmd.c @@ -184,6 +184,8 @@ cryptodev_null_create(const char *name, internals->max_nb_qpairs = init_params->max_nb_queue_pairs; + rte_cryptodev_pmd_probing_finish(dev); + return 0; } diff --git a/drivers/crypto/octeontx/otx_cryptodev.c b/drivers/crypto/octeontx/otx_cryptodev.c index c294f86d79..05b78329d6 100644 --- a/drivers/crypto/octeontx/otx_cryptodev.c +++ b/drivers/crypto/octeontx/otx_cryptodev.c @@ -56,8 +56,10 @@ otx_cpt_pci_probe(struct rte_pci_driver *pci_drv, /* Invoke PMD device initialization function */ retval = otx_cpt_dev_create(cryptodev); - if (retval == 0) + if (retval == 0) { + rte_cryptodev_pmd_probing_finish(cryptodev); return 0; + } CPT_LOG_ERR("[DRV %s]: Failed to create device " "(vendor_id: 0x%x device_id: 0x%x", diff --git a/drivers/crypto/octeontx2/otx2_cryptodev.c b/drivers/crypto/octeontx2/otx2_cryptodev.c index 85b1f00263..fc7ad05366 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev.c @@ -135,6 +135,8 @@ otx2_cpt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, if (rte_eal_process_type() == RTE_PROC_SECONDARY) otx2_cpt_set_enqdeq_fns(dev); + rte_cryptodev_pmd_probing_finish(dev); + return 0; otx2_dev_fini: diff --git a/drivers/crypto/openssl/rte_openssl_pmd.c b/drivers/crypto/openssl/rte_openssl_pmd.c index 13c6ea8724..5794ed8159 100644 --- a/drivers/crypto/openssl/rte_openssl_pmd.c +++ b/drivers/crypto/openssl/rte_openssl_pmd.c @@ -2213,6 +2213,8 @@ cryptodev_openssl_create(const char *name, internals->max_nb_qpairs = init_params->max_nb_queue_pairs; + rte_cryptodev_pmd_probing_finish(dev); + return 0; init_error: diff --git a/drivers/crypto/qat/qat_asym_pmd.c b/drivers/crypto/qat/qat_asym_pmd.c index e91bb0d317..0944d27a4d 100644 --- a/drivers/crypto/qat/qat_asym_pmd.c +++ b/drivers/crypto/qat/qat_asym_pmd.c @@ -343,6 +343,9 @@ qat_asym_dev_create(struct qat_pci_device *qat_pci_dev, } qat_pci_dev->asym_dev = internals; + + rte_cryptodev_pmd_probing_finish(cryptodev); + QAT_LOG(DEBUG, "Created QAT ASYM device %s as cryptodev instance %d", cryptodev->data->name, internals->asym_dev_id); return 0; diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c index efda921c05..d4f087733f 100644 --- a/drivers/crypto/qat/qat_sym_pmd.c +++ b/drivers/crypto/qat/qat_sym_pmd.c @@ -500,6 +500,8 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev, QAT_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d", cryptodev->data->name, internals->sym_dev_id); + rte_cryptodev_pmd_probing_finish(cryptodev); + return 0; error: diff --git a/drivers/crypto/scheduler/scheduler_pmd.c b/drivers/crypto/scheduler/scheduler_pmd.c index 560c26af50..dd198080bf 100644 --- a/drivers/crypto/scheduler/scheduler_pmd.c +++ b/drivers/crypto/scheduler/scheduler_pmd.c @@ -229,6 +229,8 @@ cryptodev_scheduler_create(const char *name, return -ENOMEM; } + rte_cryptodev_pmd_probing_finish(dev); + return 0; } diff --git a/drivers/crypto/virtio/virtio_cryptodev.c b/drivers/crypto/virtio/virtio_cryptodev.c index 8faa39df4a..ed64866758 100644 --- a/drivers/crypto/virtio/virtio_cryptodev.c +++ b/drivers/crypto/virtio/virtio_cryptodev.c @@ -754,6 +754,8 @@ crypto_virtio_create(const char *name, struct rte_pci_device *pci_dev, VIRTIO_CRYPTO_PMD_GUEST_FEATURES) < 0) return -1; + rte_cryptodev_pmd_probing_finish(cryptodev); + return 0; } From patchwork Wed Oct 20 11:27:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 102427 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 87612A0C43; Wed, 20 Oct 2021 13:29:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 933F641196; Wed, 20 Oct 2021 13:29:03 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 0062641196 for ; Wed, 20 Oct 2021 13:29:01 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19K8Is3Y021017; Wed, 20 Oct 2021 04:28:56 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=6rgK3OKHN4x8oJSLNbp6QUsAtojPf63rYAdTFaEhwl0=; b=XpT1dcdlQolc82Hz9bGlfY/hXvgQNF5Gpa8PAHT0U1G+lL3TvnXjwJDVwEuL+Yc6AhxV JtNeUGKeQDxq1VSy+RWWtrkSbpl6vTJ0R9rYQK0jT2YjkPdD2YrCM5zfqCGFAsxOrRTd KNBnZWrLAlORJhTyIm2bZ78wV2fuSbc3nlq4ClcsL/j+9kMqBnpo3Usjr88UwmP5MQQC a5yW8BAjyjsMf10M71A9LRzH2PHiAcqUzOu8ouy/cw1GtCy5b/sEKQn4T1z5xIg6nEqm p7bEpGT3rcEZGgOCJvCSlq+CwLrMIGjz0Jz/8MBOEj2UEXV4uvZa/xie3H9SDcaZOKMb 8Q== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 3bt05g4q02-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 04:28:56 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 04:28:54 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 20 Oct 2021 04:28:54 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id 57F2F3F707F; Wed, 20 Oct 2021 04:28:48 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal Date: Wed, 20 Oct 2021 16:57:53 +0530 Message-ID: <20211020112754.1270163-8-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211020112754.1270163-1-gakhil@marvell.com> References: <20211018144201.2028022-1-gakhil@marvell.com> <20211020112754.1270163-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: H5QZl_OJW0Bin-HZjuNoY-0er9Hx0v4Q X-Proofpoint-ORIG-GUID: H5QZl_OJW0Bin-HZjuNoY-0er9Hx0v4Q X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-20_04,2021-10-20_02,2020-04-07_01 Subject: [dpdk-dev] [PATCH v4 7/8] cryptodev: update fast path APIs to use new flat array X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Rework fast-path cryptodev functions to use rte_crypto_fp_ops[]. While it is an API/ABI breakage, this change is intended to be transparent for both users (no changes in user app is required) and PMD developers (no changes in PMD is required). Signed-off-by: Akhil Goyal Acked-by: Fan Zhang Acked-by: Konstantin Ananyev --- lib/cryptodev/rte_cryptodev.h | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/lib/cryptodev/rte_cryptodev.h b/lib/cryptodev/rte_cryptodev.h index ce0dca72be..56e3868ada 100644 --- a/lib/cryptodev/rte_cryptodev.h +++ b/lib/cryptodev/rte_cryptodev.h @@ -1832,13 +1832,18 @@ static inline uint16_t rte_cryptodev_dequeue_burst(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op **ops, uint16_t nb_ops) { - struct rte_cryptodev *dev = &rte_cryptodevs[dev_id]; + const struct rte_crypto_fp_ops *fp_ops; + void *qp; rte_cryptodev_trace_dequeue_burst(dev_id, qp_id, (void **)ops, nb_ops); - nb_ops = (*dev->dequeue_burst) - (dev->data->queue_pairs[qp_id], ops, nb_ops); + + fp_ops = &rte_crypto_fp_ops[dev_id]; + qp = fp_ops->qp.data[qp_id]; + + nb_ops = fp_ops->dequeue_burst(qp, ops, nb_ops); + #ifdef RTE_CRYPTO_CALLBACKS - if (unlikely(dev->deq_cbs != NULL)) { + if (unlikely(fp_ops->qp.deq_cb != NULL)) { struct rte_cryptodev_cb_rcu *list; struct rte_cryptodev_cb *cb; @@ -1848,7 +1853,7 @@ rte_cryptodev_dequeue_burst(uint8_t dev_id, uint16_t qp_id, * cb and cb->fn/cb->next, __ATOMIC_ACQUIRE memory order is * not required. */ - list = &dev->deq_cbs[qp_id]; + list = &fp_ops->qp.deq_cb[qp_id]; rte_rcu_qsbr_thread_online(list->qsbr, 0); cb = __atomic_load_n(&list->next, __ATOMIC_RELAXED); @@ -1899,10 +1904,13 @@ static inline uint16_t rte_cryptodev_enqueue_burst(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op **ops, uint16_t nb_ops) { - struct rte_cryptodev *dev = &rte_cryptodevs[dev_id]; + const struct rte_crypto_fp_ops *fp_ops; + void *qp; + fp_ops = &rte_crypto_fp_ops[dev_id]; + qp = fp_ops->qp.data[qp_id]; #ifdef RTE_CRYPTO_CALLBACKS - if (unlikely(dev->enq_cbs != NULL)) { + if (unlikely(fp_ops->qp.enq_cb != NULL)) { struct rte_cryptodev_cb_rcu *list; struct rte_cryptodev_cb *cb; @@ -1912,7 +1920,7 @@ rte_cryptodev_enqueue_burst(uint8_t dev_id, uint16_t qp_id, * cb and cb->fn/cb->next, __ATOMIC_ACQUIRE memory order is * not required. */ - list = &dev->enq_cbs[qp_id]; + list = &fp_ops->qp.enq_cb[qp_id]; rte_rcu_qsbr_thread_online(list->qsbr, 0); cb = __atomic_load_n(&list->next, __ATOMIC_RELAXED); @@ -1927,8 +1935,7 @@ rte_cryptodev_enqueue_burst(uint8_t dev_id, uint16_t qp_id, #endif rte_cryptodev_trace_enqueue_burst(dev_id, qp_id, (void **)ops, nb_ops); - return (*dev->enqueue_burst)( - dev->data->queue_pairs[qp_id], ops, nb_ops); + return fp_ops->enqueue_burst(qp, ops, nb_ops); } From patchwork Wed Oct 20 11:27:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 102428 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BB57DA0C43; Wed, 20 Oct 2021 13:29:12 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DB339411F2; Wed, 20 Oct 2021 13:29:09 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 8929340142 for ; Wed, 20 Oct 2021 13:29:08 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19K84Xms021040; Wed, 20 Oct 2021 04:29:04 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=mia7aV3q5K09J5OiFcfpKo85e0K9H0Pe6FFafNb0j+k=; b=b2JUVMVFA2BEkd9ECFVDIQev0wjHXjWbsBYNesTDTDrDKZCUjIcm28OYqT9dvQ784JJE WDO7N8Zak1Z07wR4j5OMDeuFOUoiB7a0XwXoqbbB1XgsALd+nJfccJRFC54ag85pnr8c +OTRClR4ASlYoZAx4+XM+mu798JhED69TTCTMx5B94HCiOFESuRue0tuOP9mkrsWg1Jr P11Jz6ZkiwGY5VoPFadYC6wpGPaO+FbU/g/pErQwoLVv+vLOBeIv5EwoFlwUUiPY3VWv BFzB8WArYfKPQIZwNwHSS51iGUu9RkIoAGOPgMbJDK7H2jVOSvBnhiuA9/M3m3Ome6FW zg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 3bt05g4q0h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 04:29:04 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 04:29:02 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 20 Oct 2021 04:29:02 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id D82D23F7080; Wed, 20 Oct 2021 04:28:54 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal , Rebecca Troy Date: Wed, 20 Oct 2021 16:57:54 +0530 Message-ID: <20211020112754.1270163-9-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211020112754.1270163-1-gakhil@marvell.com> References: <20211018144201.2028022-1-gakhil@marvell.com> <20211020112754.1270163-1-gakhil@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: bZ-8BfsH5eNQP6h3e0j9yYarmryO_XMk X-Proofpoint-ORIG-GUID: bZ-8BfsH5eNQP6h3e0j9yYarmryO_XMk X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-20_04,2021-10-20_02,2020-04-07_01 Subject: [dpdk-dev] [PATCH v4 8/8] cryptodev: move device specific structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The device specific structures - rte_cryptodev and rte_cryptodev_data are moved to cryptodev_pmd.h to hide it from the applications. Signed-off-by: Akhil Goyal Tested-by: Rebecca Troy Acked-by: Fan Zhang Acked-by: Konstantin Ananyev --- doc/guides/rel_notes/release_21_11.rst | 6 ++ drivers/crypto/ccp/ccp_dev.h | 2 +- drivers/crypto/cnxk/cn10k_ipsec.c | 2 +- drivers/crypto/cnxk/cn9k_ipsec.c | 2 +- .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 2 +- drivers/crypto/cnxk/cnxk_cryptodev_sec.c | 2 +- drivers/crypto/nitrox/nitrox_sym_reqmgr.c | 2 +- drivers/crypto/octeontx/otx_cryptodev.c | 1 - .../crypto/octeontx/otx_cryptodev_hw_access.c | 2 +- .../crypto/octeontx/otx_cryptodev_hw_access.h | 2 +- drivers/crypto/octeontx/otx_cryptodev_ops.h | 2 +- .../crypto/octeontx2/otx2_cryptodev_mbox.c | 2 +- drivers/crypto/scheduler/scheduler_failover.c | 2 +- .../crypto/scheduler/scheduler_multicore.c | 2 +- .../scheduler/scheduler_pkt_size_distr.c | 2 +- .../crypto/scheduler/scheduler_roundrobin.c | 2 +- drivers/event/cnxk/cnxk_eventdev.h | 2 +- drivers/event/dpaa/dpaa_eventdev.c | 2 +- drivers/event/dpaa2/dpaa2_eventdev.c | 2 +- drivers/event/octeontx/ssovf_evdev.c | 2 +- .../event/octeontx2/otx2_evdev_crypto_adptr.c | 2 +- lib/cryptodev/cryptodev_pmd.h | 65 ++++++++++++++++++ lib/cryptodev/rte_cryptodev_core.h | 67 ------------------- lib/cryptodev/version.map | 2 +- 24 files changed, 91 insertions(+), 88 deletions(-) diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index faa9164546..23bc854d16 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -328,6 +328,12 @@ ABI Changes Also, make sure to start the actual text at the margin. ======================================================= +* cryptodev: Made ``rte_cryptodev``, ``rte_cryptodev_data`` private + structures internal to DPDK. ``rte_cryptodevs`` can't be accessed directly + by user any more. While it is an ABI breakage, this change is intended + to be transparent for both users (no changes in user app is required) and + PMD developers (no changes in PMD is required). + * security: ``rte_security_set_pkt_metadata`` and ``rte_security_get_userdata`` routines used by inline outbound and inline inbound security processing were made inline and enhanced to do simple 64-bit set/get for PMDs that do not diff --git a/drivers/crypto/ccp/ccp_dev.h b/drivers/crypto/ccp/ccp_dev.h index ca5145c278..85c8fc47a2 100644 --- a/drivers/crypto/ccp/ccp_dev.h +++ b/drivers/crypto/ccp/ccp_dev.h @@ -17,7 +17,7 @@ #include #include #include -#include +#include /**< CCP sspecific */ #define MAX_HW_QUEUES 5 diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c index defc792aa8..27df1dcd64 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.c +++ b/drivers/crypto/cnxk/cn10k_ipsec.c @@ -3,7 +3,7 @@ */ #include -#include +#include #include #include #include diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c index 9ca4d20c62..53fb793654 100644 --- a/drivers/crypto/cnxk/cn9k_ipsec.c +++ b/drivers/crypto/cnxk/cn9k_ipsec.c @@ -2,7 +2,7 @@ * Copyright(C) 2021 Marvell. */ -#include +#include #include #include #include diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c index a227e6981c..a53b489a04 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c @@ -2,7 +2,7 @@ * Copyright(C) 2021 Marvell. */ -#include +#include #include #include "roc_api.h" diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_sec.c b/drivers/crypto/cnxk/cnxk_cryptodev_sec.c index 8d04d4b575..2021d5c77e 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_sec.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_sec.c @@ -2,7 +2,7 @@ * Copyright(C) 2021 Marvell. */ -#include +#include #include #include #include diff --git a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c index fe3ca25a0c..9edb0cc00f 100644 --- a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c +++ b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c @@ -3,7 +3,7 @@ */ #include -#include +#include #include #include diff --git a/drivers/crypto/octeontx/otx_cryptodev.c b/drivers/crypto/octeontx/otx_cryptodev.c index 05b78329d6..337d06aab8 100644 --- a/drivers/crypto/octeontx/otx_cryptodev.c +++ b/drivers/crypto/octeontx/otx_cryptodev.c @@ -4,7 +4,6 @@ #include #include -#include #include #include #include diff --git a/drivers/crypto/octeontx/otx_cryptodev_hw_access.c b/drivers/crypto/octeontx/otx_cryptodev_hw_access.c index 7b89a62d81..20b288334a 100644 --- a/drivers/crypto/octeontx/otx_cryptodev_hw_access.c +++ b/drivers/crypto/octeontx/otx_cryptodev_hw_access.c @@ -7,7 +7,7 @@ #include #include -#include +#include #include #include #include diff --git a/drivers/crypto/octeontx/otx_cryptodev_hw_access.h b/drivers/crypto/octeontx/otx_cryptodev_hw_access.h index 7c6b1e45b4..e48805fb09 100644 --- a/drivers/crypto/octeontx/otx_cryptodev_hw_access.h +++ b/drivers/crypto/octeontx/otx_cryptodev_hw_access.h @@ -7,7 +7,7 @@ #include #include -#include +#include #include #include #include diff --git a/drivers/crypto/octeontx/otx_cryptodev_ops.h b/drivers/crypto/octeontx/otx_cryptodev_ops.h index f234f16970..83b82ea059 100644 --- a/drivers/crypto/octeontx/otx_cryptodev_ops.h +++ b/drivers/crypto/octeontx/otx_cryptodev_ops.h @@ -5,7 +5,7 @@ #ifndef _OTX_CRYPTODEV_OPS_H_ #define _OTX_CRYPTODEV_OPS_H_ -#include +#include #define OTX_CPT_MIN_HEADROOM_REQ (24) #define OTX_CPT_MIN_TAILROOM_REQ (8) diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c index 1a8edae7eb..f9e7b0b474 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright (C) 2019 Marvell International Ltd. */ -#include +#include #include #include "otx2_cryptodev.h" diff --git a/drivers/crypto/scheduler/scheduler_failover.c b/drivers/crypto/scheduler/scheduler_failover.c index 844312dd1b..5023577ef8 100644 --- a/drivers/crypto/scheduler/scheduler_failover.c +++ b/drivers/crypto/scheduler/scheduler_failover.c @@ -2,7 +2,7 @@ * Copyright(c) 2017 Intel Corporation */ -#include +#include #include #include "rte_cryptodev_scheduler_operations.h" diff --git a/drivers/crypto/scheduler/scheduler_multicore.c b/drivers/crypto/scheduler/scheduler_multicore.c index 1e2e8dbf9f..900ab4049d 100644 --- a/drivers/crypto/scheduler/scheduler_multicore.c +++ b/drivers/crypto/scheduler/scheduler_multicore.c @@ -3,7 +3,7 @@ */ #include -#include +#include #include #include "rte_cryptodev_scheduler_operations.h" diff --git a/drivers/crypto/scheduler/scheduler_pkt_size_distr.c b/drivers/crypto/scheduler/scheduler_pkt_size_distr.c index 57e330a744..933a5c6978 100644 --- a/drivers/crypto/scheduler/scheduler_pkt_size_distr.c +++ b/drivers/crypto/scheduler/scheduler_pkt_size_distr.c @@ -2,7 +2,7 @@ * Copyright(c) 2017 Intel Corporation */ -#include +#include #include #include "rte_cryptodev_scheduler_operations.h" diff --git a/drivers/crypto/scheduler/scheduler_roundrobin.c b/drivers/crypto/scheduler/scheduler_roundrobin.c index bc4a632106..ace2dec2ec 100644 --- a/drivers/crypto/scheduler/scheduler_roundrobin.c +++ b/drivers/crypto/scheduler/scheduler_roundrobin.c @@ -2,7 +2,7 @@ * Copyright(c) 2017 Intel Corporation */ -#include +#include #include #include "rte_cryptodev_scheduler_operations.h" diff --git a/drivers/event/cnxk/cnxk_eventdev.h b/drivers/event/cnxk/cnxk_eventdev.h index 8a5c737e4b..b57004c0dc 100644 --- a/drivers/event/cnxk/cnxk_eventdev.h +++ b/drivers/event/cnxk/cnxk_eventdev.h @@ -7,7 +7,7 @@ #include -#include +#include #include #include #include diff --git a/drivers/event/dpaa/dpaa_eventdev.c b/drivers/event/dpaa/dpaa_eventdev.c index ec74160325..1d7ddfe1d1 100644 --- a/drivers/event/dpaa/dpaa_eventdev.c +++ b/drivers/event/dpaa/dpaa_eventdev.c @@ -28,7 +28,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/drivers/event/dpaa2/dpaa2_eventdev.c b/drivers/event/dpaa2/dpaa2_eventdev.c index 5ccf22f77f..e03afb2958 100644 --- a/drivers/event/dpaa2/dpaa2_eventdev.c +++ b/drivers/event/dpaa2/dpaa2_eventdev.c @@ -25,7 +25,7 @@ #include #include #include -#include +#include #include #include diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index b93f6ec8c6..9846fce34b 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -5,7 +5,7 @@ #include #include -#include +#include #include #include #include diff --git a/drivers/event/octeontx2/otx2_evdev_crypto_adptr.c b/drivers/event/octeontx2/otx2_evdev_crypto_adptr.c index d9a002625c..d59d6c53f6 100644 --- a/drivers/event/octeontx2/otx2_evdev_crypto_adptr.c +++ b/drivers/event/octeontx2/otx2_evdev_crypto_adptr.c @@ -2,7 +2,7 @@ * Copyright (C) 2020-2021 Marvell. */ -#include +#include #include #include "otx2_cryptodev.h" diff --git a/lib/cryptodev/cryptodev_pmd.h b/lib/cryptodev/cryptodev_pmd.h index 9bb1e47ae4..89bf2af399 100644 --- a/lib/cryptodev/cryptodev_pmd.h +++ b/lib/cryptodev/cryptodev_pmd.h @@ -52,6 +52,71 @@ struct rte_cryptodev_pmd_init_params { unsigned int max_nb_queue_pairs; }; +/** + * @internal + * The data part, with no function pointers, associated with each device. + * + * This structure is safe to place in shared memory to be common among + * different processes in a multi-process configuration. + */ +struct rte_cryptodev_data { + /** Device ID for this instance */ + uint8_t dev_id; + /** Socket ID where memory is allocated */ + uint8_t socket_id; + /** Unique identifier name */ + char name[RTE_CRYPTODEV_NAME_MAX_LEN]; + + __extension__ + /** Device state: STARTED(1)/STOPPED(0) */ + uint8_t dev_started : 1; + + /** Session memory pool */ + struct rte_mempool *session_pool; + /** Array of pointers to queue pairs. */ + void **queue_pairs; + /** Number of device queue pairs. */ + uint16_t nb_queue_pairs; + + /** PMD-specific private data */ + void *dev_private; +} __rte_cache_aligned; + +/** @internal The data structure associated with each crypto device. */ +struct rte_cryptodev { + /** Pointer to PMD dequeue function. */ + dequeue_pkt_burst_t dequeue_burst; + /** Pointer to PMD enqueue function. */ + enqueue_pkt_burst_t enqueue_burst; + + /** Pointer to device data */ + struct rte_cryptodev_data *data; + /** Functions exported by PMD */ + struct rte_cryptodev_ops *dev_ops; + /** Feature flags exposes HW/SW features for the given device */ + uint64_t feature_flags; + /** Backing device */ + struct rte_device *device; + + /** Crypto driver identifier*/ + uint8_t driver_id; + + /** User application callback for interrupts if present */ + struct rte_cryptodev_cb_list link_intr_cbs; + + /** Context for security ops */ + void *security_ctx; + + __extension__ + /** Flag indicating the device is attached */ + uint8_t attached : 1; + + /** User application callback for pre enqueue processing */ + struct rte_cryptodev_cb_rcu *enq_cbs; + /** User application callback for post dequeue processing */ + struct rte_cryptodev_cb_rcu *deq_cbs; +} __rte_cache_aligned; + /** Global structure used for maintaining state of allocated crypto devices */ struct rte_cryptodev_global { struct rte_cryptodev *devs; /**< Device information array */ diff --git a/lib/cryptodev/rte_cryptodev_core.h b/lib/cryptodev/rte_cryptodev_core.h index 2bb9a228c1..16832f645d 100644 --- a/lib/cryptodev/rte_cryptodev_core.h +++ b/lib/cryptodev/rte_cryptodev_core.h @@ -54,73 +54,6 @@ struct rte_crypto_fp_ops { extern struct rte_crypto_fp_ops rte_crypto_fp_ops[RTE_CRYPTO_MAX_DEVS]; -/** - * @internal - * The data part, with no function pointers, associated with each device. - * - * This structure is safe to place in shared memory to be common among - * different processes in a multi-process configuration. - */ -struct rte_cryptodev_data { - uint8_t dev_id; - /**< Device ID for this instance */ - uint8_t socket_id; - /**< Socket ID where memory is allocated */ - char name[RTE_CRYPTODEV_NAME_MAX_LEN]; - /**< Unique identifier name */ - - __extension__ - uint8_t dev_started : 1; - /**< Device state: STARTED(1)/STOPPED(0) */ - - struct rte_mempool *session_pool; - /**< Session memory pool */ - void **queue_pairs; - /**< Array of pointers to queue pairs. */ - uint16_t nb_queue_pairs; - /**< Number of device queue pairs. */ - - void *dev_private; - /**< PMD-specific private data */ -} __rte_cache_aligned; - - -/** @internal The data structure associated with each crypto device. */ -struct rte_cryptodev { - dequeue_pkt_burst_t dequeue_burst; - /**< Pointer to PMD receive function. */ - enqueue_pkt_burst_t enqueue_burst; - /**< Pointer to PMD transmit function. */ - - struct rte_cryptodev_data *data; - /**< Pointer to device data */ - struct rte_cryptodev_ops *dev_ops; - /**< Functions exported by PMD */ - uint64_t feature_flags; - /**< Feature flags exposes HW/SW features for the given device */ - struct rte_device *device; - /**< Backing device */ - - uint8_t driver_id; - /**< Crypto driver identifier*/ - - struct rte_cryptodev_cb_list link_intr_cbs; - /**< User application callback for interrupts if present */ - - void *security_ctx; - /**< Context for security ops */ - - __extension__ - uint8_t attached : 1; - /**< Flag indicating the device is attached */ - - struct rte_cryptodev_cb_rcu *enq_cbs; - /**< User application callback for pre enqueue processing */ - - struct rte_cryptodev_cb_rcu *deq_cbs; - /**< User application callback for post dequeue processing */ -} __rte_cache_aligned; - /** * The pool of rte_cryptodev structures. */ diff --git a/lib/cryptodev/version.map b/lib/cryptodev/version.map index 157dac521d..b55b4b8e7e 100644 --- a/lib/cryptodev/version.map +++ b/lib/cryptodev/version.map @@ -43,7 +43,6 @@ DPDK_22 { rte_cryptodev_sym_session_create; rte_cryptodev_sym_session_free; rte_cryptodev_sym_session_init; - rte_cryptodevs; #added in 21.11 rte_crypto_fp_ops; @@ -125,4 +124,5 @@ INTERNAL { rte_cryptodev_pmd_parse_input_args; rte_cryptodev_pmd_probing_finish; rte_cryptodev_pmd_release_device; + rte_cryptodevs; };