From patchwork Mon Oct 21 01:52:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wathsala Vithanage X-Patchwork-Id: 146339 X-Patchwork-Delegate: stephen@networkplumber.org Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ACED445B8B; Mon, 21 Oct 2024 03:53:06 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 79320402E1; Mon, 21 Oct 2024 03:53:02 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id B0A384021F for ; Mon, 21 Oct 2024 03:52:59 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C83B1FEC; Sun, 20 Oct 2024 18:53:28 -0700 (PDT) Received: from ampere-altra-2-1.usa.Arm.com (ampere-altra-2-1.usa.arm.com [10.118.91.158]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 11FBC3F73B; Sun, 20 Oct 2024 18:52:59 -0700 (PDT) From: Wathsala Vithanage To: Chenbo Xia , Nipun Gupta , Gaetan Rivet Cc: dev@dpdk.org, nd@arm.com, Wathsala Vithanage , Honnappa Nagarahalli , Dhruv Tripathi Subject: [RFC v3 1/2] pci: introduce the PCIe TLP Processing Hints API Date: Mon, 21 Oct 2024 01:52:45 +0000 Message-Id: <20241021015246.304431-2-wathsala.vithanage@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021015246.304431-1-wathsala.vithanage@arm.com> References: <20240715221141.16153-1-wathsala.vithanage@arm.com> <20241021015246.304431-1-wathsala.vithanage@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Extend the PCI driver and the library to extract the Steering Tag (ST) for a given Processor/Processor Container and Cache ID pair and validate a Processing Hint from a TPH _DSM associated with a root port device. The rte_pci_device structure passed into the rte_pci_extract_tph_st() function could be a device or a root port. If it's a device, the function should trace it back to the root port and use its TPH _DSM to extract STs. The implementation of rte_pci_extract_tph_st() is dependent on the operating system. rte_pci_extract_tph_st() should also be supplied with a rte_tph_acpi__dsm_args, and a rte_tph_acpi__dsm_return structures. These two structures are defined in the PCI library and comply with the TPH _DSM argument and return encoding specified in the PCI firmware ECN titled "Revised _DSM for Cache Locality TPH Features.". Use of rte_init_tph_acpi__dsm_args() is recommended for initializing the rte_tph_acpi__dsm_args struct which is capable of converting lcore ID, the cache level into values understood by the ACPI _DSM function. rte_tph_acpi__dsm_return struct will be initialized with the values returned by the TPH _DSM; it is up to the caller to use these values per the device's capabilities. Signed-off-by: Wathsala Vithanage Reviewed-by: Honnappa Nagarahalli Reviewed-by: Dhruv Tripathi --- drivers/bus/pci/bsd/pci.c | 12 ++++ drivers/bus/pci/linux/pci.c | 12 ++++ drivers/bus/pci/rte_bus_pci.h | 22 +++++++ drivers/bus/pci/version.map | 3 + drivers/bus/pci/windows/pci.c | 14 +++++ lib/pci/meson.build | 2 + lib/pci/rte_pci.h | 2 + lib/pci/rte_pci_tph.c | 21 +++++++ lib/pci/rte_pci_tph.h | 111 ++++++++++++++++++++++++++++++++++ 9 files changed, 199 insertions(+) create mode 100644 lib/pci/rte_pci_tph.c create mode 100644 lib/pci/rte_pci_tph.h diff --git a/drivers/bus/pci/bsd/pci.c b/drivers/bus/pci/bsd/pci.c index 2f88252418..a143cecf45 100644 --- a/drivers/bus/pci/bsd/pci.c +++ b/drivers/bus/pci/bsd/pci.c @@ -639,3 +639,15 @@ rte_pci_ioport_unmap(struct rte_pci_ioport *p) return ret; } + +int +rte_pci_extract_tph_st(const struct rte_pci_device *dev, + const struct rte_tph_acpi__dsm_args *args, + struct rte_tph_acpi__dsm_return *ret) +{ + RTE_SET_USED(dev); + RTE_SET_USED(args); + RTE_SET_USED(ret); + /* BSD doesn't support this feature yet! */ + return -1; +} diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c index 9056035b33..dffb945462 100644 --- a/drivers/bus/pci/linux/pci.c +++ b/drivers/bus/pci/linux/pci.c @@ -803,3 +803,15 @@ rte_pci_ioport_unmap(struct rte_pci_ioport *p) return ret; } + +int +rte_pci_extract_tph_st(const struct rte_pci_device *dev, + const struct rte_tph_acpi__dsm_args *args, + struct rte_tph_acpi__dsm_return *ret) +{ + RTE_SET_USED(dev); + RTE_SET_USED(args); + RTE_SET_USED(ret); + /* Linux doesn't support this feature yet! */ + return -1; +} diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h index 19a7b15b99..a8167e9b4b 100644 --- a/drivers/bus/pci/rte_bus_pci.h +++ b/drivers/bus/pci/rte_bus_pci.h @@ -312,6 +312,28 @@ void rte_pci_ioport_read(struct rte_pci_ioport *p, void rte_pci_ioport_write(struct rte_pci_ioport *p, const void *data, size_t len, off_t offset); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Extract steering tag from the ACPI TPH _DSM of the root port + * of the device is connected to. + * + * @param device + * A pointer to a rte_pci_device structure describing the device + * to use. + * @param args + * An initialized args object for the _DSM. + * @param ret + * A pointer to a _DSM return object to store the extracted steering tag. + * @return + * 0 on success, -1 on error extracting the steeting tag. + */ +__rte_experimental +int rte_pci_extract_tph_st(const struct rte_pci_device *device, + const struct rte_tph_acpi__dsm_args *args, + struct rte_tph_acpi__dsm_return *ret); + #ifdef __cplusplus } #endif diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map index cd653de5ac..5c89f80c8e 100644 --- a/drivers/bus/pci/version.map +++ b/drivers/bus/pci/version.map @@ -31,6 +31,9 @@ EXPERIMENTAL { rte_pci_find_capability; rte_pci_find_next_capability; rte_pci_has_capability_list; + + # added in 24.11 + rte_pci_extract_tph_st; }; INTERNAL { diff --git a/drivers/bus/pci/windows/pci.c b/drivers/bus/pci/windows/pci.c index 36e6f89093..761f714a18 100644 --- a/drivers/bus/pci/windows/pci.c +++ b/drivers/bus/pci/windows/pci.c @@ -500,3 +500,17 @@ rte_pci_scan(void) return ret; } + + +int +rte_pci_extract_tph_st(const struct rte_pci_device *dev, + const struct rte_tph_acpi__dsm_args *args, + struct rte_tph_acpi__dsm_return *ret) +{ + RTE_SET_USED(dev); + RTE_SET_USED(args); + RTE_SET_USED(ret); + /* This feature is not yet implemented for windows */ + return -1; +} + diff --git a/lib/pci/meson.build b/lib/pci/meson.build index dd41cd5068..85e17c4257 100644 --- a/lib/pci/meson.build +++ b/lib/pci/meson.build @@ -3,3 +3,5 @@ sources = files('rte_pci.c') headers = files('rte_pci.h') +headers = files('rte_pci_tph.h') +headers = files('rte_pci_tph.c') diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h index 9a50a12142..b7897640f1 100644 --- a/lib/pci/rte_pci.h +++ b/lib/pci/rte_pci.h @@ -16,6 +16,8 @@ #include #include +#include + #ifdef __cplusplus extern "C" { #endif diff --git a/lib/pci/rte_pci_tph.c b/lib/pci/rte_pci_tph.c new file mode 100644 index 0000000000..3b0c7d4d97 --- /dev/null +++ b/lib/pci/rte_pci_tph.c @@ -0,0 +1,21 @@ +#include +#include + +int +rte_init_tph_acpi__dsm_args(uint16_t lcore_id, uint8_t type, + uint8_t cache_level, uint8_t ph, + struct rte_tph_acpi__dsm_args *args) +{ + RTE_SET_USED(lcore_id); + RTE_SET_USED(type); + RTE_SET_USED(cache_level); + RTE_SET_USED(ph); + + if (!args) + return -EINVAL; + /* Use libhwloc or other mechanism provided by DPDK to + * map lcore_id and cache_level to hardware IDs for + * initializing args. + */ + return -ENOTSUP; +} diff --git a/lib/pci/rte_pci_tph.h b/lib/pci/rte_pci_tph.h new file mode 100644 index 0000000000..df851f5744 --- /dev/null +++ b/lib/pci/rte_pci_tph.h @@ -0,0 +1,111 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Arm Ltd. + */ + +#ifndef _RTE_PCI_TPH_H_ +#define _RTE_PCI_TPH_H_ + +/** + * @file + * + * RTE PCI TLP Processing Hints helpers + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +/** + * @warning + * @b EXPERIMENTAL: this structure may change, or be removed, without prior notice + * + * ACPI TPH _DSM input args structure. + * Refer to PCI-SIG ECN "Revised _DSM for Cache Locality TPH Features" for details. + */ +struct rte_tph_acpi__dsm_args { + uint32_t feature_id; /**< Always 0. */ + struct { + /** APIC/PPTT Processor/Processor container ID. */ + uint32_t uid; + } __rte_packed featureArg1; /**< 1st Arg. */ + struct { + /** Intended ph bits just for validating. */ + uint64_t ph : 2; + /** If type=1 uid is Processor container ID. */ + uint64_t type : 1; + /** cache_reference is valid if cache_ref_valid=1. */ + uint64_t cache_ref_valid : 1; + uint64_t reserved : 28; + /** PPTT cache ID of the desired target. */ + uint64_t cache_refernce : 32; + } __rte_packed featureArg2; /**< 2ns Arg. */ +} __rte_packed; + +/** + * @warning + * @b EXPERIMENTAL: this structure may change, or be removed, without prior notice + * + * ACPI TPH _DSM return structure. + * Refer to PCI-SIG ECN "Revised _DSM for Cache Locality TPH Features" for details. + */ +struct rte_tph_acpi__dsm_return { + uint64_t vmem_st_valid : 1; /**< if set to 1, vmem_st (8-bit ST) is valid. */ + /** if set to 1, vmem_ext_st (16-bit vmem ST) is valid. */ + uint64_t vmem_ext_st_valid : 1; + /** if set to 1, ph bits in input args is valid. */ + uint64_t vmem_ph_ignore : 1; + uint64_t reserved_1 : 5; + /** 8-bit volatile memory ST) */ + uint64_t vmem_st : 8; + /** 16-bit volatile ST) */ + uint64_t vmem_ext_st : 16; + uint64_t pmem_st_valid : 1; /**< if set to 1, pmem_st (8-bit ST) is valid. */ + /** if set to 1, pmem_ext_st (16-bit ST) is valid. */ + uint64_t pmem_ext_st_valid : 1; + /** if set to 1, ph bits in input args are valid for persistent memory. */ + uint64_t pmem_ph_ignore : 1; + uint64_t reserved_2 : 5; + /** 8-bit persistent memory ST) */ + uint64_t pmem_st : 8; + /** 16-bit persistent memory ST) */ + uint64_t pmem_ext_st : 16; +} __rte_packed; + + +/** + * + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * Initializes stashing hints configuration with a platform specific stashing hint + * that matches the lcore_id and cache_level. + * + * @param lcore_id + * The lcore_id of the processor of the cache stashing target. If is_container is set + * the target is the processor container of the CPU specified by the lcore_id. + * @param type + * If set to 1, the procssor container of the processor specified by lcore_id will be + * used at the stashing target. If set to 0, processor specified by the lcore_id will be + * used as the stashing target. + * @param cache_level + * The cache level of the processor/container specified by the lcore_id. + * @param ph + * TPH Processing Hints bits. + * @param args + * ACPI TPH _DSM object arguments structure. + * @return + * - (0) on Success. + * - 0 < or 0 > on Failure. + */ + +int rte_init_tph_acpi__dsm_args(uint16_t lcore_id, uint8_t type, + uint8_t cache_level, uint8_t ph, + struct rte_tph_acpi__dsm_args *args); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PCI_TPH_H_ */ From patchwork Mon Oct 21 01:52:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Wathsala Vithanage X-Patchwork-Id: 146340 X-Patchwork-Delegate: stephen@networkplumber.org Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CCFA445B8B; Mon, 21 Oct 2024 03:53:12 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C5AB1402E9; Mon, 21 Oct 2024 03:53:03 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id ED55F402CA for ; Mon, 21 Oct 2024 03:52:59 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 03AE71480; Sun, 20 Oct 2024 18:53:29 -0700 (PDT) Received: from ampere-altra-2-1.usa.Arm.com (ampere-altra-2-1.usa.arm.com [10.118.91.158]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 418C33F73B; Sun, 20 Oct 2024 18:52:59 -0700 (PDT) From: Wathsala Vithanage To: Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko Cc: dev@dpdk.org, nd@arm.com, Wathsala Vithanage , Honnappa Nagarahalli , Dhruv Tripathi Subject: [RFC v3 2/2] ethdev: introduce the cache stashing hints API Date: Mon, 21 Oct 2024 01:52:46 +0000 Message-Id: <20241021015246.304431-3-wathsala.vithanage@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021015246.304431-1-wathsala.vithanage@arm.com> References: <20240715221141.16153-1-wathsala.vithanage@arm.com> <20241021015246.304431-1-wathsala.vithanage@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Extend the ethdev library to enable the stashing of different data objects, such as the ones listed below, into CPU caches directly from the NIC. - Rx/Tx queue descriptors - Rx packets - Packet headers - packet payloads - Data of a packet at an offset from the start of the packet The APIs are designed in a hardware/vendor agnostic manner such that supporting PMDs could use any capabilities available in the underlying hardware for fine-grained stashing of data objects into a CPU cache (e.g., Steering Tags int PCIe TLP Processing Hints). The API provides an interface to query the availability of stashing capabilities, i.e., platform/NIC support, stashable object types, etc, via the rte_eth_dev_stashing_capabilities_get interface. The function pair rte_eth_dev_stashing_rx_config_set and rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,  cache level, and data object types) on the Rx and Tx queues. PMDs that support stashing must register their implementations with the following eth_dev_ops callbacks, which are invoked by the ethdev functions listed above. - stashing_capabilities_get - stashing_rx_hints_set - stashing_tx_hints_set Signed-off-by: Wathsala Vithanage Reviewed-by: Honnappa Nagarahalli Reviewed-by: Dhruv Tripathi --- lib/ethdev/ethdev_driver.h | 66 +++++++++++++++ lib/ethdev/rte_ethdev.c | 120 +++++++++++++++++++++++++++ lib/ethdev/rte_ethdev.h | 161 +++++++++++++++++++++++++++++++++++++ lib/ethdev/version.map | 4 + 4 files changed, 351 insertions(+) diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 1fd4562b40..7caaea54a8 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1367,6 +1367,68 @@ enum rte_eth_dev_operation { typedef uint64_t (*eth_get_restore_flags_t)(struct rte_eth_dev *dev, enum rte_eth_dev_operation op); +/** + * @internal + * Set cache stashing hints in Rx queue. + * + * @param dev + * Port (ethdev) handle. + * @param queue_id + * Rx queue. + * @param config + * Stashing hints configuration for the queue. + * + * @return + * -ENOTSUP if the device or the platform does not support cache stashing. + * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature. + * -EINVAL on invalid arguments. + * 0 on success. + */ +typedef int (*eth_stashing_rx_hints_set_t)(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_stashing_config *config); + +/** + * @internal + * Set cache stashing hints in Tx queue. + * + * @param dev + * Port (ethdev) handle. + * @param queue_id + * Tx queue. + * @param config + * Stashing hints configuration for the queue. + * + * @return + * -ENOTSUP if the device or the platform does not support cache stashing. + * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature. + * -EINVAL on invalid arguments. + * 0 on success. + */ +typedef int (*eth_stashing_tx_hints_set_t)(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_stashing_config *config); + +/** + * @internal + * Get cache stashing object types supported in the ethernet device. + * The return value indicates availability of stashing hints support + * in the hardware and the PMD. + * + * @param dev + * Port (ethdev) handle. + * @param objects + * PMD sets supported bits on return. + * + * @return + * -ENOTSUP if the device or the platform does not support cache stashing. + * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature. + * -EINVAL on NULL values for types or hints parameters. + * On return, types and hints parameters will have bits set for supported + * object types and hints. + * 0 on success. + */ +typedef int (*eth_stashing_capabilities_get_t)(struct rte_eth_dev *dev, + uint16_t *objects); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1393,6 +1455,10 @@ struct eth_dev_ops { eth_mac_addr_remove_t mac_addr_remove; /**< Remove MAC address */ eth_mac_addr_add_t mac_addr_add; /**< Add a MAC address */ eth_mac_addr_set_t mac_addr_set; /**< Set a MAC address */ + eth_stashing_rx_hints_set_t stashing_rx_hints_set; /**< Set Rx cache stashing*/ + eth_stashing_tx_hints_set_t stashing_tx_hints_set; /**< Set Tx cache stashing*/ + /** Get supported stashing hints*/ + eth_stashing_capabilities_get_t stashing_capabilities_get; /** Set list of multicast addresses */ eth_set_mc_addr_list_t set_mc_addr_list; mtu_set_t mtu_set; /**< Set MTU */ diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 6413c54e3b..d9bcc6c13d 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -153,6 +153,7 @@ static const struct { {RTE_ETH_DEV_CAPA_RXQ_SHARE, "RXQ_SHARE"}, {RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP, "FLOW_RULE_KEEP"}, {RTE_ETH_DEV_CAPA_FLOW_SHARED_OBJECT_KEEP, "FLOW_SHARED_OBJECT_KEEP"}, + {RTE_ETH_DEV_CAPA_CACHE_STASHING, "CACHE_STASHING"}, }; enum { @@ -7163,4 +7164,123 @@ int rte_eth_dev_map_aggr_tx_affinity(uint16_t port_id, uint16_t tx_queue_id, return ret; } +int +rte_eth_dev_validate_stashing_config(uint16_t port_id, uint16_t queue_id, + uint8_t queue_direction, + struct rte_eth_stashing_config *config) +{ + struct rte_eth_dev *dev; + struct rte_eth_dev_info dev_info; + uint16_t nb_queues; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + + if (!config) { + RTE_ETHDEV_LOG_LINE(ERR, "Invalid stashing configuration"); + return -EINVAL; + } + + /* + * Check for invalid objects + */ + if (!RTE_ETH_DEV_STASH_OBJECTS_VALID(config->objects)) { + RTE_ETHDEV_LOG_LINE(ERR, "Invalid stashing objects"); + return -EINVAL; + } + + dev = &rte_eth_devices[port_id]; + + nb_queues = (queue_direction == RTE_ETH_DEV_RX_QUEUE) ? + dev->data->nb_rx_queues : + dev->data->nb_tx_queues; + + if (queue_id >= nb_queues) { + RTE_ETHDEV_LOG_LINE(ERR, "Invalid Rx queue_id=%u", queue_id); + return -EINVAL; + } + + rte_eth_dev_info_get(port_id, &dev_info); + + if ((dev_info.dev_capa & RTE_ETH_DEV_CAPA_CACHE_STASHING) != + RTE_ETH_DEV_CAPA_CACHE_STASHING) + return -ENOTSUP; + + if (*dev->dev_ops->stashing_rx_hints_set == NULL || + *dev->dev_ops->stashing_tx_hints_set == NULL) { + RTE_ETHDEV_LOG_LINE(ERR, "Stashing hints are not implemented " + "in %s for %s", dev_info.driver_name, + dev_info.device->name); + return -ENOSYS; + } + + return 0; +} + +int +rte_eth_dev_stashing_rx_config_set(uint16_t port_id, uint16_t queue_id, + struct rte_eth_stashing_config *config) +{ + struct rte_eth_dev *dev; + + int ret = rte_eth_dev_validate_stashing_config(port_id, queue_id, + RTE_ETH_DEV_RX_QUEUE, + config); + if (ret < 0) + return ret; + + dev = &rte_eth_devices[port_id]; + + return eth_err(port_id, + (*dev->dev_ops->stashing_rx_hints_set)(dev, queue_id, + config)); +} + +int +rte_eth_dev_stashing_tx_config_set(uint16_t port_id, uint16_t queue_id, + struct rte_eth_stashing_config *config) +{ + struct rte_eth_dev *dev; + + int ret = rte_eth_dev_validate_stashing_config(port_id, queue_id, + RTE_ETH_DEV_TX_QUEUE, + config); + if (ret < 0) + return ret; + + dev = &rte_eth_devices[port_id]; + + return eth_err(port_id, + (*dev->dev_ops->stashing_rx_hints_set) (dev, queue_id, + config)); +} + +int +rte_eth_dev_stashing_capabilities_get(uint16_t port_id, uint16_t *objects) +{ + struct rte_eth_dev *dev; + struct rte_eth_dev_info dev_info; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + + if (!objects) + return -EINVAL; + + dev = &rte_eth_devices[port_id]; + rte_eth_dev_info_get(port_id, &dev_info); + + if ((dev_info.dev_capa & RTE_ETH_DEV_CAPA_CACHE_STASHING) != + RTE_ETH_DEV_CAPA_CACHE_STASHING) + return -ENOTSUP; + + if (*dev->dev_ops->stashing_capabilities_get == NULL) { + RTE_ETHDEV_LOG_LINE(ERR, "Stashing hints are not implemented " + "in %s for %s", dev_info.driver_name, + dev_info.device->name); + return -ENOSYS; + } + return eth_err(port_id, + (*dev->dev_ops->stashing_capabilities_get) + (dev, objects)); +} + RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO); diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index c4241d048c..c08f60ad4c 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -1653,6 +1653,9 @@ struct rte_eth_conf { #define RTE_ETH_DEV_CAPA_FLOW_SHARED_OBJECT_KEEP RTE_BIT64(4) /**@}*/ +/** Device supports stashing to CPU/system caches. */ +#define RTE_ETH_DEV_CAPA_CACHE_STASHING RTE_BIT64(5) + /* * Fallback default preferred Rx/Tx port parameters. * These are used if an application requests default parameters @@ -1824,6 +1827,7 @@ struct rte_eth_dev_info { struct rte_eth_dev_portconf default_txportconf; /** Generic device capabilities (RTE_ETH_DEV_CAPA_). */ uint64_t dev_capa; + uint16_t stashing_capa; /** * Switching information for ports on a device with a * embedded managed interconnect/switch. @@ -6115,6 +6119,163 @@ int rte_eth_cman_config_set(uint16_t port_id, const struct rte_eth_cman_config * __rte_experimental int rte_eth_cman_config_get(uint16_t port_id, struct rte_eth_cman_config *config); + + +/** Queue type is RX. */ +#define RTE_ETH_DEV_RX_QUEUE 0 +/** Queue type is TX. */ +#define RTE_ETH_DEV_TX_QUEUE 1 + + +/** + * @warning + * @b EXPERIMENTAL: this structure may change, or be removed, without prior notice + * + * A structure used for configuring the cache stashing hints. + */ +struct rte_eth_stashing_config { + /** ID of the Processor/Container the stashing hints are + * applied to + */ + uint16_t lcore_id; + /** Set if the target is a CPU containeri.lcore_id will be + * used to derive container ID + */ + uint16_t container : 1; + uint16_t padding : 7; + /** Cache level of the CPU specified by the cpu_id the + * stashing hints are applied to + */ + uint16_t cache_level : 8; + /** Object types the configuration is applied to + */ + uint16_t objects; + /** The offset if RTE_ETH_DEV_STASH_OBJECT_OFFSET bit is set + * in objects + */ + off_t offset; +}; + +/**@{@name Stashable Rx/Tx queue object types supported by the ethernet device + *@see rte_eth_dev_stashing_capabilities_get + *@see rte_eth_dev_stashing_rx_config_set + *@see rte_eth_dev_stashing_tx_config_set + */ + +/** + * Apply stashing hint to data at a given offset from the start of a + * received packet. + */ +#define RTE_ETH_DEV_STASH_OBJECT_OFFSET 0x0001 + +/** Apply stashing hint to an rx descriptor. */ +#define RTE_ETH_DEV_STASH_OBJECT_DESC 0x0002 + +/** Apply stashing hint to a header of a received packet. */ +#define RTE_ETH_DEV_STASH_OBJECT_HEADER 0x0004 + +/** Apply stashing hint to a payload of a received packet. */ +#define RTE_ETH_DEV_STASH_OBJECT_PAYLOAD 0x0008 + +#define __RTE_ETH_DEV_STASH_OBJECT_MASK 0x000f +/**@}*/ + +#define RTE_ETH_DEV_STASH_OBJECTS_VALID(t) \ + ((!((t) & (~__RTE_ETH_DEV_STASH_OBJECT_MASK))) && (t)) + +/** + * + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * @internal + * Helper function to validate stashing hints configuration. + */ +__rte_experimental +int rte_eth_dev_validate_stashing_config(uint16_t port_id, uint16_t queue_id, + uint8_t queue_direction, + struct rte_eth_stashing_config *config); + +/** + * + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * Provide cache stashing hints for improved memory access latencies for + * packets received by the NIC. + * This feature is available only in supported NICs and platforms. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param queue_id + * The index of the receive queue to which hints are applied. + * @param config + * Stashing configuration. + * @return + * - (-ENODEV) on incorrect port_ids. + * - (-EINVAL) if both RX and TX object types used in conjuection in objects + * parameter. + * - (-EINVAL) on invalid queue_id. + * - (-ENOTSUP) if RTE_ETH_DEV_CAPA_CACHE_STASHING capability is unavailable. + * - (-ENOSYS) if PMD does not implement cache stashing hints. + * - (0) on Success. + */ +__rte_experimental +int rte_eth_dev_stashing_rx_config_set(uint16_t port_id, uint16_t queue_id, + struct rte_eth_stashing_config *config); + +/** + * + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * Configure cache stashing for improved memory access latencies for Tx + * queue completion descriptors being sent to host system by the NIC. + * This feature is available only in supported NICs and platforms. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param queue_id + * The index of the receive queue to which hints are applied. + * @param config + * Stashing configuration. + * @return + * - (-ENODEV) on incorrect port_ids. + * - (-EINVAL) if both RX and TX object types are used in conjuection in objects + * parameter. + * - (-EINVAL) if hints are incompatible with TX queues. + * - (-EINVAL) on invalid queue_id. + * - (-ENOTSUP) if RTE_ETH_DEV_CAPA_CACHE_STASHING capability is unavailable. + * - (-ENOSYS) if PMD does not implement cache stashing hints. + * - (0) on Success. + */ +__rte_experimental +int rte_eth_dev_stashing_tx_config_set(uint16_t port_id, uint16_t queue_id, + struct rte_eth_stashing_config *config); + +/** + * + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * Discover cache stashing objects supported in the ethernet device. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param objects + * Supported objects vector set by the ethernet device. + * @return + * On return types and hints parameters will have bits set for supported + * object types. + * - (-ENOTSUP) if the device or the platform does not support cache stashing. + * - (-ENOSYS) if the underlying PMD hasn't implemented cache stashing + * feature. + * - (-EINVAL) on NULL values for types or hints parameters. + * - (0) on success. + */ +__rte_experimental +int rte_eth_dev_stashing_capabilities_get(uint16_t port_id, uint16_t *objects); + #include #ifdef __cplusplus diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 12f48c70a0..49c8c46a00 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -337,6 +337,10 @@ EXPERIMENTAL { rte_eth_timesync_adjust_freq; rte_flow_async_create_by_index_with_pattern; rte_tm_node_query; + rte_eth_dev_stashing_rx_config_set; + rte_eth_dev_stashing_tx_config_set; + rte_eth_dev_stashing_capabilities_get; + rte_eth_dev_validate_stashing_config; }; INTERNAL {