[RFC,v3,2/2] ethdev: introduce the cache stashing hints API
Checks
Commit Message
Extend the ethdev library to enable the stashing of different data
objects, such as the ones listed below, into CPU caches directly
from the NIC.
- Rx/Tx queue descriptors
- Rx packets
- Packet headers
- packet payloads
- Data of a packet at an offset from the start of the packet
The APIs are designed in a hardware/vendor agnostic manner such that
supporting PMDs could use any capabilities available in the underlying
hardware for fine-grained stashing of data objects into a CPU cache
(e.g., Steering Tags int PCIe TLP Processing Hints).
The API provides an interface to query the availability of stashing
capabilities, i.e., platform/NIC support, stashable object types, etc,
via the rte_eth_dev_stashing_capabilities_get interface.
The function pair rte_eth_dev_stashing_rx_config_set and
rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,
cache level, and data object types) on the Rx and Tx queues.
PMDs that support stashing must register their implementations with the
following eth_dev_ops callbacks, which are invoked by the ethdev
functions listed above.
- stashing_capabilities_get
- stashing_rx_hints_set
- stashing_tx_hints_set
Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Dhruv Tripathi <dhruv.tripathi@arm.com>
---
lib/ethdev/ethdev_driver.h | 66 +++++++++++++++
lib/ethdev/rte_ethdev.c | 120 +++++++++++++++++++++++++++
lib/ethdev/rte_ethdev.h | 161 +++++++++++++++++++++++++++++++++++++
lib/ethdev/version.map | 4 +
4 files changed, 351 insertions(+)
Comments
> +/**
> + * Apply stashing hint to data at a given offset from the start of a
> + * received packet.
> + */
> +#define RTE_ETH_DEV_STASH_OBJECT_OFFSET 0x0001
> +
> +/** Apply stashing hint to an rx descriptor. */
> +#define RTE_ETH_DEV_STASH_OBJECT_DESC 0x0002
> +
> +/** Apply stashing hint to a header of a received packet. */
> +#define RTE_ETH_DEV_STASH_OBJECT_HEADER 0x0004
> +
> +/** Apply stashing hint to a payload of a received packet. */
> +#define RTE_ETH_DEV_STASH_OBJECT_PAYLOAD 0x0008
> +
> +#define __RTE_ETH_DEV_STASH_OBJECT_MASK 0x000f
> +/**@}*/
Although I agree these would be sensible parameters, they seem orthogonal to the Ethdev RX port queue capabilities/configuration.
The RTE_ETH_DEV_STASH_OBJECT_DESC aligns well; no problem there.
How much of a packet is considered the "header"? L3, L4, Outer tunnel header, or both Outer and Inner headers? If it follows the configuration already established through the existing Ethdev APIs on the RX port queue, it aligns well too.
Now, it's the data part I'm wondering about:
If an RX port queue is configured for receiving into mbufs from one mempool containing 2 KB objects (i.e. segments of 2 KB contiguous RAM), how do the OFFSET, HEADER, and PAYLOAD hints work? And what is the corresponding Ethdev RX queue configuration?
Same questions, considering an RX port queue configured for receiving the first 128 B into an mbuf from one mempool and the remaining part of the packet into 2 KB mbufs from another mempool?
Please provide some explanatory examples using these APIs along with the existing APIs for setting up the RX port queue.
Packets may be jumbo frames, scattered into multiple 2 KB mbufs. That should not make any difference; i.e. I assume the OFFSET, HEADER and PAYLOAD hints are considered at a packet level, not segment level.
I also assume IP fragments are treated like any other IP packets; basically, the only difference is the size of the header (it only has the IP header, and no following headers).
On Mon, Oct 21, 2024 at 7:23 AM Wathsala Vithanage
<wathsala.vithanage@arm.com> wrote:
>
> Extend the ethdev library to enable the stashing of different data
> objects, such as the ones listed below, into CPU caches directly
> from the NIC.
>
> - Rx/Tx queue descriptors
> - Rx packets
> - Packet headers
> - packet payloads
> - Data of a packet at an offset from the start of the packet
>
> The APIs are designed in a hardware/vendor agnostic manner such that
> supporting PMDs could use any capabilities available in the underlying
> hardware for fine-grained stashing of data objects into a CPU cache
> (e.g., Steering Tags int PCIe TLP Processing Hints).
>
> The API provides an interface to query the availability of stashing
> capabilities, i.e., platform/NIC support, stashable object types, etc,
> via the rte_eth_dev_stashing_capabilities_get interface.
>
> The function pair rte_eth_dev_stashing_rx_config_set and
> rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,
> cache level, and data object types) on the Rx and Tx queues.
>
> PMDs that support stashing must register their implementations with the
> following eth_dev_ops callbacks, which are invoked by the ethdev
> functions listed above.
>
> - stashing_capabilities_get
> - stashing_rx_hints_set
> - stashing_tx_hints_set
>
> Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> Reviewed-by: Dhruv Tripathi <dhruv.tripathi@arm.com>
>
> +
> +/** Queue type is RX. */
> +#define RTE_ETH_DEV_RX_QUEUE 0
> +/** Queue type is TX. */
> +#define RTE_ETH_DEV_TX_QUEUE 1
> +
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this structure may change, or be removed, without prior notice
> + *
> + * A structure used for configuring the cache stashing hints.
> + */
> +struct rte_eth_stashing_config {
> + /** ID of the Processor/Container the stashing hints are
> + * applied to
> + */
> + uint16_t lcore_id;
> + /** Set if the target is a CPU containeri.lcore_id will be
> + * used to derive container ID
> + */
> + uint16_t container : 1;
> + uint16_t padding : 7;
> + /** Cache level of the CPU specified by the cpu_id the
> + * stashing hints are applied to
> + */
> + uint16_t cache_level : 8;
> + /** Object types the configuration is applied to
> + */
> + uint16_t objects;
> + /** The offset if RTE_ETH_DEV_STASH_OBJECT_OFFSET bit is set
> + * in objects
> + */
> + off_t offset;
> +};
> +
> +/**@{@name Stashable Rx/Tx queue object types supported by the ethernet device
> + *@see rte_eth_dev_stashing_capabilities_get
> + *@see rte_eth_dev_stashing_rx_config_set
> + *@see rte_eth_dev_stashing_tx_config_set
> + */
> +
> +/**
> + * Apply stashing hint to data at a given offset from the start of a
> + * received packet.
> + */
> +#define RTE_ETH_DEV_STASH_OBJECT_OFFSET 0x0001
> +
> +/** Apply stashing hint to an rx descriptor. */
> +#define RTE_ETH_DEV_STASH_OBJECT_DESC 0x0002
> +
> +/** Apply stashing hint to a header of a received packet. */
> +#define RTE_ETH_DEV_STASH_OBJECT_HEADER 0x0004
> +
> +/** Apply stashing hint to a payload of a received packet. */
> +#define RTE_ETH_DEV_STASH_OBJECT_PAYLOAD 0x0008
> +
> +#define __RTE_ETH_DEV_STASH_OBJECT_MASK 0x000f
> +/**@}*/
> +
> +#define RTE_ETH_DEV_STASH_OBJECTS_VALID(t) \
> + ((!((t) & (~__RTE_ETH_DEV_STASH_OBJECT_MASK))) && (t))
> +
I think, at one of point of time, we need to extend this to other
device class like(cryptodev etc)
where the data needs to move over bus. In that context, all the above
symbols better to be in
EAL and the device class subsystem(example ethdev) gives PMD callback.
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Thursday, 24 October 2024 07.49
>
> On Mon, Oct 21, 2024 at 7:23 AM Wathsala Vithanage
> <wathsala.vithanage@arm.com> wrote:
> >
> > Extend the ethdev library to enable the stashing of different data
> > objects, such as the ones listed below, into CPU caches directly
> > from the NIC.
> >
> > - Rx/Tx queue descriptors
> > - Rx packets
> > - Packet headers
> > - packet payloads
> > - Data of a packet at an offset from the start of the packet
> >
> > The APIs are designed in a hardware/vendor agnostic manner such that
> > supporting PMDs could use any capabilities available in the
> underlying
> > hardware for fine-grained stashing of data objects into a CPU cache
> > (e.g., Steering Tags int PCIe TLP Processing Hints).
> >
> > The API provides an interface to query the availability of stashing
> > capabilities, i.e., platform/NIC support, stashable object types,
> etc,
> > via the rte_eth_dev_stashing_capabilities_get interface.
> >
> > The function pair rte_eth_dev_stashing_rx_config_set and
> > rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,
> > cache level, and data object types) on the Rx and Tx queues.
> >
> > PMDs that support stashing must register their implementations with
> the
> > following eth_dev_ops callbacks, which are invoked by the ethdev
> > functions listed above.
> >
> > - stashing_capabilities_get
> > - stashing_rx_hints_set
> > - stashing_tx_hints_set
> >
> > Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
> > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > Reviewed-by: Dhruv Tripathi <dhruv.tripathi@arm.com>
> >
>
> > +
> > +/** Queue type is RX. */
> > +#define RTE_ETH_DEV_RX_QUEUE 0
> > +/** Queue type is TX. */
> > +#define RTE_ETH_DEV_TX_QUEUE 1
> > +
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this structure may change, or be removed,
> without prior notice
> > + *
> > + * A structure used for configuring the cache stashing hints.
This structure can only describe one stashing hint.
Please use singular, not plural, in its description.
> > + */
> > +struct rte_eth_stashing_config {
> > + /** ID of the Processor/Container the stashing hints are
> > + * applied to
> > + */
> > + uint16_t lcore_id;
The common type used for lcore_id is "unsigned int", ref. e.g. rte_lcore_id() return value.
Alternatively uint32_t, ref. LCORE_ID_ANY.
> > + /** Set if the target is a CPU containeri.lcore_id will be
> > + * used to derive container ID
> > + */
> > + uint16_t container : 1;
> > + uint16_t padding : 7;
> > + /** Cache level of the CPU specified by the cpu_id the
> > + * stashing hints are applied to
> > + */
> > + uint16_t cache_level : 8;
> > + /** Object types the configuration is applied to
> > + */
> > + uint16_t objects;
> > + /** The offset if RTE_ETH_DEV_STASH_OBJECT_OFFSET bit is set
> > + * in objects
> > + */
> > + off_t offset;
off_t is for files, ptrdiff_t is for memory.
> > +};
> > +
> > +/**@{@name Stashable Rx/Tx queue object types supported by the
> ethernet device
> > + *@see rte_eth_dev_stashing_capabilities_get
> > + *@see rte_eth_dev_stashing_rx_config_set
> > + *@see rte_eth_dev_stashing_tx_config_set
> > + */
> > +
> > +/**
> > + * Apply stashing hint to data at a given offset from the start of a
> > + * received packet.
> > + */
> > +#define RTE_ETH_DEV_STASH_OBJECT_OFFSET 0x0001
> > +
> > +/** Apply stashing hint to an rx descriptor. */
> > +#define RTE_ETH_DEV_STASH_OBJECT_DESC 0x0002
> > +
> > +/** Apply stashing hint to a header of a received packet. */
> > +#define RTE_ETH_DEV_STASH_OBJECT_HEADER 0x0004
> > +
> > +/** Apply stashing hint to a payload of a received packet. */
> > +#define RTE_ETH_DEV_STASH_OBJECT_PAYLOAD 0x0008
> > +
> > +#define __RTE_ETH_DEV_STASH_OBJECT_MASK 0x000f
> > +/**@}*/
> > +
> > +#define RTE_ETH_DEV_STASH_OBJECTS_VALID(t)
> \
> > + ((!((t) & (~__RTE_ETH_DEV_STASH_OBJECT_MASK))) && (t))
> > +
>
>
> I think, at one of point of time, we need to extend this to other
> device class like(cryptodev etc)
> where the data needs to move over bus. In that context, all the above
> symbols better to be in
> EAL and the device class subsystem(example ethdev) gives PMD callback.
+1
When generalizing this, perhaps "header" and "payload" should be renamed to "device-specific".
For ethdevs, the typical meaning of "device-specific" would be splitting at some header (as suggested by the "header" and "payload" enum values).
Furthermore, for ethdevs, using a "device-specific" would allow the device to split at some other point, controlled through other ethdev APIs.
E.g. the split point could be controlled by rte_flow; this would allow rte_flow to put entire packets in L2 cache for some packet types, and only the packet header in L2 cache for some other packet types. (Someone at the conference call suggested combining Steering Tags with rte_flow - this might be a way of doing it.)
> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Thursday, October 24, 2024 12:49 AM
> To: Wathsala Wathawana Vithanage <wathsala.vithanage@arm.com>
> Cc: thomas@monjalon.net; Ferruh Yigit <ferruh.yigit@amd.com>; Andrew
> Rybchenko <andrew.rybchenko@oktetlabs.ru>; dev@dpdk.org; nd
> <nd@arm.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> Dhruv Tripathi <Dhruv.Tripathi@arm.com>
> Subject: Re: [RFC v3 2/2] ethdev: introduce the cache stashing hints API
>
> On Mon, Oct 21, 2024 at 7:23 AM Wathsala Vithanage
> <wathsala.vithanage@arm.com> wrote:
> >
> > Extend the ethdev library to enable the stashing of different data
> > objects, such as the ones listed below, into CPU caches directly from
> > the NIC.
> >
> > - Rx/Tx queue descriptors
> > - Rx packets
> > - Packet headers
> > - packet payloads
> > - Data of a packet at an offset from the start of the packet
> >
> > The APIs are designed in a hardware/vendor agnostic manner such that
> > supporting PMDs could use any capabilities available in the underlying
> > hardware for fine-grained stashing of data objects into a CPU cache
> > (e.g., Steering Tags int PCIe TLP Processing Hints).
> >
> > The API provides an interface to query the availability of stashing
> > capabilities, i.e., platform/NIC support, stashable object types, etc,
> > via the rte_eth_dev_stashing_capabilities_get interface.
> >
> > The function pair rte_eth_dev_stashing_rx_config_set and
> > rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,
> > cache level, and data object types) on the Rx and Tx queues.
> >
> > PMDs that support stashing must register their implementations with
> > the following eth_dev_ops callbacks, which are invoked by the ethdev
> > functions listed above.
> >
> > - stashing_capabilities_get
> > - stashing_rx_hints_set
> > - stashing_tx_hints_set
> >
> > Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
> > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > Reviewed-by: Dhruv Tripathi <dhruv.tripathi@arm.com>
> >
>
> > +
> > +/** Queue type is RX. */
> > +#define RTE_ETH_DEV_RX_QUEUE 0
> > +/** Queue type is TX. */
> > +#define RTE_ETH_DEV_TX_QUEUE 1
> > +
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this structure may change, or be removed, without
> > +prior notice
> > + *
> > + * A structure used for configuring the cache stashing hints.
> > + */
> > +struct rte_eth_stashing_config {
> > + /** ID of the Processor/Container the stashing hints are
> > + * applied to
> > + */
> > + uint16_t lcore_id;
> > + /** Set if the target is a CPU containeri.lcore_id will be
> > + * used to derive container ID
> > + */
> > + uint16_t container : 1;
> > + uint16_t padding : 7;
> > + /** Cache level of the CPU specified by the cpu_id the
> > + * stashing hints are applied to
> > + */
> > + uint16_t cache_level : 8;
> > + /** Object types the configuration is applied to
> > + */
> > + uint16_t objects;
> > + /** The offset if RTE_ETH_DEV_STASH_OBJECT_OFFSET bit is set
> > + * in objects
> > + */
> > + off_t offset;
> > +};
> > +
> > +/**@{@name Stashable Rx/Tx queue object types supported by the
> > +ethernet device *@see rte_eth_dev_stashing_capabilities_get
> > + *@see rte_eth_dev_stashing_rx_config_set
> > + *@see rte_eth_dev_stashing_tx_config_set
> > + */
> > +
> > +/**
> > + * Apply stashing hint to data at a given offset from the start of a
> > + * received packet.
> > + */
> > +#define RTE_ETH_DEV_STASH_OBJECT_OFFSET 0x0001
> > +
> > +/** Apply stashing hint to an rx descriptor. */
> > +#define RTE_ETH_DEV_STASH_OBJECT_DESC 0x0002
> > +
> > +/** Apply stashing hint to a header of a received packet. */
> > +#define RTE_ETH_DEV_STASH_OBJECT_HEADER 0x0004
> > +
> > +/** Apply stashing hint to a payload of a received packet. */
> > +#define RTE_ETH_DEV_STASH_OBJECT_PAYLOAD 0x0008
> > +
> > +#define __RTE_ETH_DEV_STASH_OBJECT_MASK 0x000f
> > +/**@}*/
> > +
> > +#define RTE_ETH_DEV_STASH_OBJECTS_VALID(t) \
> > + ((!((t) & (~__RTE_ETH_DEV_STASH_OBJECT_MASK))) && (t))
> > +
>
>
> I think, at one of point of time, we need to extend this to other device class
> like(cryptodev etc) where the data needs to move over bus. In that context, all
> the above symbols better to be in EAL and the device class subsystem(example
> ethdev) gives PMD callback.
+1
I will make that change in the RFC v4.
> -----Original Message-----
> From: Morten Brørup <mb@smartsharesystems.com>
> Sent: Thursday, October 24, 2024 2:00 AM
> To: Jerin Jacob <jerinjacobk@gmail.com>; Wathsala Wathawana Vithanage
> <wathsala.vithanage@arm.com>
> Cc: thomas@monjalon.net; Ferruh Yigit <ferruh.yigit@amd.com>; Andrew
> Rybchenko <andrew.rybchenko@oktetlabs.ru>; dev@dpdk.org; nd
> <nd@arm.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> Dhruv Tripathi <Dhruv.Tripathi@arm.com>
> Subject: RE: [RFC v3 2/2] ethdev: introduce the cache stashing hints API
>
> > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > Sent: Thursday, 24 October 2024 07.49
> >
> > On Mon, Oct 21, 2024 at 7:23 AM Wathsala Vithanage
> > <wathsala.vithanage@arm.com> wrote:
> > >
> > > Extend the ethdev library to enable the stashing of different data
> > > objects, such as the ones listed below, into CPU caches directly
> > > from the NIC.
> > >
> > > - Rx/Tx queue descriptors
> > > - Rx packets
> > > - Packet headers
> > > - packet payloads
> > > - Data of a packet at an offset from the start of the packet
> > >
> > > The APIs are designed in a hardware/vendor agnostic manner such that
> > > supporting PMDs could use any capabilities available in the
> > underlying
> > > hardware for fine-grained stashing of data objects into a CPU cache
> > > (e.g., Steering Tags int PCIe TLP Processing Hints).
> > >
> > > The API provides an interface to query the availability of stashing
> > > capabilities, i.e., platform/NIC support, stashable object types,
> > etc,
> > > via the rte_eth_dev_stashing_capabilities_get interface.
> > >
> > > The function pair rte_eth_dev_stashing_rx_config_set and
> > > rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,
> > > cache level, and data object types) on the Rx and Tx queues.
> > >
> > > PMDs that support stashing must register their implementations with
> > the
> > > following eth_dev_ops callbacks, which are invoked by the ethdev
> > > functions listed above.
> > >
> > > - stashing_capabilities_get
> > > - stashing_rx_hints_set
> > > - stashing_tx_hints_set
> > >
> > > Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
> > > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > > Reviewed-by: Dhruv Tripathi <dhruv.tripathi@arm.com>
> > >
> >
> > > +
> > > +/** Queue type is RX. */
> > > +#define RTE_ETH_DEV_RX_QUEUE 0
> > > +/** Queue type is TX. */
> > > +#define RTE_ETH_DEV_TX_QUEUE 1
> > > +
> > > +
> > > +/**
> > > + * @warning
> > > + * @b EXPERIMENTAL: this structure may change, or be removed,
> > without prior notice
> > > + *
> > > + * A structure used for configuring the cache stashing hints.
>
> This structure can only describe one stashing hint.
> Please use singular, not plural, in its description.
>
> > > + */
> > > +struct rte_eth_stashing_config {
> > > + /** ID of the Processor/Container the stashing hints are
> > > + * applied to
> > > + */
> > > + uint16_t lcore_id;
>
> The common type used for lcore_id is "unsigned int", ref. e.g. rte_lcore_id()
> return value.
> Alternatively uint32_t, ref. LCORE_ID_ANY.
>
+1
> > > + /** Set if the target is a CPU containeri.lcore_id will be
> > > + * used to derive container ID
> > > + */
> > > + uint16_t container : 1;
> > > + uint16_t padding : 7;
> > > + /** Cache level of the CPU specified by the cpu_id the
> > > + * stashing hints are applied to
> > > + */
> > > + uint16_t cache_level : 8;
> > > + /** Object types the configuration is applied to
> > > + */
> > > + uint16_t objects;
> > > + /** The offset if RTE_ETH_DEV_STASH_OBJECT_OFFSET bit is set
> > > + * in objects
> > > + */
> > > + off_t offset;
>
> off_t is for files, ptrdiff_t is for memory.
>
+1
> > > +};
> > > +
> > > +/**@{@name Stashable Rx/Tx queue object types supported by the
> > ethernet device
> > > + *@see rte_eth_dev_stashing_capabilities_get
> > > + *@see rte_eth_dev_stashing_rx_config_set
> > > + *@see rte_eth_dev_stashing_tx_config_set
> > > + */
> > > +
> > > +/**
> > > + * Apply stashing hint to data at a given offset from the start of
> > > +a
> > > + * received packet.
> > > + */
> > > +#define RTE_ETH_DEV_STASH_OBJECT_OFFSET 0x0001
> > > +
> > > +/** Apply stashing hint to an rx descriptor. */
> > > +#define RTE_ETH_DEV_STASH_OBJECT_DESC 0x0002
> > > +
> > > +/** Apply stashing hint to a header of a received packet. */
> > > +#define RTE_ETH_DEV_STASH_OBJECT_HEADER 0x0004
> > > +
> > > +/** Apply stashing hint to a payload of a received packet. */
> > > +#define RTE_ETH_DEV_STASH_OBJECT_PAYLOAD 0x0008
> > > +
> > > +#define __RTE_ETH_DEV_STASH_OBJECT_MASK 0x000f
> > > +/**@}*/
> > > +
> > > +#define RTE_ETH_DEV_STASH_OBJECTS_VALID(t)
> > \
> > > + ((!((t) & (~__RTE_ETH_DEV_STASH_OBJECT_MASK))) && (t))
> > > +
> >
> >
> > I think, at one of point of time, we need to extend this to other
> > device class like(cryptodev etc) where the data needs to move over
> > bus. In that context, all the above symbols better to be in EAL and
> > the device class subsystem(example ethdev) gives PMD callback.
>
> +1
>
> When generalizing this, perhaps "header" and "payload" should be renamed
> to "device-specific".
>
> For ethdevs, the typical meaning of "device-specific" would be splitting at
> some header (as suggested by the "header" and "payload" enum values).
>
> Furthermore, for ethdevs, using a "device-specific" would allow the device to
> split at some other point, controlled through other ethdev APIs.
> E.g. the split point could be controlled by rte_flow; this would allow rte_flow
> to put entire packets in L2 cache for some packet types, and only the packet
> header in L2 cache for some other packet types. (Someone at the conference
> call suggested combining Steering Tags with rte_flow - this might be a way of
> doing it.)
+1
On Mon, 21 Oct 2024 01:52:46 +0000
Wathsala Vithanage <wathsala.vithanage@arm.com> wrote:
> Extend the ethdev library to enable the stashing of different data
> objects, such as the ones listed below, into CPU caches directly
> from the NIC.
>
> - Rx/Tx queue descriptors
> - Rx packets
> - Packet headers
> - packet payloads
> - Data of a packet at an offset from the start of the packet
>
> The APIs are designed in a hardware/vendor agnostic manner such that
> supporting PMDs could use any capabilities available in the underlying
> hardware for fine-grained stashing of data objects into a CPU cache
> (e.g., Steering Tags int PCIe TLP Processing Hints).
>
> The API provides an interface to query the availability of stashing
> capabilities, i.e., platform/NIC support, stashable object types, etc,
> via the rte_eth_dev_stashing_capabilities_get interface.
>
> The function pair rte_eth_dev_stashing_rx_config_set and
> rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,
> cache level, and data object types) on the Rx and Tx queues.
>
> PMDs that support stashing must register their implementations with the
> following eth_dev_ops callbacks, which are invoked by the ethdev
> functions listed above.
>
> - stashing_capabilities_get
> - stashing_rx_hints_set
> - stashing_tx_hints_set
>
> Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> Reviewed-by: Dhruv Tripathi <dhruv.tripathi@arm.com>
>
> ---
> lib/ethdev/ethdev_driver.h | 66 +++++++++++++++
> lib/ethdev/rte_ethdev.c | 120 +++++++++++++++++++++++++++
> lib/ethdev/rte_ethdev.h | 161 +++++++++++++++++++++++++++++++++++++
> lib/ethdev/version.map | 4 +
> 4 files changed, 351 insertions(+)
>
> diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
> index 1fd4562b40..7caaea54a8 100644
> --- a/lib/ethdev/ethdev_driver.h
> +++ b/lib/ethdev/ethdev_driver.h
> @@ -1367,6 +1367,68 @@ enum rte_eth_dev_operation {
> typedef uint64_t (*eth_get_restore_flags_t)(struct rte_eth_dev *dev,
> enum rte_eth_dev_operation op);
>
> +/**
> + * @internal
> + * Set cache stashing hints in Rx queue.
> + *
> + * @param dev
> + * Port (ethdev) handle.
> + * @param queue_id
> + * Rx queue.
> + * @param config
> + * Stashing hints configuration for the queue.
> + *
> + * @return
> + * -ENOTSUP if the device or the platform does not support cache stashing.
> + * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature.
> + * -EINVAL on invalid arguments.
> + * 0 on success.
> + */
> +typedef int (*eth_stashing_rx_hints_set_t)(struct rte_eth_dev *dev, uint16_t queue_id,
> + struct rte_eth_stashing_config *config);
> +
> +/**
> + * @internal
> + * Set cache stashing hints in Tx queue.
> + *
> + * @param dev
> + * Port (ethdev) handle.
> + * @param queue_id
> + * Tx queue.
> + * @param config
> + * Stashing hints configuration for the queue.
> + *
> + * @return
> + * -ENOTSUP if the device or the platform does not support cache stashing.
> + * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature.
> + * -EINVAL on invalid arguments.
> + * 0 on success.
> + */
> +typedef int (*eth_stashing_tx_hints_set_t)(struct rte_eth_dev *dev, uint16_t queue_id,
> + struct rte_eth_stashing_config *config);
> +
> +/**
> + * @internal
> + * Get cache stashing object types supported in the ethernet device.
> + * The return value indicates availability of stashing hints support
> + * in the hardware and the PMD.
> + *
> + * @param dev
> + * Port (ethdev) handle.
> + * @param objects
> + * PMD sets supported bits on return.
> + *
> + * @return
> + * -ENOTSUP if the device or the platform does not support cache stashing.
> + * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature.
> + * -EINVAL on NULL values for types or hints parameters.
> + * On return, types and hints parameters will have bits set for supported
> + * object types and hints.
> + * 0 on success.
> + */
> +typedef int (*eth_stashing_capabilities_get_t)(struct rte_eth_dev *dev,
> + uint16_t *objects);
> +
> /**
> * @internal A structure containing the functions exported by an Ethernet driver.
> */
> @@ -1393,6 +1455,10 @@ struct eth_dev_ops {
> eth_mac_addr_remove_t mac_addr_remove; /**< Remove MAC address */
> eth_mac_addr_add_t mac_addr_add; /**< Add a MAC address */
> eth_mac_addr_set_t mac_addr_set; /**< Set a MAC address */
> + eth_stashing_rx_hints_set_t stashing_rx_hints_set; /**< Set Rx cache stashing*/
> + eth_stashing_tx_hints_set_t stashing_tx_hints_set; /**< Set Tx cache stashing*/
> + /** Get supported stashing hints*/
> + eth_stashing_capabilities_get_t stashing_capabilities_get;
> /** Set list of multicast addresses */
> eth_set_mc_addr_list_t set_mc_addr_list;
> mtu_set_t mtu_set; /**< Set MTU */
Since eth_dev_ops is visible in application binary, it is part of the ABI.
Therefore it can not be changed until 25.11 release.
On Tue, Dec 3, 2024 at 10:13 PM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> On Mon, 21 Oct 2024 01:52:46 +0000
> Wathsala Vithanage <wathsala.vithanage@arm.com> wrote:
>
> > Extend the ethdev library to enable the stashing of different data
> > objects, such as the ones listed below, into CPU caches directly
> > from the NIC.
> >
> > - Rx/Tx queue descriptors
> > - Rx packets
> > - Packet headers
> > - packet payloads
> > - Data of a packet at an offset from the start of the packet
> >
> > The APIs are designed in a hardware/vendor agnostic manner such that
> > supporting PMDs could use any capabilities available in the underlying
> > hardware for fine-grained stashing of data objects into a CPU cache
> > (e.g., Steering Tags int PCIe TLP Processing Hints).
> >
> > The API provides an interface to query the availability of stashing
> > capabilities, i.e., platform/NIC support, stashable object types, etc,
> > via the rte_eth_dev_stashing_capabilities_get interface.
> >
> > The function pair rte_eth_dev_stashing_rx_config_set and
> > rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,
> > cache level, and data object types) on the Rx and Tx queues.
> >
> > PMDs that support stashing must register their implementations with the
> > following eth_dev_ops callbacks, which are invoked by the ethdev
> > functions listed above.
> >
> > - stashing_capabilities_get
> > - stashing_rx_hints_set
> > - stashing_tx_hints_set
> >
> > Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
> > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > Reviewed-by: Dhruv Tripathi <dhruv.tripathi@arm.com>
> >
> > ---
> > lib/ethdev/ethdev_driver.h | 66 +++++++++++++++
> > lib/ethdev/rte_ethdev.c | 120 +++++++++++++++++++++++++++
> > lib/ethdev/rte_ethdev.h | 161 +++++++++++++++++++++++++++++++++++++
> > lib/ethdev/version.map | 4 +
> > 4 files changed, 351 insertions(+)
> >
> > diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
> > index 1fd4562b40..7caaea54a8 100644
> > --- a/lib/ethdev/ethdev_driver.h
> > +++ b/lib/ethdev/ethdev_driver.h
> > @@ -1367,6 +1367,68 @@ enum rte_eth_dev_operation {
> > typedef uint64_t (*eth_get_restore_flags_t)(struct rte_eth_dev *dev,
> > enum rte_eth_dev_operation op);
> >
> > +/**
> > + * @internal
> > + * Set cache stashing hints in Rx queue.
> > + *
> > + * @param dev
> > + * Port (ethdev) handle.
> > + * @param queue_id
> > + * Rx queue.
> > + * @param config
> > + * Stashing hints configuration for the queue.
> > + *
> > + * @return
> > + * -ENOTSUP if the device or the platform does not support cache stashing.
> > + * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature.
> > + * -EINVAL on invalid arguments.
> > + * 0 on success.
> > + */
> > +typedef int (*eth_stashing_rx_hints_set_t)(struct rte_eth_dev *dev, uint16_t queue_id,
> > + struct rte_eth_stashing_config *config);
> > +
> > +/**
> > + * @internal
> > + * Set cache stashing hints in Tx queue.
> > + *
> > + * @param dev
> > + * Port (ethdev) handle.
> > + * @param queue_id
> > + * Tx queue.
> > + * @param config
> > + * Stashing hints configuration for the queue.
> > + *
> > + * @return
> > + * -ENOTSUP if the device or the platform does not support cache stashing.
> > + * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature.
> > + * -EINVAL on invalid arguments.
> > + * 0 on success.
> > + */
> > +typedef int (*eth_stashing_tx_hints_set_t)(struct rte_eth_dev *dev, uint16_t queue_id,
> > + struct rte_eth_stashing_config *config);
> > +
> > +/**
> > + * @internal
> > + * Get cache stashing object types supported in the ethernet device.
> > + * The return value indicates availability of stashing hints support
> > + * in the hardware and the PMD.
> > + *
> > + * @param dev
> > + * Port (ethdev) handle.
> > + * @param objects
> > + * PMD sets supported bits on return.
> > + *
> > + * @return
> > + * -ENOTSUP if the device or the platform does not support cache stashing.
> > + * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature.
> > + * -EINVAL on NULL values for types or hints parameters.
> > + * On return, types and hints parameters will have bits set for supported
> > + * object types and hints.
> > + * 0 on success.
> > + */
> > +typedef int (*eth_stashing_capabilities_get_t)(struct rte_eth_dev *dev,
> > + uint16_t *objects);
> > +
> > /**
> > * @internal A structure containing the functions exported by an Ethernet driver.
> > */
> > @@ -1393,6 +1455,10 @@ struct eth_dev_ops {
> > eth_mac_addr_remove_t mac_addr_remove; /**< Remove MAC address */
> > eth_mac_addr_add_t mac_addr_add; /**< Add a MAC address */
> > eth_mac_addr_set_t mac_addr_set; /**< Set a MAC address */
> > + eth_stashing_rx_hints_set_t stashing_rx_hints_set; /**< Set Rx cache stashing*/
> > + eth_stashing_tx_hints_set_t stashing_tx_hints_set; /**< Set Tx cache stashing*/
> > + /** Get supported stashing hints*/
> > + eth_stashing_capabilities_get_t stashing_capabilities_get;
> > /** Set list of multicast addresses */
> > eth_set_mc_addr_list_t set_mc_addr_list;
> > mtu_set_t mtu_set; /**< Set MTU */
>
> Since eth_dev_ops is visible in application binary, it is part of the ABI.
> Therefore it can not be changed until 25.11 release.
The layout of eth_dev_ops is not exposed to applications as it is in a
private header.
Could you clarify where you see a breakage for an application?
I see an ABI breakage for out of tree drivers though.
This could be avoided by moving those added ops at the end of the struct?
Your right my test was crude. Just do build and look at symbol table of
static linked binary.
I was confused since pointer is exposed but not data structure
On Thu, Dec 5, 2024, 07:40 David Marchand <david.marchand@redhat.com> wrote:
> On Tue, Dec 3, 2024 at 10:13 PM Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> >
> > On Mon, 21 Oct 2024 01:52:46 +0000
> > Wathsala Vithanage <wathsala.vithanage@arm.com> wrote:
> >
> > > Extend the ethdev library to enable the stashing of different data
> > > objects, such as the ones listed below, into CPU caches directly
> > > from the NIC.
> > >
> > > - Rx/Tx queue descriptors
> > > - Rx packets
> > > - Packet headers
> > > - packet payloads
> > > - Data of a packet at an offset from the start of the packet
> > >
> > > The APIs are designed in a hardware/vendor agnostic manner such that
> > > supporting PMDs could use any capabilities available in the underlying
> > > hardware for fine-grained stashing of data objects into a CPU cache
> > > (e.g., Steering Tags int PCIe TLP Processing Hints).
> > >
> > > The API provides an interface to query the availability of stashing
> > > capabilities, i.e., platform/NIC support, stashable object types, etc,
> > > via the rte_eth_dev_stashing_capabilities_get interface.
> > >
> > > The function pair rte_eth_dev_stashing_rx_config_set and
> > > rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU,
> > > cache level, and data object types) on the Rx and Tx queues.
> > >
> > > PMDs that support stashing must register their implementations with the
> > > following eth_dev_ops callbacks, which are invoked by the ethdev
> > > functions listed above.
> > >
> > > - stashing_capabilities_get
> > > - stashing_rx_hints_set
> > > - stashing_tx_hints_set
> > >
> > > Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
> > > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > > Reviewed-by: Dhruv Tripathi <dhruv.tripathi@arm.com>
> > >
> > > ---
> > > lib/ethdev/ethdev_driver.h | 66 +++++++++++++++
> > > lib/ethdev/rte_ethdev.c | 120 +++++++++++++++++++++++++++
> > > lib/ethdev/rte_ethdev.h | 161 +++++++++++++++++++++++++++++++++++++
> > > lib/ethdev/version.map | 4 +
> > > 4 files changed, 351 insertions(+)
> > >
> > > diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
> > > index 1fd4562b40..7caaea54a8 100644
> > > --- a/lib/ethdev/ethdev_driver.h
> > > +++ b/lib/ethdev/ethdev_driver.h
> > > @@ -1367,6 +1367,68 @@ enum rte_eth_dev_operation {
> > > typedef uint64_t (*eth_get_restore_flags_t)(struct rte_eth_dev *dev,
> > > enum rte_eth_dev_operation
> op);
> > >
> > > +/**
> > > + * @internal
> > > + * Set cache stashing hints in Rx queue.
> > > + *
> > > + * @param dev
> > > + * Port (ethdev) handle.
> > > + * @param queue_id
> > > + * Rx queue.
> > > + * @param config
> > > + * Stashing hints configuration for the queue.
> > > + *
> > > + * @return
> > > + * -ENOTSUP if the device or the platform does not support cache
> stashing.
> > > + * -ENOSYS if the underlying PMD hasn't implemented cache stashing
> feature.
> > > + * -EINVAL on invalid arguments.
> > > + * 0 on success.
> > > + */
> > > +typedef int (*eth_stashing_rx_hints_set_t)(struct rte_eth_dev *dev,
> uint16_t queue_id,
> > > + struct
> rte_eth_stashing_config *config);
> > > +
> > > +/**
> > > + * @internal
> > > + * Set cache stashing hints in Tx queue.
> > > + *
> > > + * @param dev
> > > + * Port (ethdev) handle.
> > > + * @param queue_id
> > > + * Tx queue.
> > > + * @param config
> > > + * Stashing hints configuration for the queue.
> > > + *
> > > + * @return
> > > + * -ENOTSUP if the device or the platform does not support cache
> stashing.
> > > + * -ENOSYS if the underlying PMD hasn't implemented cache stashing
> feature.
> > > + * -EINVAL on invalid arguments.
> > > + * 0 on success.
> > > + */
> > > +typedef int (*eth_stashing_tx_hints_set_t)(struct rte_eth_dev *dev,
> uint16_t queue_id,
> > > + struct
> rte_eth_stashing_config *config);
> > > +
> > > +/**
> > > + * @internal
> > > + * Get cache stashing object types supported in the ethernet device.
> > > + * The return value indicates availability of stashing hints support
> > > + * in the hardware and the PMD.
> > > + *
> > > + * @param dev
> > > + * Port (ethdev) handle.
> > > + * @param objects
> > > + * PMD sets supported bits on return.
> > > + *
> > > + * @return
> > > + * -ENOTSUP if the device or the platform does not support cache
> stashing.
> > > + * -ENOSYS if the underlying PMD hasn't implemented cache stashing
> feature.
> > > + * -EINVAL on NULL values for types or hints parameters.
> > > + * On return, types and hints parameters will have bits set for
> supported
> > > + * object types and hints.
> > > + * 0 on success.
> > > + */
> > > +typedef int (*eth_stashing_capabilities_get_t)(struct rte_eth_dev
> *dev,
> > > + uint16_t *objects);
> > > +
> > > /**
> > > * @internal A structure containing the functions exported by an
> Ethernet driver.
> > > */
> > > @@ -1393,6 +1455,10 @@ struct eth_dev_ops {
> > > eth_mac_addr_remove_t mac_addr_remove; /**< Remove MAC
> address */
> > > eth_mac_addr_add_t mac_addr_add; /**< Add a MAC address
> */
> > > eth_mac_addr_set_t mac_addr_set; /**< Set a MAC address
> */
> > > + eth_stashing_rx_hints_set_t stashing_rx_hints_set; /**< Set Rx
> cache stashing*/
> > > + eth_stashing_tx_hints_set_t stashing_tx_hints_set; /**< Set Tx
> cache stashing*/
> > > + /** Get supported stashing hints*/
> > > + eth_stashing_capabilities_get_t stashing_capabilities_get;
> > > /** Set list of multicast addresses */
> > > eth_set_mc_addr_list_t set_mc_addr_list;
> > > mtu_set_t mtu_set; /**< Set MTU */
> >
> > Since eth_dev_ops is visible in application binary, it is part of the
> ABI.
> > Therefore it can not be changed until 25.11 release.
>
> The layout of eth_dev_ops is not exposed to applications as it is in a
> private header.
> Could you clarify where you see a breakage for an application?
>
>
> I see an ABI breakage for out of tree drivers though.
> This could be avoided by moving those added ops at the end of the struct?
>
>
> --
> David Marchand
>
>
@@ -1367,6 +1367,68 @@ enum rte_eth_dev_operation {
typedef uint64_t (*eth_get_restore_flags_t)(struct rte_eth_dev *dev,
enum rte_eth_dev_operation op);
+/**
+ * @internal
+ * Set cache stashing hints in Rx queue.
+ *
+ * @param dev
+ * Port (ethdev) handle.
+ * @param queue_id
+ * Rx queue.
+ * @param config
+ * Stashing hints configuration for the queue.
+ *
+ * @return
+ * -ENOTSUP if the device or the platform does not support cache stashing.
+ * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature.
+ * -EINVAL on invalid arguments.
+ * 0 on success.
+ */
+typedef int (*eth_stashing_rx_hints_set_t)(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_stashing_config *config);
+
+/**
+ * @internal
+ * Set cache stashing hints in Tx queue.
+ *
+ * @param dev
+ * Port (ethdev) handle.
+ * @param queue_id
+ * Tx queue.
+ * @param config
+ * Stashing hints configuration for the queue.
+ *
+ * @return
+ * -ENOTSUP if the device or the platform does not support cache stashing.
+ * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature.
+ * -EINVAL on invalid arguments.
+ * 0 on success.
+ */
+typedef int (*eth_stashing_tx_hints_set_t)(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_stashing_config *config);
+
+/**
+ * @internal
+ * Get cache stashing object types supported in the ethernet device.
+ * The return value indicates availability of stashing hints support
+ * in the hardware and the PMD.
+ *
+ * @param dev
+ * Port (ethdev) handle.
+ * @param objects
+ * PMD sets supported bits on return.
+ *
+ * @return
+ * -ENOTSUP if the device or the platform does not support cache stashing.
+ * -ENOSYS if the underlying PMD hasn't implemented cache stashing feature.
+ * -EINVAL on NULL values for types or hints parameters.
+ * On return, types and hints parameters will have bits set for supported
+ * object types and hints.
+ * 0 on success.
+ */
+typedef int (*eth_stashing_capabilities_get_t)(struct rte_eth_dev *dev,
+ uint16_t *objects);
+
/**
* @internal A structure containing the functions exported by an Ethernet driver.
*/
@@ -1393,6 +1455,10 @@ struct eth_dev_ops {
eth_mac_addr_remove_t mac_addr_remove; /**< Remove MAC address */
eth_mac_addr_add_t mac_addr_add; /**< Add a MAC address */
eth_mac_addr_set_t mac_addr_set; /**< Set a MAC address */
+ eth_stashing_rx_hints_set_t stashing_rx_hints_set; /**< Set Rx cache stashing*/
+ eth_stashing_tx_hints_set_t stashing_tx_hints_set; /**< Set Tx cache stashing*/
+ /** Get supported stashing hints*/
+ eth_stashing_capabilities_get_t stashing_capabilities_get;
/** Set list of multicast addresses */
eth_set_mc_addr_list_t set_mc_addr_list;
mtu_set_t mtu_set; /**< Set MTU */
@@ -153,6 +153,7 @@ static const struct {
{RTE_ETH_DEV_CAPA_RXQ_SHARE, "RXQ_SHARE"},
{RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP, "FLOW_RULE_KEEP"},
{RTE_ETH_DEV_CAPA_FLOW_SHARED_OBJECT_KEEP, "FLOW_SHARED_OBJECT_KEEP"},
+ {RTE_ETH_DEV_CAPA_CACHE_STASHING, "CACHE_STASHING"},
};
enum {
@@ -7163,4 +7164,123 @@ int rte_eth_dev_map_aggr_tx_affinity(uint16_t port_id, uint16_t tx_queue_id,
return ret;
}
+int
+rte_eth_dev_validate_stashing_config(uint16_t port_id, uint16_t queue_id,
+ uint8_t queue_direction,
+ struct rte_eth_stashing_config *config)
+{
+ struct rte_eth_dev *dev;
+ struct rte_eth_dev_info dev_info;
+ uint16_t nb_queues;
+
+ RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+
+ if (!config) {
+ RTE_ETHDEV_LOG_LINE(ERR, "Invalid stashing configuration");
+ return -EINVAL;
+ }
+
+ /*
+ * Check for invalid objects
+ */
+ if (!RTE_ETH_DEV_STASH_OBJECTS_VALID(config->objects)) {
+ RTE_ETHDEV_LOG_LINE(ERR, "Invalid stashing objects");
+ return -EINVAL;
+ }
+
+ dev = &rte_eth_devices[port_id];
+
+ nb_queues = (queue_direction == RTE_ETH_DEV_RX_QUEUE) ?
+ dev->data->nb_rx_queues :
+ dev->data->nb_tx_queues;
+
+ if (queue_id >= nb_queues) {
+ RTE_ETHDEV_LOG_LINE(ERR, "Invalid Rx queue_id=%u", queue_id);
+ return -EINVAL;
+ }
+
+ rte_eth_dev_info_get(port_id, &dev_info);
+
+ if ((dev_info.dev_capa & RTE_ETH_DEV_CAPA_CACHE_STASHING) !=
+ RTE_ETH_DEV_CAPA_CACHE_STASHING)
+ return -ENOTSUP;
+
+ if (*dev->dev_ops->stashing_rx_hints_set == NULL ||
+ *dev->dev_ops->stashing_tx_hints_set == NULL) {
+ RTE_ETHDEV_LOG_LINE(ERR, "Stashing hints are not implemented "
+ "in %s for %s", dev_info.driver_name,
+ dev_info.device->name);
+ return -ENOSYS;
+ }
+
+ return 0;
+}
+
+int
+rte_eth_dev_stashing_rx_config_set(uint16_t port_id, uint16_t queue_id,
+ struct rte_eth_stashing_config *config)
+{
+ struct rte_eth_dev *dev;
+
+ int ret = rte_eth_dev_validate_stashing_config(port_id, queue_id,
+ RTE_ETH_DEV_RX_QUEUE,
+ config);
+ if (ret < 0)
+ return ret;
+
+ dev = &rte_eth_devices[port_id];
+
+ return eth_err(port_id,
+ (*dev->dev_ops->stashing_rx_hints_set)(dev, queue_id,
+ config));
+}
+
+int
+rte_eth_dev_stashing_tx_config_set(uint16_t port_id, uint16_t queue_id,
+ struct rte_eth_stashing_config *config)
+{
+ struct rte_eth_dev *dev;
+
+ int ret = rte_eth_dev_validate_stashing_config(port_id, queue_id,
+ RTE_ETH_DEV_TX_QUEUE,
+ config);
+ if (ret < 0)
+ return ret;
+
+ dev = &rte_eth_devices[port_id];
+
+ return eth_err(port_id,
+ (*dev->dev_ops->stashing_rx_hints_set) (dev, queue_id,
+ config));
+}
+
+int
+rte_eth_dev_stashing_capabilities_get(uint16_t port_id, uint16_t *objects)
+{
+ struct rte_eth_dev *dev;
+ struct rte_eth_dev_info dev_info;
+
+ RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+
+ if (!objects)
+ return -EINVAL;
+
+ dev = &rte_eth_devices[port_id];
+ rte_eth_dev_info_get(port_id, &dev_info);
+
+ if ((dev_info.dev_capa & RTE_ETH_DEV_CAPA_CACHE_STASHING) !=
+ RTE_ETH_DEV_CAPA_CACHE_STASHING)
+ return -ENOTSUP;
+
+ if (*dev->dev_ops->stashing_capabilities_get == NULL) {
+ RTE_ETHDEV_LOG_LINE(ERR, "Stashing hints are not implemented "
+ "in %s for %s", dev_info.driver_name,
+ dev_info.device->name);
+ return -ENOSYS;
+ }
+ return eth_err(port_id,
+ (*dev->dev_ops->stashing_capabilities_get)
+ (dev, objects));
+}
+
RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO);
@@ -1653,6 +1653,9 @@ struct rte_eth_conf {
#define RTE_ETH_DEV_CAPA_FLOW_SHARED_OBJECT_KEEP RTE_BIT64(4)
/**@}*/
+/** Device supports stashing to CPU/system caches. */
+#define RTE_ETH_DEV_CAPA_CACHE_STASHING RTE_BIT64(5)
+
/*
* Fallback default preferred Rx/Tx port parameters.
* These are used if an application requests default parameters
@@ -1824,6 +1827,7 @@ struct rte_eth_dev_info {
struct rte_eth_dev_portconf default_txportconf;
/** Generic device capabilities (RTE_ETH_DEV_CAPA_). */
uint64_t dev_capa;
+ uint16_t stashing_capa;
/**
* Switching information for ports on a device with a
* embedded managed interconnect/switch.
@@ -6115,6 +6119,163 @@ int rte_eth_cman_config_set(uint16_t port_id, const struct rte_eth_cman_config *
__rte_experimental
int rte_eth_cman_config_get(uint16_t port_id, struct rte_eth_cman_config *config);
+
+
+/** Queue type is RX. */
+#define RTE_ETH_DEV_RX_QUEUE 0
+/** Queue type is TX. */
+#define RTE_ETH_DEV_TX_QUEUE 1
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change, or be removed, without prior notice
+ *
+ * A structure used for configuring the cache stashing hints.
+ */
+struct rte_eth_stashing_config {
+ /** ID of the Processor/Container the stashing hints are
+ * applied to
+ */
+ uint16_t lcore_id;
+ /** Set if the target is a CPU containeri.lcore_id will be
+ * used to derive container ID
+ */
+ uint16_t container : 1;
+ uint16_t padding : 7;
+ /** Cache level of the CPU specified by the cpu_id the
+ * stashing hints are applied to
+ */
+ uint16_t cache_level : 8;
+ /** Object types the configuration is applied to
+ */
+ uint16_t objects;
+ /** The offset if RTE_ETH_DEV_STASH_OBJECT_OFFSET bit is set
+ * in objects
+ */
+ off_t offset;
+};
+
+/**@{@name Stashable Rx/Tx queue object types supported by the ethernet device
+ *@see rte_eth_dev_stashing_capabilities_get
+ *@see rte_eth_dev_stashing_rx_config_set
+ *@see rte_eth_dev_stashing_tx_config_set
+ */
+
+/**
+ * Apply stashing hint to data at a given offset from the start of a
+ * received packet.
+ */
+#define RTE_ETH_DEV_STASH_OBJECT_OFFSET 0x0001
+
+/** Apply stashing hint to an rx descriptor. */
+#define RTE_ETH_DEV_STASH_OBJECT_DESC 0x0002
+
+/** Apply stashing hint to a header of a received packet. */
+#define RTE_ETH_DEV_STASH_OBJECT_HEADER 0x0004
+
+/** Apply stashing hint to a payload of a received packet. */
+#define RTE_ETH_DEV_STASH_OBJECT_PAYLOAD 0x0008
+
+#define __RTE_ETH_DEV_STASH_OBJECT_MASK 0x000f
+/**@}*/
+
+#define RTE_ETH_DEV_STASH_OBJECTS_VALID(t) \
+ ((!((t) & (~__RTE_ETH_DEV_STASH_OBJECT_MASK))) && (t))
+
+/**
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * @internal
+ * Helper function to validate stashing hints configuration.
+ */
+__rte_experimental
+int rte_eth_dev_validate_stashing_config(uint16_t port_id, uint16_t queue_id,
+ uint8_t queue_direction,
+ struct rte_eth_stashing_config *config);
+
+/**
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Provide cache stashing hints for improved memory access latencies for
+ * packets received by the NIC.
+ * This feature is available only in supported NICs and platforms.
+ *
+ * @param port_id
+ * The port identifier of the Ethernet device.
+ * @param queue_id
+ * The index of the receive queue to which hints are applied.
+ * @param config
+ * Stashing configuration.
+ * @return
+ * - (-ENODEV) on incorrect port_ids.
+ * - (-EINVAL) if both RX and TX object types used in conjuection in objects
+ * parameter.
+ * - (-EINVAL) on invalid queue_id.
+ * - (-ENOTSUP) if RTE_ETH_DEV_CAPA_CACHE_STASHING capability is unavailable.
+ * - (-ENOSYS) if PMD does not implement cache stashing hints.
+ * - (0) on Success.
+ */
+__rte_experimental
+int rte_eth_dev_stashing_rx_config_set(uint16_t port_id, uint16_t queue_id,
+ struct rte_eth_stashing_config *config);
+
+/**
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Configure cache stashing for improved memory access latencies for Tx
+ * queue completion descriptors being sent to host system by the NIC.
+ * This feature is available only in supported NICs and platforms.
+ *
+ * @param port_id
+ * The port identifier of the Ethernet device.
+ * @param queue_id
+ * The index of the receive queue to which hints are applied.
+ * @param config
+ * Stashing configuration.
+ * @return
+ * - (-ENODEV) on incorrect port_ids.
+ * - (-EINVAL) if both RX and TX object types are used in conjuection in objects
+ * parameter.
+ * - (-EINVAL) if hints are incompatible with TX queues.
+ * - (-EINVAL) on invalid queue_id.
+ * - (-ENOTSUP) if RTE_ETH_DEV_CAPA_CACHE_STASHING capability is unavailable.
+ * - (-ENOSYS) if PMD does not implement cache stashing hints.
+ * - (0) on Success.
+ */
+__rte_experimental
+int rte_eth_dev_stashing_tx_config_set(uint16_t port_id, uint16_t queue_id,
+ struct rte_eth_stashing_config *config);
+
+/**
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Discover cache stashing objects supported in the ethernet device.
+ *
+ * @param port_id
+ * The port identifier of the Ethernet device.
+ * @param objects
+ * Supported objects vector set by the ethernet device.
+ * @return
+ * On return types and hints parameters will have bits set for supported
+ * object types.
+ * - (-ENOTSUP) if the device or the platform does not support cache stashing.
+ * - (-ENOSYS) if the underlying PMD hasn't implemented cache stashing
+ * feature.
+ * - (-EINVAL) on NULL values for types or hints parameters.
+ * - (0) on success.
+ */
+__rte_experimental
+int rte_eth_dev_stashing_capabilities_get(uint16_t port_id, uint16_t *objects);
+
#include <rte_ethdev_core.h>
#ifdef __cplusplus
@@ -337,6 +337,10 @@ EXPERIMENTAL {
rte_eth_timesync_adjust_freq;
rte_flow_async_create_by_index_with_pattern;
rte_tm_node_query;
+ rte_eth_dev_stashing_rx_config_set;
+ rte_eth_dev_stashing_tx_config_set;
+ rte_eth_dev_stashing_capabilities_get;
+ rte_eth_dev_validate_stashing_config;
};
INTERNAL {