[v3] net/af_packet: cache align Rx/Tx structs
Checks
Commit Message
Cache align Rx and Tx queue struct to avoid false sharing.
The RX struct happens to be 64 bytes on x86_64 already, so cache
alignment has no effect there, but it does on 32-bit ISAs.
The TX struct is 56 bytes on x86_64.
Both structs keep counters, and in the RX case they are updated even
for empty polls.
v3: Move __rte_cache_aligned directive to a MSVC-compatible location.
Fixes: 364e08f2bbc0 ("af_packet: add PMD for AF_PACKET-based virtual devices")
Cc: stable@dpdk.org
Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
Comments
> From: Mattias Rönnblom [mailto:mattias.ronnblom@ericsson.com]
> Sent: Friday, 26 April 2024 11.05
>
> Cache align Rx and Tx queue struct to avoid false sharing.
Should an RTE_CACHE_GUARD be added at the end of these two structs?
If not,
Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
On Fri, 26 Apr 2024 11:05:02 +0200
Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote:
> Cache align Rx and Tx queue struct to avoid false sharing.
>
> The RX struct happens to be 64 bytes on x86_64 already, so cache
> alignment has no effect there, but it does on 32-bit ISAs.
>
> The TX struct is 56 bytes on x86_64.
>
> Both structs keep counters, and in the RX case they are updated even
> for empty polls.
>
> v3: Move __rte_cache_aligned directive to a MSVC-compatible location.
>
> Fixes: 364e08f2bbc0 ("af_packet: add PMD for AF_PACKET-based virtual devices")
> Cc: stable@dpdk.org
>
> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Looks good.
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
If you want to go further with this:
- could rx and tx rings be next to each other so that any of the lookahead
in cpu helps, not hurts?
- what about secondary process support
- would sendmmsg() and recvmmsg() help for bursting.
On Fri, Apr 26, 2024 at 11:05:02AM +0200, Mattias Rönnblom wrote:
> Cache align Rx and Tx queue struct to avoid false sharing.
>
> The RX struct happens to be 64 bytes on x86_64 already, so cache
> alignment has no effect there, but it does on 32-bit ISAs.
>
> The TX struct is 56 bytes on x86_64.
>
> Both structs keep counters, and in the RX case they are updated even
> for empty polls.
>
> v3: Move __rte_cache_aligned directive to a MSVC-compatible location.
>
> Fixes: 364e08f2bbc0 ("af_packet: add PMD for AF_PACKET-based virtual devices")
> Cc: stable@dpdk.org
>
> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> ---
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
On 4/26/2024 4:41 PM, Tyler Retzlaff wrote:
> On Fri, Apr 26, 2024 at 11:05:02AM +0200, Mattias Rönnblom wrote:
>> Cache align Rx and Tx queue struct to avoid false sharing.
>>
>> The RX struct happens to be 64 bytes on x86_64 already, so cache
>> alignment has no effect there, but it does on 32-bit ISAs.
>>
>> The TX struct is 56 bytes on x86_64.
>>
>> Both structs keep counters, and in the RX case they are updated even
>> for empty polls.
>>
>> v3: Move __rte_cache_aligned directive to a MSVC-compatible location.
>>
>> Fixes: 364e08f2bbc0 ("af_packet: add PMD for AF_PACKET-based virtual devices")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>> ---
>
> Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
>
Applied to dpdk-next-net/main, thanks.
@@ -6,6 +6,7 @@
* All rights reserved.
*/
+#include <rte_common.h>
#include <rte_string_fns.h>
#include <rte_mbuf.h>
#include <ethdev_driver.h>
@@ -39,7 +40,7 @@
#define DFLT_FRAME_SIZE (1 << 11)
#define DFLT_FRAME_COUNT (1 << 9)
-struct pkt_rx_queue {
+struct __rte_cache_aligned pkt_rx_queue {
int sockfd;
struct iovec *rd;
@@ -55,7 +56,7 @@ struct pkt_rx_queue {
volatile unsigned long rx_bytes;
};
-struct pkt_tx_queue {
+struct __rte_cache_aligned pkt_tx_queue {
int sockfd;
unsigned int frame_data_size;