diff mbox series

net/i40e: increase descriptor queue length to 8160

Message ID 20240527161921.183866-1-igootorov@gmail.com (mailing list archive)
State Accepted, archived
Delegated to: Bruce Richardson
Headers
Series net/i40e: increase descriptor queue length to 8160 |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/loongarch-compilation success Compilation OK
ci/loongarch-unit-testing success Unit Testing PASS
ci/Intel-compilation success Compilation OK
ci/intel-Testing success Testing PASS
ci/intel-Functional success Functional PASS
ci/github-robot: build success github build: passed
ci/iol-abi-testing success Testing PASS
ci/iol-compile-amd64-testing success Testing PASS
ci/iol-broadcom-Functional success Functional Testing PASS
ci/iol-sample-apps-testing success Testing PASS
ci/iol-unit-arm64-testing success Testing PASS
ci/iol-compile-arm64-testing success Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS

Commit Message

Igor Gutorov May 27, 2024, 4:19 p.m. UTC 
According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
as QLEN in table 8-12, page 1083.

I've tested this change with an XXV710 NIC and it has positive effect on
performance under high load scenarios. Where previously I'd get
~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.

Signed-off-by: Igor Gutorov <igootorov@gmail.com>
---
 drivers/net/i40e/i40e_rxtx.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Igor Gutorov June 3, 2024, 10:41 a.m. UTC | #1 
On Mon, May 27, 2024 at 7:20 PM Igor Gutorov <igootorov@gmail.com> wrote:
>
> According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> as QLEN in table 8-12, page 1083.
>
> I've tested this change with an XXV710 NIC and it has positive effect on
> performance under high load scenarios. Where previously I'd get
> ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
>
> Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> ---
>  drivers/net/i40e/i40e_rxtx.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> index 2f2f890855..33fc9770d9 100644
> --- a/drivers/net/i40e/i40e_rxtx.h
> +++ b/drivers/net/i40e/i40e_rxtx.h
> @@ -25,7 +25,7 @@
>  #define I40E_RX_MAX_DATA_BUF_SIZE      (16 * 1024 - 128)
>
>  #define        I40E_MIN_RING_DESC      64
> -#define        I40E_MAX_RING_DESC      4096
> +#define        I40E_MAX_RING_DESC      8160
>
>  #define I40E_FDIR_NUM_TX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
>  #define I40E_FDIR_NUM_RX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> --
> 2.45.1
>

CC'ing to a different email (got an address rejected error, the
MAINTAINERS file might need an update).

Also, I've noticed that an X722 NIC is also supported by the i40e
driver, but it seems the datasheet isn't on Intel's website for that
NIC. Does anybody know if X722 has 4096 or 8160 usable descriptors? If
someone could test this patch against X722, that would be great.
Igor Gutorov June 5, 2024, 10:47 a.m. UTC | #2 
On Mon, Jun 3, 2024 at 1:41 PM Igor Gutorov <igootorov@gmail.com> wrote:
>
> On Mon, May 27, 2024 at 7:20 PM Igor Gutorov <igootorov@gmail.com> wrote:
> >
> > According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> > queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> > as QLEN in table 8-12, page 1083.
> >
> > I've tested this change with an XXV710 NIC and it has positive effect on
> > performance under high load scenarios. Where previously I'd get
> > ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
> >
> > Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> > ---
> >  drivers/net/i40e/i40e_rxtx.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> > index 2f2f890855..33fc9770d9 100644
> > --- a/drivers/net/i40e/i40e_rxtx.h
> > +++ b/drivers/net/i40e/i40e_rxtx.h
> > @@ -25,7 +25,7 @@
> >  #define I40E_RX_MAX_DATA_BUF_SIZE      (16 * 1024 - 128)
> >
> >  #define        I40E_MIN_RING_DESC      64
> > -#define        I40E_MAX_RING_DESC      4096
> > +#define        I40E_MAX_RING_DESC      8160
> >
> >  #define I40E_FDIR_NUM_TX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> >  #define I40E_FDIR_NUM_RX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> > --
> > 2.45.1
> >
>
> CC'ing to a different email (got an address rejected error, the
> MAINTAINERS file might need an update).
>
> Also, I've noticed that an X722 NIC is also supported by the i40e
> driver, but it seems the datasheet isn't on Intel's website for that
> NIC. Does anybody know if X722 has 4096 or 8160 usable descriptors? If
> someone could test this patch against X722, that would be great.

Apologies for pinging again. Who should I send my patch to? The
MAINTAINERS file shows Yuying Zhang as the maintainer of the net/i40e
driver and contains 2 emails, but both Yuying.Zhang@intel.com and
yuying.zhang@intel.com emails are non-existent (550 Address rejected).
Morten Brørup June 6, 2024, 9:18 a.m. UTC | #3 
> From: Igor Gutorov [mailto:igootorov@gmail.com]
> Sent: Wednesday, 5 June 2024 12.47
> 
> On Mon, Jun 3, 2024 at 1:41 PM Igor Gutorov <igootorov@gmail.com> wrote:
> >
> > On Mon, May 27, 2024 at 7:20 PM Igor Gutorov <igootorov@gmail.com> wrote:
> > >
> > > According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> > > queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> > > as QLEN in table 8-12, page 1083.
> > >
> > > I've tested this change with an XXV710 NIC and it has positive effect on
> > > performance under high load scenarios. Where previously I'd get
> > > ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
> > >
> > > Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> > > ---
> > >  drivers/net/i40e/i40e_rxtx.h | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> > > index 2f2f890855..33fc9770d9 100644
> > > --- a/drivers/net/i40e/i40e_rxtx.h
> > > +++ b/drivers/net/i40e/i40e_rxtx.h
> > > @@ -25,7 +25,7 @@
> > >  #define I40E_RX_MAX_DATA_BUF_SIZE      (16 * 1024 - 128)
> > >
> > >  #define        I40E_MIN_RING_DESC      64
> > > -#define        I40E_MAX_RING_DESC      4096
> > > +#define        I40E_MAX_RING_DESC      8160
> > >
> > >  #define I40E_FDIR_NUM_TX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> > >  #define I40E_FDIR_NUM_RX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> > > --
> > > 2.45.1
> > >
> >
> > CC'ing to a different email (got an address rejected error, the
> > MAINTAINERS file might need an update).
> >
> > Also, I've noticed that an X722 NIC is also supported by the i40e
> > driver, but it seems the datasheet isn't on Intel's website for that
> > NIC. Does anybody know if X722 has 4096 or 8160 usable descriptors?

It does.
Refer to table 38-419, page 3086 in the Intel C620 Chipset datasheet:
https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/c620-series-chipset-datasheet.pdf

> > If
> > someone could test this patch against X722, that would be great.

I don't have one, so I cannot test it. But:
Acked-by: Morten Brørup <mb@smartsharesystems.com>

> 
> Apologies for pinging again. Who should I send my patch to? The
> MAINTAINERS file shows Yuying Zhang as the maintainer of the net/i40e
> driver and contains 2 emails, but both Yuying.Zhang@intel.com and
> yuying.zhang@intel.com emails are non-existent (550 Address rejected).

The i40e DPDK driver is currently without maintainers.
Sending the patch to the dev@dpdk.org mailing list suffices.
Bruce Richardson June 18, 2024, 12:12 p.m. UTC | #4 
On Mon, May 27, 2024 at 07:19:21PM +0300, Igor Gutorov wrote:
> According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> as QLEN in table 8-12, page 1083.
> 
> I've tested this change with an XXV710 NIC and it has positive effect on
> performance under high load scenarios. Where previously I'd get
> ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
> 
> Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> ---
>  drivers/net/i40e/i40e_rxtx.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> index 2f2f890855..33fc9770d9 100644
> --- a/drivers/net/i40e/i40e_rxtx.h
> +++ b/drivers/net/i40e/i40e_rxtx.h
> @@ -25,7 +25,7 @@
>  #define I40E_RX_MAX_DATA_BUF_SIZE	(16 * 1024 - 128)
>  
>  #define	I40E_MIN_RING_DESC	64
> -#define	I40E_MAX_RING_DESC	4096
> +#define	I40E_MAX_RING_DESC	8160
>  

since the ring memory allocation in the driver always seems to allocate
space for up to the max number of descriptors, this will lead to an
increase of memory footprint of ~128k per Rx Q, and ~64k per Tx Q. However,
I'd view this as relatively harmless given what we have now. If it is an
issue, we should look to fix it in a separate patch, rather than blocking
this simple change.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Bruce Richardson June 28, 2024, 12:28 p.m. UTC | #5 
On Tue, Jun 18, 2024 at 01:12:13PM +0100, Bruce Richardson wrote:
> On Mon, May 27, 2024 at 07:19:21PM +0300, Igor Gutorov wrote:
> > According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> > queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> > as QLEN in table 8-12, page 1083.
> > 
> > I've tested this change with an XXV710 NIC and it has positive effect on
> > performance under high load scenarios. Where previously I'd get
> > ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
> > 
> > Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> > ---
> >  drivers/net/i40e/i40e_rxtx.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> > index 2f2f890855..33fc9770d9 100644
> > --- a/drivers/net/i40e/i40e_rxtx.h
> > +++ b/drivers/net/i40e/i40e_rxtx.h
> > @@ -25,7 +25,7 @@
> >  #define I40E_RX_MAX_DATA_BUF_SIZE	(16 * 1024 - 128)
> >  
> >  #define	I40E_MIN_RING_DESC	64
> > -#define	I40E_MAX_RING_DESC	4096
> > +#define	I40E_MAX_RING_DESC	8160
> >  
> 
> since the ring memory allocation in the driver always seems to allocate
> space for up to the max number of descriptors, this will lead to an
> increase of memory footprint of ~128k per Rx Q, and ~64k per Tx Q. However,
> I'd view this as relatively harmless given what we have now. If it is an
> issue, we should look to fix it in a separate patch, rather than blocking
> this simple change.
> 
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Patch applied to dpdk-next-net-intel,

Thanks,
/Bruce
diff mbox series

Patch

diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
index 2f2f890855..33fc9770d9 100644
--- a/drivers/net/i40e/i40e_rxtx.h
+++ b/drivers/net/i40e/i40e_rxtx.h
@@ -25,7 +25,7 @@ 
 #define I40E_RX_MAX_DATA_BUF_SIZE	(16 * 1024 - 128)
 
 #define	I40E_MIN_RING_DESC	64
-#define	I40E_MAX_RING_DESC	4096
+#define	I40E_MAX_RING_DESC	8160
 
 #define I40E_FDIR_NUM_TX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
 #define I40E_FDIR_NUM_RX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)