diff mbox series

[v4,1/2] examples/l3fwd: relax RSS requirement with option

Message ID 20231113160255.124385-2-taozj888@163.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series example/l3fwd: relax l3fwd rx RSS/Offload if needed |

Checks

Context Check Description
ci/checkpatch success coding style OK

Commit Message

Trevor Tao Nov. 13, 2023, 4:02 p.m. UTC 
  Now the port Rx mq_mode had been set to RTE_ETH_MQ_RX_RSS
by default, but some hw and/or virtual interface does not
support the RSS and offload mode presupposed, e.g., some
virtio interfaces in the cloud don't support
RSS and the error msg may like:

virtio_dev_configure(): RSS support requested but not supported by
the device
Port0 dev_configure = -95

So to enable the l3fwd running in that environment, the Rx mode requirement
can be relaxed to reflect the hardware feature reality here, and the l3fwd
can run smoothly then.

An option named "disable-rss" is added to disable the RX RSS explicitly,
and it's disabled by default.

Signed-off-by: Trevor Tao <taozj888@163.com>
---
 examples/l3fwd/main.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

Comments

Stephen Hemminger Nov. 13, 2023, 4:34 p.m. UTC | #1 
On Mon, 13 Nov 2023 16:02:54 +0000
Trevor Tao <taozj888@163.com> wrote:

> Now the port Rx mq_mode had been set to RTE_ETH_MQ_RX_RSS
> by default, but some hw and/or virtual interface does not
> support the RSS and offload mode presupposed, e.g., some
> virtio interfaces in the cloud don't support
> RSS and the error msg may like:
> 
> virtio_dev_configure(): RSS support requested but not supported by
> the device
> Port0 dev_configure = -95
> 
> So to enable the l3fwd running in that environment, the Rx mode requirement
> can be relaxed to reflect the hardware feature reality here, and the l3fwd
> can run smoothly then.
> 
> An option named "disable-rss" is added to disable the RX RSS explicitly,
> and it's disabled by default.
> 
> Signed-off-by: Trevor Tao <taozj888@163.com>

Why is running with > 1 rx queue and RSS disabled useful?
What happens is all packets arrive on 1st queue and you burn a thread
polling an always empty queue.

I would prefer not adding yet another command line option and
just "do the right thing". If number of rx queues > 1, check that device
supports RSS before using it. If device does not support RSS give
an error and exit.
Stephen Hemminger Nov. 13, 2023, 5:53 p.m. UTC | #2 
On Mon, 13 Nov 2023 08:34:47 -0800
Stephen Hemminger <stephen@networkplumber.org> wrote:

> On Mon, 13 Nov 2023 16:02:54 +0000
> Trevor Tao <taozj888@163.com> wrote:
> 
> > Now the port Rx mq_mode had been set to RTE_ETH_MQ_RX_RSS
> > by default, but some hw and/or virtual interface does not
> > support the RSS and offload mode presupposed, e.g., some
> > virtio interfaces in the cloud don't support
> > RSS and the error msg may like:
> > 
> > virtio_dev_configure(): RSS support requested but not supported by
> > the device
> > Port0 dev_configure = -95
> > 
> > So to enable the l3fwd running in that environment, the Rx mode requirement
> > can be relaxed to reflect the hardware feature reality here, and the l3fwd
> > can run smoothly then.
> > 
> > An option named "disable-rss" is added to disable the RX RSS explicitly,
> > and it's disabled by default.
> > 
> > Signed-off-by: Trevor Tao <taozj888@163.com>  
> 
> Why is running with > 1 rx queue and RSS disabled useful?
> What happens is all packets arrive on 1st queue and you burn a thread
> polling an always empty queue.
> 
> I would prefer not adding yet another command line option and
> just "do the right thing". If number of rx queues > 1, check that device
> supports RSS before using it. If device does not support RSS give
> an error and exit.

Something like this maybe:
diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c
index 6063eb139900..c747ef8aadfb 100644
--- a/examples/l3fwd/main.c
+++ b/examples/l3fwd/main.c
@@ -1257,8 +1257,15 @@ l3fwd_poll_resource_setup(void)
 		local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
 			dev_info.flow_type_rss_offloads;
 
-		if (dev_info.max_rx_queues == 1)
+		if (nb_rx_queue > dev_info.max_rx_queues)
+			rte_exit(EXIT_FAILURE, "Port %u only supports %u queues\n",
+				 portid, dev_info.max_rx_queues);
+
+		if (nb_rx_queue == 1)
 			local_port_conf.rxmode.mq_mode = RTE_ETH_MQ_RX_NONE;
+		else if (!(dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_RSS_HASH))
+			rte_exit(EXIT_FAILURE, "Port %u does not support RSS but %u queues requested\n",
+				 portid, nb_rx_queue);
 
 		if (local_port_conf.rx_adv_conf.rss_conf.rss_hf !=
 				port_conf.rx_adv_conf.rss_conf.rss_hf) {
Konstantin Ananyev Nov. 13, 2023, 8:01 p.m. UTC | #3 
> > > Now the port Rx mq_mode had been set to RTE_ETH_MQ_RX_RSS
> > > by default, but some hw and/or virtual interface does not
> > > support the RSS and offload mode presupposed, e.g., some
> > > virtio interfaces in the cloud don't support
> > > RSS and the error msg may like:
> > >
> > > virtio_dev_configure(): RSS support requested but not supported by
> > > the device
> > > Port0 dev_configure = -95
> > >
> > > So to enable the l3fwd running in that environment, the Rx mode requirement
> > > can be relaxed to reflect the hardware feature reality here, and the l3fwd
> > > can run smoothly then.
> > >
> > > An option named "disable-rss" is added to disable the RX RSS explicitly,
> > > and it's disabled by default.
> > >
> > > Signed-off-by: Trevor Tao <taozj888@163.com>
> >
> > Why is running with > 1 rx queue and RSS disabled useful?
> > What happens is all packets arrive on 1st queue and you burn a thread
> > polling an always empty queue.
> >
> > I would prefer not adding yet another command line option and
> > just "do the right thing". If number of rx queues > 1, check that device
> > supports RSS before using it. If device does not support RSS give
> > an error and exit.

I think that's pretty much what we have right now in l3fwd.
As I understand, the rational was: some virtual NICs does not support RSS (vritio?),
but there is still a desire to run l3fwd app on such devices.
I presume for performance comparison purposes.
So the idea was to relax NIC requirements and allow to run l3fwd with disabled RSS.
I suppose the main question here: what then will be the mechanism to distribute
packets over different RX queues?
From my understanding with '--disable-rss' option enabled it is undefined and 
depends on particular NIC.
In case of virtio, my guess - it will be hypervisor counterpart (vhost) to determine packet distribution.

> 
> Something like this maybe:
> diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c
> index 6063eb139900..c747ef8aadfb 100644
> --- a/examples/l3fwd/main.c
> +++ b/examples/l3fwd/main.c
> @@ -1257,8 +1257,15 @@ l3fwd_poll_resource_setup(void)
>  		local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
>  			dev_info.flow_type_rss_offloads;
> 
> -		if (dev_info.max_rx_queues == 1)
> +		if (nb_rx_queue > dev_info.max_rx_queues)
> +			rte_exit(EXIT_FAILURE, "Port %u only supports %u queues\n",
> +				 portid, dev_info.max_rx_queues);
> +
> +		if (nb_rx_queue == 1)
>  			local_port_conf.rxmode.mq_mode = RTE_ETH_MQ_RX_NONE;
> +		else if (!(dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_RSS_HASH))
> +			rte_exit(EXIT_FAILURE, "Port %u does not support RSS but %u queues requested\n",
> +				 portid, nb_rx_queue);
> 
>  		if (local_port_conf.rx_adv_conf.rss_conf.rss_hf !=
>  				port_conf.rx_adv_conf.rss_conf.rss_hf) {
diff mbox series

Patch

diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c
index 6063eb1399..b42365ef1b 100644
--- a/examples/l3fwd/main.c
+++ b/examples/l3fwd/main.c
@@ -73,6 +73,7 @@  static enum L3FWD_LOOKUP_MODE lookup_mode;
 static int numa_on = 1; /**< NUMA is enabled by default. */
 static int parse_ptype; /**< Parse packet type using rx callback, and */
 			/**< disabled by default */
+static int disable_rss; /**< Disable the RX RSS mode */
 static int per_port_pool; /**< Use separate buffer pools per port; disabled */
 			  /**< by default */
 
@@ -678,6 +679,7 @@  static const char short_options[] =
 #define CMD_LINE_OPT_MAX_PKT_LEN "max-pkt-len"
 #define CMD_LINE_OPT_HASH_ENTRY_NUM "hash-entry-num"
 #define CMD_LINE_OPT_PARSE_PTYPE "parse-ptype"
+#define CMD_LINE_OPT_DISABLE_RSS "disable-rss"
 #define CMD_LINE_OPT_PER_PORT_POOL "per-port-pool"
 #define CMD_LINE_OPT_MODE "mode"
 #define CMD_LINE_OPT_EVENTQ_SYNC "eventq-sched"
@@ -705,6 +707,7 @@  enum {
 	CMD_LINE_OPT_MAX_PKT_LEN_NUM,
 	CMD_LINE_OPT_HASH_ENTRY_NUM_NUM,
 	CMD_LINE_OPT_PARSE_PTYPE_NUM,
+	CMD_LINE_OPT_DISABLE_RSS_NUM,
 	CMD_LINE_OPT_RULE_IPV4_NUM,
 	CMD_LINE_OPT_RULE_IPV6_NUM,
 	CMD_LINE_OPT_ALG_NUM,
@@ -728,6 +731,7 @@  static const struct option lgopts[] = {
 	{CMD_LINE_OPT_MAX_PKT_LEN, 1, 0, CMD_LINE_OPT_MAX_PKT_LEN_NUM},
 	{CMD_LINE_OPT_HASH_ENTRY_NUM, 1, 0, CMD_LINE_OPT_HASH_ENTRY_NUM_NUM},
 	{CMD_LINE_OPT_PARSE_PTYPE, 0, 0, CMD_LINE_OPT_PARSE_PTYPE_NUM},
+	{CMD_LINE_OPT_DISABLE_RSS, 0, 0, CMD_LINE_OPT_DISABLE_RSS_NUM},
 	{CMD_LINE_OPT_PER_PORT_POOL, 0, 0, CMD_LINE_OPT_PARSE_PER_PORT_POOL},
 	{CMD_LINE_OPT_MODE, 1, 0, CMD_LINE_OPT_MODE_NUM},
 	{CMD_LINE_OPT_EVENTQ_SYNC, 1, 0, CMD_LINE_OPT_EVENTQ_SYNC_NUM},
@@ -853,6 +857,11 @@  parse_args(int argc, char **argv)
 			parse_ptype = 1;
 			break;
 
+		case CMD_LINE_OPT_DISABLE_RSS_NUM:
+			printf("Disable RX RSS\n");
+			disable_rss = 1;
+			break;
+
 		case CMD_LINE_OPT_PARSE_PER_PORT_POOL:
 			printf("per port buffer pool is enabled\n");
 			per_port_pool = 1;
@@ -1257,7 +1266,7 @@  l3fwd_poll_resource_setup(void)
 		local_port_conf.rx_adv_conf.rss_conf.rss_hf &=
 			dev_info.flow_type_rss_offloads;
 
-		if (dev_info.max_rx_queues == 1)
+		if (disable_rss == 1 || dev_info.max_rx_queues == 1)
 			local_port_conf.rxmode.mq_mode = RTE_ETH_MQ_RX_NONE;
 
 		if (local_port_conf.rx_adv_conf.rss_conf.rss_hf !=