[RFC] ethdev: sharing indirect actions between ports

  The RTE Flow API implements the concept of shared objects,
known as indirect actions (RTE_FLOW_ACTION_TYPE_INDIRECT).
An application can create the indirect action of desired
type and configuration with rte_flow_action_handle_create
call and then specify the obtained action handle in multiple
flows.

The initial concept supposes the action handle has strict
attachment to the port it was created on and to be used
exclusively in the flows being installed on the port.

Nowadays the multipath network topologies are quite common,
packets belonging to the same connection might arrive and
be sent over multiple ports, and there is the raising demand
to handle these "spread" connections. To fulfil this demand
it is proposed to extend indirect action sharing across the
multiple ports. This kind of sharing would be extremely useful
for the meters and counters, allowing to manage the single
connection over the multiple ports.

This cross-port object sharing is hard to implement in
generic way merely with software on the upper layers, but
can be provided by the driver over the single hardware
instance, where  multiple ports reside on the same physical
NIC and share the same hardware context.

To allow this action sharing application should specify
the "host port" during flow configuring to claim the intention
to share the indirect actions. All indirect actions reside within
"host port" context and can be shared in flows being installed
on the host port and on all the ports referencing this one.

If sharing between host and port being configured is not supported
the configuration should be rejected with error. There might be
multiple independent (mutual exclusive) sharing domains with
dedicated host and referencing ports.

To manage the shared indirect action any port from sharing domain
can be specified. To share or not the created action is up to
application, no API change is needed.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 lib/ethdev/rte_flow.c |  6 ++++++
 lib/ethdev/rte_flow.h | 11 +++++++++++
 2 files changed, 17 insertions(+)
  

Hi Slava,

Looks good to me,
Best,
Ori

> -----Original Message-----
> From: Slava Ovsiienko <viacheslavo@nvidia.com>
> Sent: Wednesday, 28 December 2022 18:55
> 
> The RTE Flow API implements the concept of shared objects,
> known as indirect actions (RTE_FLOW_ACTION_TYPE_INDIRECT).
> An application can create the indirect action of desired
> type and configuration with rte_flow_action_handle_create
> call and then specify the obtained action handle in multiple
> flows.
> 
> The initial concept supposes the action handle has strict
> attachment to the port it was created on and to be used
> exclusively in the flows being installed on the port.
> 
> Nowadays the multipath network topologies are quite common,
> packets belonging to the same connection might arrive and
> be sent over multiple ports, and there is the raising demand
> to handle these "spread" connections. To fulfil this demand
> it is proposed to extend indirect action sharing across the
> multiple ports. This kind of sharing would be extremely useful
> for the meters and counters, allowing to manage the single
> connection over the multiple ports.
> 
> This cross-port object sharing is hard to implement in
> generic way merely with software on the upper layers, but
> can be provided by the driver over the single hardware
> instance, where  multiple ports reside on the same physical
> NIC and share the same hardware context.
> 
> To allow this action sharing application should specify
> the "host port" during flow configuring to claim the intention
> to share the indirect actions. All indirect actions reside within
> "host port" context and can be shared in flows being installed
> on the host port and on all the ports referencing this one.
> 
> If sharing between host and port being configured is not supported
> the configuration should be rejected with error. There might be
> multiple independent (mutual exclusive) sharing domains with
> dedicated host and referencing ports.
> 
> To manage the shared indirect action any port from sharing domain
> can be specified. To share or not the created action is up to
> application, no API change is needed.
> 
> Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> ---
>  lib/ethdev/rte_flow.c |  6 ++++++
>  lib/ethdev/rte_flow.h | 11 +++++++++++
>  2 files changed, 17 insertions(+)
> 
> diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
> index 7d0c24366c..692d37925a 100644
> --- a/lib/ethdev/rte_flow.c
> +++ b/lib/ethdev/rte_flow.c
> @@ -1476,6 +1476,12 @@ rte_flow_configure(uint16_t port_id,
>  		RTE_FLOW_LOG(ERR, "Port %"PRIu16" queue info is
> NULL.\n", port_id);
>  		return -EINVAL;
>  	}
> +	if ((port_attr->flags & RTE_FLOW_PORT_FLAG_SHARE_INDIRECT) &&
> +	     !rte_eth_dev_is_valid_port(port_attr->host_port_id)) {
> +		return rte_flow_error_set(error, ENODEV,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> +					  NULL, rte_strerror(ENODEV));
> +	}
>  	if (likely(!!ops->configure)) {
>  		ret = ops->configure(dev, port_attr, nb_queue, queue_attr,
> error);
>  		if (ret == 0)
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index b60987db4b..c784f4ec30 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -4903,6 +4903,13 @@ rte_flow_info_get(uint16_t port_id,
>  		  struct rte_flow_queue_info *queue_info,
>  		  struct rte_flow_error *error);
> 
> +/**
> + * Indicate all steering objects should be created on contexts
> + * of the host port, providing indirect object sharing beetween
> + * ports.
> + */
> +#define RTE_FLOW_PORT_FLAG_SHARE_INDIRECT RTE_BIT32(0)
> +
>  /**
>   * @warning
>   * @b EXPERIMENTAL: this API may change without prior notice.
> @@ -4932,6 +4939,10 @@ struct rte_flow_port_attr {
>  	 * @see RTE_FLOW_ACTION_TYPE_CONNTRACK
>  	 */
>  	uint32_t nb_conn_tracks;
> +	/**
> +	 * Port to base shared objects on.
> +	 */
> +	uint16_t host_port_id;
>  	/**
>  	 * Port flags (RTE_FLOW_PORT_FLAG_*).
>  	 */
> --
> 2.18.1
  

28/12/2022 17:54, Viacheslav Ovsiienko:
> The RTE Flow API implements the concept of shared objects,
> known as indirect actions (RTE_FLOW_ACTION_TYPE_INDIRECT).
> An application can create the indirect action of desired
> type and configuration with rte_flow_action_handle_create
> call and then specify the obtained action handle in multiple
> flows.
> 
> The initial concept supposes the action handle has strict
> attachment to the port it was created on and to be used
> exclusively in the flows being installed on the port.
> 
> Nowadays the multipath network topologies are quite common,
> packets belonging to the same connection might arrive and
> be sent over multiple ports, and there is the raising demand
> to handle these "spread" connections. To fulfil this demand
> it is proposed to extend indirect action sharing across the
> multiple ports. This kind of sharing would be extremely useful
> for the meters and counters, allowing to manage the single
> connection over the multiple ports.
> 
> This cross-port object sharing is hard to implement in
> generic way merely with software on the upper layers, but
> can be provided by the driver over the single hardware
> instance, where  multiple ports reside on the same physical
> NIC and share the same hardware context.
> 
> To allow this action sharing application should specify
> the "host port" during flow configuring to claim the intention
> to share the indirect actions. All indirect actions reside within
> "host port" context and can be shared in flows being installed

I don't like the word "host" because it may refer to the host CPU.
Also if I understand well, the application must choose one port
between all ports of the NIC and keep using the same.
I guess we don't want to create a NIC id.
So I would suggest to rename to nic_ref_port or something like that.

> on the host port and on all the ports referencing this one.

Not "all the ports" but only on sibling ports (ports of the same NIC).

> If sharing between host and port being configured is not supported
> the configuration should be rejected with error. There might be
> multiple independent (mutual exclusive) sharing domains with
> dedicated host and referencing ports.
> 
> To manage the shared indirect action any port from sharing domain
> can be specified. To share or not the created action is up to
> application, no API change is needed.
[...]
> +	/**
> +	 * Port to base shared objects on.
> +	 */
> +	uint16_t host_port_id;

Is it a DPDK ethdev port ID?

You should add this feature to the release notes and in rte_flow guide.

PS: please Cc ethdev maintainers.
  

Hi

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Wednesday, 18 January 2023 14:07
> 
> 28/12/2022 17:54, Viacheslav Ovsiienko:
> > The RTE Flow API implements the concept of shared objects,
> > known as indirect actions (RTE_FLOW_ACTION_TYPE_INDIRECT).
> > An application can create the indirect action of desired
> > type and configuration with rte_flow_action_handle_create
> > call and then specify the obtained action handle in multiple
> > flows.
> >
> > The initial concept supposes the action handle has strict
> > attachment to the port it was created on and to be used
> > exclusively in the flows being installed on the port.
> >
> > Nowadays the multipath network topologies are quite common,
> > packets belonging to the same connection might arrive and
> > be sent over multiple ports, and there is the raising demand
> > to handle these "spread" connections. To fulfil this demand
> > it is proposed to extend indirect action sharing across the
> > multiple ports. This kind of sharing would be extremely useful
> > for the meters and counters, allowing to manage the single
> > connection over the multiple ports.
> >
> > This cross-port object sharing is hard to implement in
> > generic way merely with software on the upper layers, but
> > can be provided by the driver over the single hardware
> > instance, where  multiple ports reside on the same physical
> > NIC and share the same hardware context.
> >
> > To allow this action sharing application should specify
> > the "host port" during flow configuring to claim the intention
> > to share the indirect actions. All indirect actions reside within
> > "host port" context and can be shared in flows being installed
> 
> I don't like the word "host" because it may refer to the host CPU.
> Also if I understand well, the application must choose one port
> between all ports of the NIC and keep using the same.
> I guess we don't want to create a NIC id.
> So I would suggest to rename to nic_ref_port or something like that.
> 

I think that host is the correct word since this port hosts all resources for other
ports. (this is also why the host is used in case of CPU 😊)
I don't think it is correct to use bad wording due to the fact that some one else
also uses this word. 
in rte_flow we never talk about host CPU so I don't think this is confusing.

> > on the host port and on all the ports referencing this one.
> 
> Not "all the ports" but only on sibling ports (ports of the same NIC).
> 
> > If sharing between host and port being configured is not supported
> > the configuration should be rejected with error. There might be
> > multiple independent (mutual exclusive) sharing domains with
> > dedicated host and referencing ports.
> >
> > To manage the shared indirect action any port from sharing domain
> > can be specified. To share or not the created action is up to
> > application, no API change is needed.
> [...]
> > +	/**
> > +	 * Port to base shared objects on.
> > +	 */
> > +	uint16_t host_port_id;
> 
> Is it a DPDK ethdev port ID?
> 
> You should add this feature to the release notes and in rte_flow guide.
> 
> PS: please Cc ethdev maintainers.
>
  

18/01/2023 16:17, Ori Kam:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 28/12/2022 17:54, Viacheslav Ovsiienko:
> > > The RTE Flow API implements the concept of shared objects,
> > > known as indirect actions (RTE_FLOW_ACTION_TYPE_INDIRECT).
> > > An application can create the indirect action of desired
> > > type and configuration with rte_flow_action_handle_create
> > > call and then specify the obtained action handle in multiple
> > > flows.
> > >
> > > The initial concept supposes the action handle has strict
> > > attachment to the port it was created on and to be used
> > > exclusively in the flows being installed on the port.
> > >
> > > Nowadays the multipath network topologies are quite common,
> > > packets belonging to the same connection might arrive and
> > > be sent over multiple ports, and there is the raising demand
> > > to handle these "spread" connections. To fulfil this demand
> > > it is proposed to extend indirect action sharing across the
> > > multiple ports. This kind of sharing would be extremely useful
> > > for the meters and counters, allowing to manage the single
> > > connection over the multiple ports.
> > >
> > > This cross-port object sharing is hard to implement in
> > > generic way merely with software on the upper layers, but
> > > can be provided by the driver over the single hardware
> > > instance, where  multiple ports reside on the same physical
> > > NIC and share the same hardware context.
> > >
> > > To allow this action sharing application should specify
> > > the "host port" during flow configuring to claim the intention
> > > to share the indirect actions. All indirect actions reside within
> > > "host port" context and can be shared in flows being installed
> > 
> > I don't like the word "host" because it may refer to the host CPU.
> > Also if I understand well, the application must choose one port
> > between all ports of the NIC and keep using the same.
> > I guess we don't want to create a NIC id.
> > So I would suggest to rename to nic_ref_port or something like that.
> > 
> 
> I think that host is the correct word since this port hosts all resources for other
> ports. (this is also why the host is used in case of CPU 😊)
> I don't think it is correct to use bad wording due to the fact that some one else
> also uses this word.
> in rte_flow we never talk about host CPU so I don't think this is confusing.

The confusion is that we can think of a port on the host.
  

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Wednesday, January 18, 2023 6:22 PM
> To: Slava Ovsiienko <viacheslavo@nvidia.com>; Ori Kam
> <orika@nvidia.com>
> Cc: dev@dpdk.org; Matan Azrad <matan@nvidia.com>; Raslan Darawsheh
> <rasland@nvidia.com>; andrew.rybchenko@oktetlabs.ru;
> ivan.malov@oktetlabs.ru; ferruh.yigit@amd.com
> Subject: Re: [RFC] ethdev: sharing indirect actions between ports
> 
> 18/01/2023 16:17, Ori Kam:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 28/12/2022 17:54, Viacheslav Ovsiienko:
> > > > The RTE Flow API implements the concept of shared objects, known
> > > > as indirect actions (RTE_FLOW_ACTION_TYPE_INDIRECT).
> > > > An application can create the indirect action of desired type and
> > > > configuration with rte_flow_action_handle_create call and then
> > > > specify the obtained action handle in multiple flows.
> > > >
> > > > The initial concept supposes the action handle has strict
> > > > attachment to the port it was created on and to be used
> > > > exclusively in the flows being installed on the port.
> > > >
> > > > Nowadays the multipath network topologies are quite common,
> > > > packets belonging to the same connection might arrive and be sent
> > > > over multiple ports, and there is the raising demand to handle
> > > > these "spread" connections. To fulfil this demand it is proposed
> > > > to extend indirect action sharing across the multiple ports. This
> > > > kind of sharing would be extremely useful for the meters and
> > > > counters, allowing to manage the single connection over the
> > > > multiple ports.
> > > >
> > > > This cross-port object sharing is hard to implement in generic way
> > > > merely with software on the upper layers, but can be provided by
> > > > the driver over the single hardware instance, where  multiple
> > > > ports reside on the same physical NIC and share the same hardware
> > > > context.
> > > >
> > > > To allow this action sharing application should specify the "host
> > > > port" during flow configuring to claim the intention to share the
> > > > indirect actions. All indirect actions reside within "host port"
> > > > context and can be shared in flows being installed
> > >
> > > I don't like the word "host" because it may refer to the host CPU.
> > > Also if I understand well, the application must choose one port
> > > between all ports of the NIC and keep using the same.
> > > I guess we don't want to create a NIC id.
> > > So I would suggest to rename to nic_ref_port or something like that.
> > >
> >
> > I think that host is the correct word since this port hosts all
> > resources for other ports. (this is also why the host is used in case
> > of CPU 😊)
> > I don't think it is correct to use bad wording due to the fact that
> > some one else also uses this word.
> > in rte_flow we never talk about host CPU so I don't think this is confusing.
> 
> The confusion is that we can think of a port on the host.

In my humble opinion, "_port_id" suffix explicitly specifies what field is and does not leave
too much space for confusion.

"root_port_id"? "base_port_id"?  "container_port_id" ? "mgmnt_port_id" ?
 Looks worse as for me and does not reflect the exact meaning.
As Ori mentioned this is DPDK port ID that embraces all the shared actions.
It plays a host role for them.

With best regards, Slava

> 
>
  

On 1/18/23 19:37, Slava Ovsiienko wrote:
> 
> 
>> -----Original Message-----
>> From: Thomas Monjalon <thomas@monjalon.net>
>> Sent: Wednesday, January 18, 2023 6:22 PM
>> To: Slava Ovsiienko <viacheslavo@nvidia.com>; Ori Kam
>> <orika@nvidia.com>
>> Cc: dev@dpdk.org; Matan Azrad <matan@nvidia.com>; Raslan Darawsheh
>> <rasland@nvidia.com>; andrew.rybchenko@oktetlabs.ru;
>> ivan.malov@oktetlabs.ru; ferruh.yigit@amd.com
>> Subject: Re: [RFC] ethdev: sharing indirect actions between ports
>>
>> 18/01/2023 16:17, Ori Kam:
>>> From: Thomas Monjalon <thomas@monjalon.net>
>>>> 28/12/2022 17:54, Viacheslav Ovsiienko:
>>>>> The RTE Flow API implements the concept of shared objects, known
>>>>> as indirect actions (RTE_FLOW_ACTION_TYPE_INDIRECT).
>>>>> An application can create the indirect action of desired type and
>>>>> configuration with rte_flow_action_handle_create call and then
>>>>> specify the obtained action handle in multiple flows.
>>>>>
>>>>> The initial concept supposes the action handle has strict
>>>>> attachment to the port it was created on and to be used
>>>>> exclusively in the flows being installed on the port.
>>>>>
>>>>> Nowadays the multipath network topologies are quite common,
>>>>> packets belonging to the same connection might arrive and be sent
>>>>> over multiple ports, and there is the raising demand to handle
>>>>> these "spread" connections. To fulfil this demand it is proposed
>>>>> to extend indirect action sharing across the multiple ports. This
>>>>> kind of sharing would be extremely useful for the meters and
>>>>> counters, allowing to manage the single connection over the
>>>>> multiple ports.
>>>>>
>>>>> This cross-port object sharing is hard to implement in generic way
>>>>> merely with software on the upper layers, but can be provided by
>>>>> the driver over the single hardware instance, where  multiple
>>>>> ports reside on the same physical NIC and share the same hardware
>>>>> context.
>>>>>
>>>>> To allow this action sharing application should specify the "host
>>>>> port" during flow configuring to claim the intention to share the
>>>>> indirect actions. All indirect actions reside within "host port"
>>>>> context and can be shared in flows being installed
>>>>
>>>> I don't like the word "host" because it may refer to the host CPU.
>>>> Also if I understand well, the application must choose one port
>>>> between all ports of the NIC and keep using the same.
>>>> I guess we don't want to create a NIC id.
>>>> So I would suggest to rename to nic_ref_port or something like that.
>>>>
>>>
>>> I think that host is the correct word since this port hosts all
>>> resources for other ports. (this is also why the host is used in case
>>> of CPU 😊)
>>> I don't think it is correct to use bad wording due to the fact that
>>> some one else also uses this word.
>>> in rte_flow we never talk about host CPU so I don't think this is confusing.
>>
>> The confusion is that we can think of a port on the host.
> 
> In my humble opinion, "_port_id" suffix explicitly specifies what field is and does not leave
> too much space for confusion.
> 
> "root_port_id"? "base_port_id"?  "container_port_id" ? "mgmnt_port_id" ?
>   Looks worse as for me and does not reflect the exact meaning.
> As Ori mentioned this is DPDK port ID that embraces all the shared actions.
> It plays a host role for them.

Maybe 'owner_port_id' or 'rsrc_port_id' ?
  

> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Friday, 20 January 2023 14:23
> 
> On 1/18/23 19:37, Slava Ovsiienko wrote:
> >
> >
> >> -----Original Message-----
> >> From: Thomas Monjalon <thomas@monjalon.net>
> >> Sent: Wednesday, January 18, 2023 6:22 PM
> >> To: Slava Ovsiienko <viacheslavo@nvidia.com>; Ori Kam
> >> <orika@nvidia.com>
> >> Cc: dev@dpdk.org; Matan Azrad <matan@nvidia.com>; Raslan
> Darawsheh
> >> <rasland@nvidia.com>; andrew.rybchenko@oktetlabs.ru;
> >> ivan.malov@oktetlabs.ru; ferruh.yigit@amd.com
> >> Subject: Re: [RFC] ethdev: sharing indirect actions between ports
> >>
> >> 18/01/2023 16:17, Ori Kam:
> >>> From: Thomas Monjalon <thomas@monjalon.net>
> >>>> 28/12/2022 17:54, Viacheslav Ovsiienko:
> >>>>> The RTE Flow API implements the concept of shared objects, known
> >>>>> as indirect actions (RTE_FLOW_ACTION_TYPE_INDIRECT).
> >>>>> An application can create the indirect action of desired type and
> >>>>> configuration with rte_flow_action_handle_create call and then
> >>>>> specify the obtained action handle in multiple flows.
> >>>>>
> >>>>> The initial concept supposes the action handle has strict
> >>>>> attachment to the port it was created on and to be used
> >>>>> exclusively in the flows being installed on the port.
> >>>>>
> >>>>> Nowadays the multipath network topologies are quite common,
> >>>>> packets belonging to the same connection might arrive and be sent
> >>>>> over multiple ports, and there is the raising demand to handle
> >>>>> these "spread" connections. To fulfil this demand it is proposed
> >>>>> to extend indirect action sharing across the multiple ports. This
> >>>>> kind of sharing would be extremely useful for the meters and
> >>>>> counters, allowing to manage the single connection over the
> >>>>> multiple ports.
> >>>>>
> >>>>> This cross-port object sharing is hard to implement in generic way
> >>>>> merely with software on the upper layers, but can be provided by
> >>>>> the driver over the single hardware instance, where  multiple
> >>>>> ports reside on the same physical NIC and share the same hardware
> >>>>> context.
> >>>>>
> >>>>> To allow this action sharing application should specify the "host
> >>>>> port" during flow configuring to claim the intention to share the
> >>>>> indirect actions. All indirect actions reside within "host port"
> >>>>> context and can be shared in flows being installed
> >>>>
> >>>> I don't like the word "host" because it may refer to the host CPU.
> >>>> Also if I understand well, the application must choose one port
> >>>> between all ports of the NIC and keep using the same.
> >>>> I guess we don't want to create a NIC id.
> >>>> So I would suggest to rename to nic_ref_port or something like that.
> >>>>
> >>>
> >>> I think that host is the correct word since this port hosts all
> >>> resources for other ports. (this is also why the host is used in case
> >>> of CPU 😊)
> >>> I don't think it is correct to use bad wording due to the fact that
> >>> some one else also uses this word.
> >>> in rte_flow we never talk about host CPU so I don't think this is
> confusing.
> >>
> >> The confusion is that we can think of a port on the host.
> >
> > In my humble opinion, "_port_id" suffix explicitly specifies what field is and
> does not leave
> > too much space for confusion.
> >
> > "root_port_id"? "base_port_id"?  "container_port_id" ? "mgmnt_port_id"
> ?
> >   Looks worse as for me and does not reflect the exact meaning.
> > As Ori mentioned this is DPDK port ID that embraces all the shared actions.
> > It plays a host role for them.
> 
> Maybe 'owner_port_id' or 'rsrc_port_id' ?
> 
Rsrc?
Owner_port looks O.K but I'm not sure what is the issue with the original suggestion.

Best,
Ori