[v7,2/2] ethdev: move egress metadata to dynamic field

Message ID 1572527121-13133-3-git-send-email-viacheslavo@mellanox.com (mailing list archive)
State Superseded, archived
Delegated to: Ferruh Yigit
Headers
Series extend flow metadata feature |

Checks

Context Check Description
ci/Intel-compilation fail apply issues
ci/checkpatch success coding style OK

Commit Message

Slava Ovsiienko Oct. 31, 2019, 1:05 p.m. UTC
  The dynamic mbuf fields were introduced by [1]. The egress metadata is
good candidate to be moved from statically allocated field tx_metadata to
dynamic one. Because mbufs are used in half-duplex fashion only, it is
safe to share this dynamic field with ingress metadata.

The shared dynamic field contains either egress (if application going to
transmit mbuf with tx_burst) or ingress (if mbuf is received with rx_burst)
metadata and can be accessed by RTE_FLOW_DYNF_METADATA() macro or with
rte_flow_dynf_metadata_set() and rte_flow_dynf_metadata_get() helper
routines. PKT_TX_DYNF_METADATA/PKT_RX_DYNF_METADATA flag will be set
along with the data.

The mbuf dynamic field must be registered by calling
rte_flow_dynf_metadata_register() prior accessing the data.

The availability of dynamic mbuf metadata field can be checked with
rte_flow_dynf_metadata_avail() routine.

DEV_TX_OFFLOAD_MATCH_METADATA offload and configuration flag is removed.
The metadata support in PMDs is engaged on dynamic field registration.

Metadata feature is getting complex. We might have some set of actions
and items that might be supported by PMDs in multiple combinations,
the supported values and masks are the subjects to query by perfroming
trials (with rte_flow_validate).

[1] http://patches.dpdk.org/patch/62040/

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
---

v7: - updates release notes
v6: - http://patches.dpdk.org/patch/62244/

 app/test-pmd/cmdline.c                 |  3 ++-
 app/test-pmd/testpmd.c                 |  4 ----
 app/test-pmd/testpmd.h                 |  2 +-
 app/test-pmd/util.c                    | 15 +++++++++------
 app/test/test_mbuf.c                   |  1 -
 doc/guides/prog_guide/rte_flow.rst     |  6 +++---
 doc/guides/rel_notes/release_19_11.rst |  5 +++++
 drivers/net/mlx5/mlx5_flow_dv.c        | 19 ++++++-------------
 drivers/net/mlx5/mlx5_rxtx.c           | 22 +++++++++++-----------
 drivers/net/mlx5/mlx5_rxtx_vec.h       |  6 ------
 drivers/net/mlx5/mlx5_txq.c            |  4 ----
 lib/librte_ethdev/rte_ethdev.c         |  1 -
 lib/librte_ethdev/rte_ethdev.h         |  5 -----
 lib/librte_ethdev/rte_flow.h           | 19 ++++++++++---------
 lib/librte_mbuf/rte_mbuf.c             |  2 --
 lib/librte_mbuf/rte_mbuf_core.h        | 19 +------------------
 16 files changed, 48 insertions(+), 85 deletions(-)
  

Comments

Ori Kam Oct. 31, 2019, 1:33 p.m. UTC | #1
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Viacheslav Ovsiienko
> Sent: Thursday, October 31, 2019 3:05 PM
> To: dev@dpdk.org
> Cc: Matan Azrad <matan@mellanox.com>; Raslan Darawsheh
> <rasland@mellanox.com>; Thomas Monjalon <thomas@monjalon.net>;
> olivier.matz@6wind.com; arybchenko@solarflare.com; Ori Kam
> <orika@mellanox.com>
> Subject: [dpdk-dev] [PATCH v7 2/2] ethdev: move egress metadata to dynamic
> field
> 
> The dynamic mbuf fields were introduced by [1]. The egress metadata is
> good candidate to be moved from statically allocated field tx_metadata to
> dynamic one. Because mbufs are used in half-duplex fashion only, it is
> safe to share this dynamic field with ingress metadata.
> 
> The shared dynamic field contains either egress (if application going to
> transmit mbuf with tx_burst) or ingress (if mbuf is received with rx_burst)
> metadata and can be accessed by RTE_FLOW_DYNF_METADATA() macro or
> with
> rte_flow_dynf_metadata_set() and rte_flow_dynf_metadata_get() helper
> routines. PKT_TX_DYNF_METADATA/PKT_RX_DYNF_METADATA flag will be set
> along with the data.
> 
> The mbuf dynamic field must be registered by calling
> rte_flow_dynf_metadata_register() prior accessing the data.
> 
> The availability of dynamic mbuf metadata field can be checked with
> rte_flow_dynf_metadata_avail() routine.
> 
> DEV_TX_OFFLOAD_MATCH_METADATA offload and configuration flag is
> removed.
> The metadata support in PMDs is engaged on dynamic field registration.
> 
> Metadata feature is getting complex. We might have some set of actions
> and items that might be supported by PMDs in multiple combinations,
> the supported values and masks are the subjects to query by perfroming
> trials (with rte_flow_validate).
> 
> [1]
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpatches.dp
> dk.org%2Fpatch%2F62040%2F&amp;data=02%7C01%7Corika%40mellanox.com
> %7C59c1c0e65ad7436cb8f708d75e031341%7Ca652971c7d2e4d9ba6a4d14925
> 6f461b%7C0%7C0%7C637081239629362825&amp;sdata=A%2Fe1hjy02uGsgK5A
> ouK6pF%2BRdju5lcANP3ye3xx6qKs%3D&amp;reserved=0
> 
> Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> ---
> 
> v7: - updates release notes
> v6: -
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpatches.dp
> dk.org%2Fpatch%2F62244%2F&amp;data=02%7C01%7Corika%40mellanox.com
> %7C59c1c0e65ad7436cb8f708d75e031341%7Ca652971c7d2e4d9ba6a4d14925
> 6f461b%7C0%7C0%7C637081239629362825&amp;sdata=72hW3BfXHQ0KGzNq
> o3M1TR%2BI4oQdJ5sU4wYzK74rF%2FA%3D&amp;reserved=0
> 
>  app/test-pmd/cmdline.c                 |  3 ++-
>  app/test-pmd/testpmd.c                 |  4 ----
>  app/test-pmd/testpmd.h                 |  2 +-
>  app/test-pmd/util.c                    | 15 +++++++++------
>  app/test/test_mbuf.c                   |  1 -
>  doc/guides/prog_guide/rte_flow.rst     |  6 +++---
>  doc/guides/rel_notes/release_19_11.rst |  5 +++++
>  drivers/net/mlx5/mlx5_flow_dv.c        | 19 ++++++-------------
>  drivers/net/mlx5/mlx5_rxtx.c           | 22 +++++++++++-----------
>  drivers/net/mlx5/mlx5_rxtx_vec.h       |  6 ------
>  drivers/net/mlx5/mlx5_txq.c            |  4 ----
>  lib/librte_ethdev/rte_ethdev.c         |  1 -
>  lib/librte_ethdev/rte_ethdev.h         |  5 -----
>  lib/librte_ethdev/rte_flow.h           | 19 ++++++++++---------
>  lib/librte_mbuf/rte_mbuf.c             |  2 --
>  lib/librte_mbuf/rte_mbuf_core.h        | 19 +------------------
>  16 files changed, 48 insertions(+), 85 deletions(-)
> 

Acked-by: Ori Kam <orika@mellanox.com>

Thanks,
Ori
  
Olivier Matz Oct. 31, 2019, 3:51 p.m. UTC | #2
Hi,

On Thu, Oct 31, 2019 at 01:05:21PM +0000, Viacheslav Ovsiienko wrote:
> The dynamic mbuf fields were introduced by [1]. The egress metadata is
> good candidate to be moved from statically allocated field tx_metadata to
> dynamic one. Because mbufs are used in half-duplex fashion only, it is
> safe to share this dynamic field with ingress metadata.
> 
> The shared dynamic field contains either egress (if application going to
> transmit mbuf with tx_burst) or ingress (if mbuf is received with rx_burst)
> metadata and can be accessed by RTE_FLOW_DYNF_METADATA() macro or with
> rte_flow_dynf_metadata_set() and rte_flow_dynf_metadata_get() helper
> routines. PKT_TX_DYNF_METADATA/PKT_RX_DYNF_METADATA flag will be set
> along with the data.
> 
> The mbuf dynamic field must be registered by calling
> rte_flow_dynf_metadata_register() prior accessing the data.
> 
> The availability of dynamic mbuf metadata field can be checked with
> rte_flow_dynf_metadata_avail() routine.
> 
> DEV_TX_OFFLOAD_MATCH_METADATA offload and configuration flag is removed.
> The metadata support in PMDs is engaged on dynamic field registration.
> 
> Metadata feature is getting complex. We might have some set of actions
> and items that might be supported by PMDs in multiple combinations,
> the supported values and masks are the subjects to query by perfroming
> trials (with rte_flow_validate).
> 
> [1] http://patches.dpdk.org/patch/62040/
> 
> Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> ---
> 
> v7: - updates release notes
> v6: - http://patches.dpdk.org/patch/62244/
> 
>  app/test-pmd/cmdline.c                 |  3 ++-
>  app/test-pmd/testpmd.c                 |  4 ----
>  app/test-pmd/testpmd.h                 |  2 +-
>  app/test-pmd/util.c                    | 15 +++++++++------
>  app/test/test_mbuf.c                   |  1 -
>  doc/guides/prog_guide/rte_flow.rst     |  6 +++---
>  doc/guides/rel_notes/release_19_11.rst |  5 +++++
>  drivers/net/mlx5/mlx5_flow_dv.c        | 19 ++++++-------------
>  drivers/net/mlx5/mlx5_rxtx.c           | 22 +++++++++++-----------
>  drivers/net/mlx5/mlx5_rxtx_vec.h       |  6 ------
>  drivers/net/mlx5/mlx5_txq.c            |  4 ----
>  lib/librte_ethdev/rte_ethdev.c         |  1 -
>  lib/librte_ethdev/rte_ethdev.h         |  5 -----
>  lib/librte_ethdev/rte_flow.h           | 19 ++++++++++---------
>  lib/librte_mbuf/rte_mbuf.c             |  2 --
>  lib/librte_mbuf/rte_mbuf_core.h        | 19 +------------------
>  16 files changed, 48 insertions(+), 85 deletions(-)
> 
> diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
> index 4478069..49c45a3 100644
> --- a/app/test-pmd/cmdline.c
> +++ b/app/test-pmd/cmdline.c
> @@ -18718,12 +18718,13 @@ struct cmd_config_tx_metadata_specific_result {
>  
>  	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
>  		return;
> -	ports[res->port_id].tx_metadata = rte_cpu_to_be_32(res->value);
> +	ports[res->port_id].tx_metadata = res->value;
>  	/* Add/remove callback to insert valid metadata in every Tx packet. */
>  	if (ports[res->port_id].tx_metadata)
>  		add_tx_md_callback(res->port_id);
>  	else
>  		remove_tx_md_callback(res->port_id);
> +	rte_flow_dynf_metadata_register();
>  }
>  
>  cmdline_parse_token_string_t cmd_config_tx_metadata_specific_port =
> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
> index 0fc5b45..206c12b 100644
> --- a/app/test-pmd/testpmd.c
> +++ b/app/test-pmd/testpmd.c
> @@ -1167,10 +1167,6 @@ struct extmem_param {
>  		      DEV_TX_OFFLOAD_MBUF_FAST_FREE))
>  			port->dev_conf.txmode.offloads &=
>  				~DEV_TX_OFFLOAD_MBUF_FAST_FREE;
> -		if (!(port->dev_info.tx_offload_capa &
> -			DEV_TX_OFFLOAD_MATCH_METADATA))
> -			port->dev_conf.txmode.offloads &=
> -				~DEV_TX_OFFLOAD_MATCH_METADATA;
>  		if (numa_support) {
>  			if (port_numa[pid] != NUMA_NO_CONFIG)
>  				port_per_socket[port_numa[pid]]++;
> diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
> index 8da1e8e..caabf32 100644
> --- a/app/test-pmd/testpmd.h
> +++ b/app/test-pmd/testpmd.h
> @@ -193,7 +193,7 @@ struct rte_port {
>  	struct softnic_port     softport;  /**< softnic params */
>  #endif
>  	/**< metadata value to insert in Tx packets. */
> -	rte_be32_t		tx_metadata;
> +	uint32_t		tx_metadata;
>  	const struct rte_eth_rxtx_callback *tx_set_md_cb[MAX_QUEUE_ID+1];
>  };
>  
> diff --git a/app/test-pmd/util.c b/app/test-pmd/util.c
> index 56075b3..cf41864 100644
> --- a/app/test-pmd/util.c
> +++ b/app/test-pmd/util.c
> @@ -82,8 +82,9 @@
>  			       mb->vlan_tci, mb->vlan_tci_outer);
>  		else if (ol_flags & PKT_RX_VLAN)
>  			printf(" - VLAN tci=0x%x", mb->vlan_tci);
> -		if (ol_flags & PKT_TX_METADATA)
> -			printf(" - Tx metadata: 0x%x", mb->tx_metadata);
> +		if (ol_flags & PKT_TX_DYNF_METADATA)
> +			printf(" - Tx metadata: 0x%x",
> +			       *RTE_FLOW_DYNF_METADATA(mb));
>  		if (ol_flags & PKT_RX_DYNF_METADATA)
>  			printf(" - Rx metadata: 0x%x",
>  			       *RTE_FLOW_DYNF_METADATA(mb));
> @@ -188,10 +189,12 @@
>  	 * Add metadata value to every Tx packet,
>  	 * and set ol_flags accordingly.
>  	 */
> -	for (i = 0; i < nb_pkts; i++) {
> -		pkts[i]->tx_metadata = ports[port_id].tx_metadata;
> -		pkts[i]->ol_flags |= PKT_TX_METADATA;
> -	}
> +	if (rte_flow_dynf_metadata_avail())
> +		for (i = 0; i < nb_pkts; i++) {
> +			*RTE_FLOW_DYNF_METADATA(pkts[i]) =
> +						ports[port_id].tx_metadata;
> +			pkts[i]->ol_flags |= PKT_TX_DYNF_METADATA;
> +		}
>  	return nb_pkts;
>  }
>  
> diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> index 854bc26..61ecffc 100644
> --- a/app/test/test_mbuf.c
> +++ b/app/test/test_mbuf.c
> @@ -1669,7 +1669,6 @@ struct flag_name {
>  		VAL_NAME(PKT_TX_SEC_OFFLOAD),
>  		VAL_NAME(PKT_TX_UDP_SEG),
>  		VAL_NAME(PKT_TX_OUTER_UDP_CKSUM),
> -		VAL_NAME(PKT_TX_METADATA),
>  	};
>  
>  	/* Test case to check with valid flag */
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index c943aca..630e4c0 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -664,7 +664,7 @@ Item: ``META``
>  Matches 32 bit metadata item set.
>  
>  On egress, metadata can be set either by mbuf metadata field with
> -PKT_TX_METADATA flag or ``SET_META`` action. On ingress, ``SET_META``
> +PKT_TX_DYNF_METADATA flag or ``SET_META`` action. On ingress, ``SET_META``
>  action sets metadata for a packet and the metadata will be reported via
>  ``metadata`` dynamic field of ``rte_mbuf`` with PKT_RX_DYNF_METADATA flag.
>  
> @@ -2482,8 +2482,8 @@ Action: ``SET_META``
>  
>  Set metadata. Item ``META`` matches metadata.
>  
> -Metadata set by mbuf metadata field with PKT_TX_METADATA flag on egress will be
> -overridden by this action. On ingress, the metadata will be carried by
> +Metadata set by mbuf metadata field with PKT_TX_DYNF_METADATA flag on egress
> +will be overridden by this action. On ingress, the metadata will be carried by
>  ``metadata`` dynamic field of ``rte_mbuf`` which can be accessed by
>  ``RTE_FLOW_DYNF_METADATA()``. PKT_RX_DYNF_METADATA flag will be set along
>  with the data.
> diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
> index 963c4f8..2e9a596 100644
> --- a/doc/guides/rel_notes/release_19_11.rst
> +++ b/doc/guides/rel_notes/release_19_11.rst
> @@ -357,6 +357,11 @@ API Changes
>    is the minor compatibility issue for applications in case of 32-bit values
>    supported.
>  
> +* metadata: the tx_metadata mbuf field is moved to dymanic one.
> +  PKT_TX_METADATA flag is replaced with PKT_TX_DYNF_METADATA.
> +  DEV_TX_OFFLOAD_MATCH_METADATA offload flag is removed, now metadata
> +  support in PMD is engaged on dynamic field registration.
> +
>  * sched: The pipe nodes configuration parameters such as number of pipes,
>    pipe queue sizes, pipe profiles, etc., are moved from port level structure
>    to subport level. This allows different subports of the same port to
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index d9a7fd4..f961bff 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -793,7 +793,7 @@ struct field_modify_info modify_tcp[] = {
>   *   0 on success, a negative errno value otherwise and rte_errno is set.
>   */
>  static int
> -flow_dv_validate_item_meta(struct rte_eth_dev *dev,
> +flow_dv_validate_item_meta(struct rte_eth_dev *dev __rte_unused,
>  			   const struct rte_flow_item *item,
>  			   const struct rte_flow_attr *attr,
>  			   struct rte_flow_error *error)
> @@ -801,17 +801,10 @@ struct field_modify_info modify_tcp[] = {
>  	const struct rte_flow_item_meta *spec = item->spec;
>  	const struct rte_flow_item_meta *mask = item->mask;
>  	const struct rte_flow_item_meta nic_mask = {
> -		.data = RTE_BE32(UINT32_MAX)
> +		.data = UINT32_MAX
>  	};
>  	int ret;
> -	uint64_t offloads = dev->data->dev_conf.txmode.offloads;
>  
> -	if (!(offloads & DEV_TX_OFFLOAD_MATCH_METADATA))
> -		return rte_flow_error_set(error, EPERM,
> -					  RTE_FLOW_ERROR_TYPE_ITEM,
> -					  NULL,
> -					  "match on metadata offload "
> -					  "configuration is off for this port");
>  	if (!spec)
>  		return rte_flow_error_set(error, EINVAL,
>  					  RTE_FLOW_ERROR_TYPE_ITEM_SPEC,
> @@ -4750,10 +4743,10 @@ struct field_modify_info modify_tcp[] = {
>  		meta_m = &rte_flow_item_meta_mask;
>  	meta_v = (const void *)item->spec;
>  	if (meta_v) {
> -		MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_a,
> -			 rte_be_to_cpu_32(meta_m->data));
> -		MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_a,
> -			 rte_be_to_cpu_32(meta_v->data & meta_m->data));
> +		MLX5_SET(fte_match_set_misc2, misc2_m,
> +			 metadata_reg_a, meta_m->data);
> +		MLX5_SET(fte_match_set_misc2, misc2_v,
> +			 metadata_reg_a, meta_v->data & meta_m->data);
>  	}
>  }
>  
> diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
> index f597c89..88a4378 100644
> --- a/drivers/net/mlx5/mlx5_rxtx.c
> +++ b/drivers/net/mlx5/mlx5_rxtx.c
> @@ -2281,8 +2281,8 @@ enum mlx5_txcmp_code {
>  	es->swp_offs = txq_mbuf_to_swp(loc, &es->swp_flags, olx);
>  	/* Fill metadata field if needed. */
>  	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
> -		       loc->mbuf->ol_flags & PKT_TX_METADATA ?
> -		       loc->mbuf->tx_metadata : 0 : 0;
> +		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
> +		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
>  	/* Engage VLAN tag insertion feature if requested. */
>  	if (MLX5_TXOFF_CONFIG(VLAN) &&
>  	    loc->mbuf->ol_flags & PKT_TX_VLAN_PKT) {
> @@ -2341,8 +2341,8 @@ enum mlx5_txcmp_code {
>  	es->swp_offs = txq_mbuf_to_swp(loc, &es->swp_flags, olx);
>  	/* Fill metadata field if needed. */
>  	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
> -		       loc->mbuf->ol_flags & PKT_TX_METADATA ?
> -		       loc->mbuf->tx_metadata : 0 : 0;
> +		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
> +		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
>  	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
>  				(sizeof(uint16_t) +
>  				 sizeof(rte_v128u32_t)),
> @@ -2434,8 +2434,8 @@ enum mlx5_txcmp_code {
>  	es->swp_offs = txq_mbuf_to_swp(loc, &es->swp_flags, olx);
>  	/* Fill metadata field if needed. */
>  	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
> -		       loc->mbuf->ol_flags & PKT_TX_METADATA ?
> -		       loc->mbuf->tx_metadata : 0 : 0;
> +		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
> +		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
>  	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
>  				(sizeof(uint16_t) +
>  				 sizeof(rte_v128u32_t)),
> @@ -2628,8 +2628,8 @@ enum mlx5_txcmp_code {
>  	es->swp_offs = txq_mbuf_to_swp(loc, &es->swp_flags, olx);
>  	/* Fill metadata field if needed. */
>  	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
> -		       loc->mbuf->ol_flags & PKT_TX_METADATA ?
> -		       loc->mbuf->tx_metadata : 0 : 0;
> +		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
> +		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
>  	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
>  				(sizeof(uint16_t) +
>  				 sizeof(rte_v128u32_t)),
> @@ -3700,8 +3700,8 @@ enum mlx5_txcmp_code {
>  		return false;
>  	/* Fill metadata field if needed. */
>  	if (MLX5_TXOFF_CONFIG(METADATA) &&
> -		es->metadata != (loc->mbuf->ol_flags & PKT_TX_METADATA ?
> -				 loc->mbuf->tx_metadata : 0))
> +		es->metadata != (loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
> +				 *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0))
>  		return false;
>  	/* There must be no VLAN packets in eMPW loop. */
>  	if (MLX5_TXOFF_CONFIG(VLAN))
> @@ -5149,7 +5149,7 @@ enum mlx5_txcmp_code {
>  		 */
>  		olx |= MLX5_TXOFF_CONFIG_EMPW;
>  	}
> -	if (tx_offloads & DEV_TX_OFFLOAD_MATCH_METADATA) {
> +	if (rte_flow_dynf_metadata_avail()) {
>  		/* We should support Flow metadata. */
>  		olx |= MLX5_TXOFF_CONFIG_METADATA;
>  	}
> diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.h b/drivers/net/mlx5/mlx5_rxtx_vec.h
> index b54ff72..85e0bd5 100644
> --- a/drivers/net/mlx5/mlx5_rxtx_vec.h
> +++ b/drivers/net/mlx5/mlx5_rxtx_vec.h
> @@ -19,12 +19,6 @@
>  	 DEV_TX_OFFLOAD_TCP_CKSUM | \
>  	 DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM)
>  
> -/* HW offload capabilities of vectorized Tx. */
> -#define MLX5_VEC_TX_OFFLOAD_CAP \
> -	(MLX5_VEC_TX_CKSUM_OFFLOAD_CAP | \
> -	 DEV_TX_OFFLOAD_MATCH_METADATA | \
> -	 DEV_TX_OFFLOAD_MULTI_SEGS)
> -
>  /*
>   * Compile time sanity check for vectorized functions.
>   */
> diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c
> index dfc379c..97991f0 100644
> --- a/drivers/net/mlx5/mlx5_txq.c
> +++ b/drivers/net/mlx5/mlx5_txq.c
> @@ -128,10 +128,6 @@
>  			offloads |= (DEV_TX_OFFLOAD_VXLAN_TNL_TSO |
>  				     DEV_TX_OFFLOAD_GRE_TNL_TSO);
>  	}
> -#ifdef HAVE_IBV_FLOW_DV_SUPPORT
> -	if (config->dv_flow_en)
> -		offloads |= DEV_TX_OFFLOAD_MATCH_METADATA;
> -#endif
>  	return offloads;
>  }
>  
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 68aca1f..23b751f 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -161,7 +161,6 @@ struct rte_eth_xstats_name_off {
>  	RTE_TX_OFFLOAD_BIT2STR(UDP_TNL_TSO),
>  	RTE_TX_OFFLOAD_BIT2STR(IP_TNL_TSO),
>  	RTE_TX_OFFLOAD_BIT2STR(OUTER_UDP_CKSUM),
> -	RTE_TX_OFFLOAD_BIT2STR(MATCH_METADATA),
>  };
>  
>  #undef RTE_TX_OFFLOAD_BIT2STR
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 9b69255..28e29c7 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -1145,11 +1145,6 @@ struct rte_eth_conf {
>  #define DEV_TX_OFFLOAD_IP_TNL_TSO       0x00080000
>  /** Device supports outer UDP checksum */
>  #define DEV_TX_OFFLOAD_OUTER_UDP_CKSUM  0x00100000
> -/**
> - * Device supports match on metadata Tx offload..
> - * Application must set PKT_TX_METADATA and mbuf metadata field.
> - */
> -#define DEV_TX_OFFLOAD_MATCH_METADATA   0x00200000
>  
>  #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
>  /**< Device supports Rx queue setup after device started*/
> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
> index f6e050c..51d8292 100644
> --- a/lib/librte_ethdev/rte_flow.h
> +++ b/lib/librte_ethdev/rte_flow.h
> @@ -1268,12 +1268,12 @@ struct rte_flow_item_icmp6_nd_opt_tla_eth {
>  /**
>   * RTE_FLOW_ITEM_TYPE_META
>   *
> - * Matches a specified metadata value. On egress, metadata can be set either by
> - * mbuf tx_metadata field with PKT_TX_METADATA flag or
> - * RTE_FLOW_ACTION_TYPE_SET_META. On ingress, RTE_FLOW_ACTION_TYPE_SET_META sets
> - * metadata for a packet and the metadata will be reported via mbuf metadata
> - * dynamic field with PKT_RX_DYNF_METADATA flag. The dynamic mbuf field must be
> - * registered in advance by rte_flow_dynf_metadata_register().
> + * Matches a specified metadata value. On egress, metadata can be set
> + * either by mbuf dynamic metadata field with PKT_TX_DYNF_METADATA flag or
> + * RTE_FLOW_ACTION_TYPE_SET_META. On ingress, RTE_FLOW_ACTION_TYPE_SET_META
> + * sets metadata for a packet and the metadata will be reported via mbuf
> + * metadata dynamic field with PKT_RX_DYNF_METADATA flag. The dynamic mbuf
> + * field must be registered in advance by rte_flow_dynf_metadata_register().
>   */
>  struct rte_flow_item_meta {
>  	uint32_t data;
> @@ -2450,8 +2450,8 @@ struct rte_flow_action_set_mac {
>   *
>   * RTE_FLOW_ACTION_TYPE_SET_META
>   *
> - * Set metadata. Metadata set by mbuf tx_metadata field with
> - * PKT_TX_METADATA flag on egress will be overridden by this action. On
> + * Set metadata. Metadata set by mbuf metadata dynamic field with
> + * PKT_TX_DYNF_DATA flag on egress will be overridden by this action. On
>   * ingress, the metadata will be carried by mbuf metadata dynamic field
>   * with PKT_RX_DYNF_METADATA flag if set.  The dynamic mbuf field must be
>   * registered in advance by rte_flow_dynf_metadata_register().
> @@ -2478,8 +2478,9 @@ struct rte_flow_action_set_meta {
>  #define RTE_FLOW_DYNF_METADATA(m) \
>  	RTE_MBUF_DYNFIELD((m), rte_flow_dynf_metadata_offs, uint32_t *)
>  
> -/* Mbuf dynamic flag for metadata. */
> +/* Mbuf dynamic flags for metadata. */
>  #define PKT_RX_DYNF_METADATA (rte_flow_dynf_metadata_mask)
> +#define PKT_TX_DYNF_METADATA (rte_flow_dynf_metadata_mask)

Should we have 2 defines pointing to the same mask? Shall we use
PKT_DYNF_METADATA for both?


>  
>  __rte_experimental
>  static inline uint32_t
> diff --git a/lib/librte_mbuf/rte_mbuf.c b/lib/librte_mbuf/rte_mbuf.c
> index 8c51dc1..35df1c4 100644
> --- a/lib/librte_mbuf/rte_mbuf.c
> +++ b/lib/librte_mbuf/rte_mbuf.c
> @@ -670,7 +670,6 @@ const char *rte_get_tx_ol_flag_name(uint64_t mask)
>  	case PKT_TX_SEC_OFFLOAD: return "PKT_TX_SEC_OFFLOAD";
>  	case PKT_TX_UDP_SEG: return "PKT_TX_UDP_SEG";
>  	case PKT_TX_OUTER_UDP_CKSUM: return "PKT_TX_OUTER_UDP_CKSUM";
> -	case PKT_TX_METADATA: return "PKT_TX_METADATA";
>  	default: return NULL;
>  	}
>  }
> @@ -707,7 +706,6 @@ const char *rte_get_tx_ol_flag_name(uint64_t mask)
>  		{ PKT_TX_SEC_OFFLOAD, PKT_TX_SEC_OFFLOAD, NULL },
>  		{ PKT_TX_UDP_SEG, PKT_TX_UDP_SEG, NULL },
>  		{ PKT_TX_OUTER_UDP_CKSUM, PKT_TX_OUTER_UDP_CKSUM, NULL },
> -		{ PKT_TX_METADATA, PKT_TX_METADATA, NULL },
>  	};
>  	const char *name;
>  	unsigned int i;
> diff --git a/lib/librte_mbuf/rte_mbuf_core.h b/lib/librte_mbuf/rte_mbuf_core.h
> index 3022701..edfc7e9 100644
> --- a/lib/librte_mbuf/rte_mbuf_core.h
> +++ b/lib/librte_mbuf/rte_mbuf_core.h
> @@ -192,11 +192,6 @@
>  /* add new TX flags here, don't forget to update PKT_LAST_FREE  */
>  
>  /**
> - * Indicate that the metadata field in the mbuf is in use.
> - */
> -#define PKT_TX_METADATA	(1ULL << 40)
> -
> -/**

You should also update PKT_LAST_FREE just above.


>   * Outer UDP checksum offload flag. This flag is used for enabling
>   * outer UDP checksum in PMD. To use outer UDP checksum, the user needs to
>   * 1) Enable the following in mbuf,
> @@ -389,8 +384,7 @@
>  		PKT_TX_MACSEC |		 \
>  		PKT_TX_SEC_OFFLOAD |	 \
>  		PKT_TX_UDP_SEG |	 \
> -		PKT_TX_OUTER_UDP_CKSUM | \
> -		PKT_TX_METADATA)
> +		PKT_TX_OUTER_UDP_CKSUM)
>  
>  /**
>   * Mbuf having an external buffer attached. shinfo in mbuf must be filled.
> @@ -601,17 +595,6 @@ struct rte_mbuf {
>  			/**< User defined tags. See rte_distributor_process() */
>  			uint32_t usr;
>  		} hash;                   /**< hash information */
> -		struct {
> -			/**
> -			 * Application specific metadata value
> -			 * for egress flow rule match.
> -			 * Valid if PKT_TX_METADATA is set.
> -			 * Located here to allow conjunct use
> -			 * with hash.sched.hi.
> -			 */
> -			uint32_t tx_metadata;
> -			uint32_t reserved;
> -		};
>  	};
>  
>  	/** Outer VLAN TCI (CPU order), valid if PKT_RX_QINQ is set. */
> -- 
> 1.8.3.1
>
  
Slava Ovsiienko Oct. 31, 2019, 4:07 p.m. UTC | #3
Hi, 

> -----Original Message-----
> From: Olivier Matz <olivier.matz@6wind.com>
> Sent: Thursday, October 31, 2019 17:51
> To: Slava Ovsiienko <viacheslavo@mellanox.com>
> Cc: dev@dpdk.org; Matan Azrad <matan@mellanox.com>; Raslan
> Darawsheh <rasland@mellanox.com>; Thomas Monjalon
> <thomas@monjalon.net>; arybchenko@solarflare.com; Ori Kam
> <orika@mellanox.com>
> Subject: Re: [PATCH v7 2/2] ethdev: move egress metadata to dynamic field
> 
> Hi,
> 
> On Thu, Oct 31, 2019 at 01:05:21PM +0000, Viacheslav Ovsiienko wrote:
> > The dynamic mbuf fields were introduced by [1]. The egress metadata is
> > good candidate to be moved from statically allocated field tx_metadata
> > to dynamic one. Because mbufs are used in half-duplex fashion only, it
> > is safe to share this dynamic field with ingress metadata.
> >
> > The shared dynamic field contains either egress (if application going
> > to transmit mbuf with tx_burst) or ingress (if mbuf is received with
> > rx_burst) metadata and can be accessed by RTE_FLOW_DYNF_METADATA()
> > macro or with
> > rte_flow_dynf_metadata_set() and rte_flow_dynf_metadata_get() helper
> > routines. PKT_TX_DYNF_METADATA/PKT_RX_DYNF_METADATA flag will be
> set
> > along with the data.
> >
> > The mbuf dynamic field must be registered by calling
> > rte_flow_dynf_metadata_register() prior accessing the data.
> >
> > The availability of dynamic mbuf metadata field can be checked with
> > rte_flow_dynf_metadata_avail() routine.
> >
> > DEV_TX_OFFLOAD_MATCH_METADATA offload and configuration flag is
> removed.
> > The metadata support in PMDs is engaged on dynamic field registration.
> >
> > Metadata feature is getting complex. We might have some set of actions
> > and items that might be supported by PMDs in multiple combinations,
> > the supported values and masks are the subjects to query by perfroming
> > trials (with rte_flow_validate).
> >

[snip]

> > +/* Mbuf dynamic flags for metadata. */
> >  #define PKT_RX_DYNF_METADATA (rte_flow_dynf_metadata_mask)
> > +#define PKT_TX_DYNF_METADATA (rte_flow_dynf_metadata_mask)
> 
> Should we have 2 defines pointing to the same mask? Shall we use
> PKT_DYNF_METADATA for both?
It is just a style issue, we have two sets of PKT_RX_xxxx and PKT_TX_xxx flags,
it just looks nice to use appropriate flags in RX/TX parts of datapath.

> >
> >  __rte_experimental
> >  static inline uint32_t
> > diff --git a/lib/librte_mbuf/rte_mbuf.c b/lib/librte_mbuf/rte_mbuf.c
> > index 8c51dc1..35df1c4 100644
> > --- a/lib/librte_mbuf/rte_mbuf.c
> > +++ b/lib/librte_mbuf/rte_mbuf.c
> > @@ -670,7 +670,6 @@ const char *rte_get_tx_ol_flag_name(uint64_t
> mask)
> >  	case PKT_TX_SEC_OFFLOAD: return "PKT_TX_SEC_OFFLOAD";
> >  	case PKT_TX_UDP_SEG: return "PKT_TX_UDP_SEG";
> >  	case PKT_TX_OUTER_UDP_CKSUM: return
> "PKT_TX_OUTER_UDP_CKSUM";
> > -	case PKT_TX_METADATA: return "PKT_TX_METADATA";
> >  	default: return NULL;
> >  	}
> >  }
> > @@ -707,7 +706,6 @@ const char *rte_get_tx_ol_flag_name(uint64_t
> mask)
> >  		{ PKT_TX_SEC_OFFLOAD, PKT_TX_SEC_OFFLOAD, NULL },
> >  		{ PKT_TX_UDP_SEG, PKT_TX_UDP_SEG, NULL },
> >  		{ PKT_TX_OUTER_UDP_CKSUM,
> PKT_TX_OUTER_UDP_CKSUM, NULL },
> > -		{ PKT_TX_METADATA, PKT_TX_METADATA, NULL },
> >  	};
> >  	const char *name;
> >  	unsigned int i;
> > diff --git a/lib/librte_mbuf/rte_mbuf_core.h
> > b/lib/librte_mbuf/rte_mbuf_core.h index 3022701..edfc7e9 100644
> > --- a/lib/librte_mbuf/rte_mbuf_core.h
> > +++ b/lib/librte_mbuf/rte_mbuf_core.h
> > @@ -192,11 +192,6 @@
> >  /* add new TX flags here, don't forget to update PKT_LAST_FREE  */
> >
> >  /**
> > - * Indicate that the metadata field in the mbuf is in use.
> > - */
> > -#define PKT_TX_METADATA	(1ULL << 40)
> > -
> > -/**
> 
> You should also update PKT_LAST_FREE just above.
> 
Yes, my bad, will fix.

[snip] 

With best regards, Slava
  

Patch

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index 4478069..49c45a3 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -18718,12 +18718,13 @@  struct cmd_config_tx_metadata_specific_result {
 
 	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
 		return;
-	ports[res->port_id].tx_metadata = rte_cpu_to_be_32(res->value);
+	ports[res->port_id].tx_metadata = res->value;
 	/* Add/remove callback to insert valid metadata in every Tx packet. */
 	if (ports[res->port_id].tx_metadata)
 		add_tx_md_callback(res->port_id);
 	else
 		remove_tx_md_callback(res->port_id);
+	rte_flow_dynf_metadata_register();
 }
 
 cmdline_parse_token_string_t cmd_config_tx_metadata_specific_port =
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 0fc5b45..206c12b 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1167,10 +1167,6 @@  struct extmem_param {
 		      DEV_TX_OFFLOAD_MBUF_FAST_FREE))
 			port->dev_conf.txmode.offloads &=
 				~DEV_TX_OFFLOAD_MBUF_FAST_FREE;
-		if (!(port->dev_info.tx_offload_capa &
-			DEV_TX_OFFLOAD_MATCH_METADATA))
-			port->dev_conf.txmode.offloads &=
-				~DEV_TX_OFFLOAD_MATCH_METADATA;
 		if (numa_support) {
 			if (port_numa[pid] != NUMA_NO_CONFIG)
 				port_per_socket[port_numa[pid]]++;
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 8da1e8e..caabf32 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -193,7 +193,7 @@  struct rte_port {
 	struct softnic_port     softport;  /**< softnic params */
 #endif
 	/**< metadata value to insert in Tx packets. */
-	rte_be32_t		tx_metadata;
+	uint32_t		tx_metadata;
 	const struct rte_eth_rxtx_callback *tx_set_md_cb[MAX_QUEUE_ID+1];
 };
 
diff --git a/app/test-pmd/util.c b/app/test-pmd/util.c
index 56075b3..cf41864 100644
--- a/app/test-pmd/util.c
+++ b/app/test-pmd/util.c
@@ -82,8 +82,9 @@ 
 			       mb->vlan_tci, mb->vlan_tci_outer);
 		else if (ol_flags & PKT_RX_VLAN)
 			printf(" - VLAN tci=0x%x", mb->vlan_tci);
-		if (ol_flags & PKT_TX_METADATA)
-			printf(" - Tx metadata: 0x%x", mb->tx_metadata);
+		if (ol_flags & PKT_TX_DYNF_METADATA)
+			printf(" - Tx metadata: 0x%x",
+			       *RTE_FLOW_DYNF_METADATA(mb));
 		if (ol_flags & PKT_RX_DYNF_METADATA)
 			printf(" - Rx metadata: 0x%x",
 			       *RTE_FLOW_DYNF_METADATA(mb));
@@ -188,10 +189,12 @@ 
 	 * Add metadata value to every Tx packet,
 	 * and set ol_flags accordingly.
 	 */
-	for (i = 0; i < nb_pkts; i++) {
-		pkts[i]->tx_metadata = ports[port_id].tx_metadata;
-		pkts[i]->ol_flags |= PKT_TX_METADATA;
-	}
+	if (rte_flow_dynf_metadata_avail())
+		for (i = 0; i < nb_pkts; i++) {
+			*RTE_FLOW_DYNF_METADATA(pkts[i]) =
+						ports[port_id].tx_metadata;
+			pkts[i]->ol_flags |= PKT_TX_DYNF_METADATA;
+		}
 	return nb_pkts;
 }
 
diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
index 854bc26..61ecffc 100644
--- a/app/test/test_mbuf.c
+++ b/app/test/test_mbuf.c
@@ -1669,7 +1669,6 @@  struct flag_name {
 		VAL_NAME(PKT_TX_SEC_OFFLOAD),
 		VAL_NAME(PKT_TX_UDP_SEG),
 		VAL_NAME(PKT_TX_OUTER_UDP_CKSUM),
-		VAL_NAME(PKT_TX_METADATA),
 	};
 
 	/* Test case to check with valid flag */
diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index c943aca..630e4c0 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -664,7 +664,7 @@  Item: ``META``
 Matches 32 bit metadata item set.
 
 On egress, metadata can be set either by mbuf metadata field with
-PKT_TX_METADATA flag or ``SET_META`` action. On ingress, ``SET_META``
+PKT_TX_DYNF_METADATA flag or ``SET_META`` action. On ingress, ``SET_META``
 action sets metadata for a packet and the metadata will be reported via
 ``metadata`` dynamic field of ``rte_mbuf`` with PKT_RX_DYNF_METADATA flag.
 
@@ -2482,8 +2482,8 @@  Action: ``SET_META``
 
 Set metadata. Item ``META`` matches metadata.
 
-Metadata set by mbuf metadata field with PKT_TX_METADATA flag on egress will be
-overridden by this action. On ingress, the metadata will be carried by
+Metadata set by mbuf metadata field with PKT_TX_DYNF_METADATA flag on egress
+will be overridden by this action. On ingress, the metadata will be carried by
 ``metadata`` dynamic field of ``rte_mbuf`` which can be accessed by
 ``RTE_FLOW_DYNF_METADATA()``. PKT_RX_DYNF_METADATA flag will be set along
 with the data.
diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 963c4f8..2e9a596 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -357,6 +357,11 @@  API Changes
   is the minor compatibility issue for applications in case of 32-bit values
   supported.
 
+* metadata: the tx_metadata mbuf field is moved to dymanic one.
+  PKT_TX_METADATA flag is replaced with PKT_TX_DYNF_METADATA.
+  DEV_TX_OFFLOAD_MATCH_METADATA offload flag is removed, now metadata
+  support in PMD is engaged on dynamic field registration.
+
 * sched: The pipe nodes configuration parameters such as number of pipes,
   pipe queue sizes, pipe profiles, etc., are moved from port level structure
   to subport level. This allows different subports of the same port to
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index d9a7fd4..f961bff 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -793,7 +793,7 @@  struct field_modify_info modify_tcp[] = {
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-flow_dv_validate_item_meta(struct rte_eth_dev *dev,
+flow_dv_validate_item_meta(struct rte_eth_dev *dev __rte_unused,
 			   const struct rte_flow_item *item,
 			   const struct rte_flow_attr *attr,
 			   struct rte_flow_error *error)
@@ -801,17 +801,10 @@  struct field_modify_info modify_tcp[] = {
 	const struct rte_flow_item_meta *spec = item->spec;
 	const struct rte_flow_item_meta *mask = item->mask;
 	const struct rte_flow_item_meta nic_mask = {
-		.data = RTE_BE32(UINT32_MAX)
+		.data = UINT32_MAX
 	};
 	int ret;
-	uint64_t offloads = dev->data->dev_conf.txmode.offloads;
 
-	if (!(offloads & DEV_TX_OFFLOAD_MATCH_METADATA))
-		return rte_flow_error_set(error, EPERM,
-					  RTE_FLOW_ERROR_TYPE_ITEM,
-					  NULL,
-					  "match on metadata offload "
-					  "configuration is off for this port");
 	if (!spec)
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ITEM_SPEC,
@@ -4750,10 +4743,10 @@  struct field_modify_info modify_tcp[] = {
 		meta_m = &rte_flow_item_meta_mask;
 	meta_v = (const void *)item->spec;
 	if (meta_v) {
-		MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_a,
-			 rte_be_to_cpu_32(meta_m->data));
-		MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_a,
-			 rte_be_to_cpu_32(meta_v->data & meta_m->data));
+		MLX5_SET(fte_match_set_misc2, misc2_m,
+			 metadata_reg_a, meta_m->data);
+		MLX5_SET(fte_match_set_misc2, misc2_v,
+			 metadata_reg_a, meta_v->data & meta_m->data);
 	}
 }
 
diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index f597c89..88a4378 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -2281,8 +2281,8 @@  enum mlx5_txcmp_code {
 	es->swp_offs = txq_mbuf_to_swp(loc, &es->swp_flags, olx);
 	/* Fill metadata field if needed. */
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
-		       loc->mbuf->ol_flags & PKT_TX_METADATA ?
-		       loc->mbuf->tx_metadata : 0 : 0;
+		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
+		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
 	/* Engage VLAN tag insertion feature if requested. */
 	if (MLX5_TXOFF_CONFIG(VLAN) &&
 	    loc->mbuf->ol_flags & PKT_TX_VLAN_PKT) {
@@ -2341,8 +2341,8 @@  enum mlx5_txcmp_code {
 	es->swp_offs = txq_mbuf_to_swp(loc, &es->swp_flags, olx);
 	/* Fill metadata field if needed. */
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
-		       loc->mbuf->ol_flags & PKT_TX_METADATA ?
-		       loc->mbuf->tx_metadata : 0 : 0;
+		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
+		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
 	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
 				(sizeof(uint16_t) +
 				 sizeof(rte_v128u32_t)),
@@ -2434,8 +2434,8 @@  enum mlx5_txcmp_code {
 	es->swp_offs = txq_mbuf_to_swp(loc, &es->swp_flags, olx);
 	/* Fill metadata field if needed. */
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
-		       loc->mbuf->ol_flags & PKT_TX_METADATA ?
-		       loc->mbuf->tx_metadata : 0 : 0;
+		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
+		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
 	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
 				(sizeof(uint16_t) +
 				 sizeof(rte_v128u32_t)),
@@ -2628,8 +2628,8 @@  enum mlx5_txcmp_code {
 	es->swp_offs = txq_mbuf_to_swp(loc, &es->swp_flags, olx);
 	/* Fill metadata field if needed. */
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
-		       loc->mbuf->ol_flags & PKT_TX_METADATA ?
-		       loc->mbuf->tx_metadata : 0 : 0;
+		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
+		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
 	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
 				(sizeof(uint16_t) +
 				 sizeof(rte_v128u32_t)),
@@ -3700,8 +3700,8 @@  enum mlx5_txcmp_code {
 		return false;
 	/* Fill metadata field if needed. */
 	if (MLX5_TXOFF_CONFIG(METADATA) &&
-		es->metadata != (loc->mbuf->ol_flags & PKT_TX_METADATA ?
-				 loc->mbuf->tx_metadata : 0))
+		es->metadata != (loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
+				 *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0))
 		return false;
 	/* There must be no VLAN packets in eMPW loop. */
 	if (MLX5_TXOFF_CONFIG(VLAN))
@@ -5149,7 +5149,7 @@  enum mlx5_txcmp_code {
 		 */
 		olx |= MLX5_TXOFF_CONFIG_EMPW;
 	}
-	if (tx_offloads & DEV_TX_OFFLOAD_MATCH_METADATA) {
+	if (rte_flow_dynf_metadata_avail()) {
 		/* We should support Flow metadata. */
 		olx |= MLX5_TXOFF_CONFIG_METADATA;
 	}
diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.h b/drivers/net/mlx5/mlx5_rxtx_vec.h
index b54ff72..85e0bd5 100644
--- a/drivers/net/mlx5/mlx5_rxtx_vec.h
+++ b/drivers/net/mlx5/mlx5_rxtx_vec.h
@@ -19,12 +19,6 @@ 
 	 DEV_TX_OFFLOAD_TCP_CKSUM | \
 	 DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM)
 
-/* HW offload capabilities of vectorized Tx. */
-#define MLX5_VEC_TX_OFFLOAD_CAP \
-	(MLX5_VEC_TX_CKSUM_OFFLOAD_CAP | \
-	 DEV_TX_OFFLOAD_MATCH_METADATA | \
-	 DEV_TX_OFFLOAD_MULTI_SEGS)
-
 /*
  * Compile time sanity check for vectorized functions.
  */
diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c
index dfc379c..97991f0 100644
--- a/drivers/net/mlx5/mlx5_txq.c
+++ b/drivers/net/mlx5/mlx5_txq.c
@@ -128,10 +128,6 @@ 
 			offloads |= (DEV_TX_OFFLOAD_VXLAN_TNL_TSO |
 				     DEV_TX_OFFLOAD_GRE_TNL_TSO);
 	}
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
-	if (config->dv_flow_en)
-		offloads |= DEV_TX_OFFLOAD_MATCH_METADATA;
-#endif
 	return offloads;
 }
 
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 68aca1f..23b751f 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -161,7 +161,6 @@  struct rte_eth_xstats_name_off {
 	RTE_TX_OFFLOAD_BIT2STR(UDP_TNL_TSO),
 	RTE_TX_OFFLOAD_BIT2STR(IP_TNL_TSO),
 	RTE_TX_OFFLOAD_BIT2STR(OUTER_UDP_CKSUM),
-	RTE_TX_OFFLOAD_BIT2STR(MATCH_METADATA),
 };
 
 #undef RTE_TX_OFFLOAD_BIT2STR
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 9b69255..28e29c7 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1145,11 +1145,6 @@  struct rte_eth_conf {
 #define DEV_TX_OFFLOAD_IP_TNL_TSO       0x00080000
 /** Device supports outer UDP checksum */
 #define DEV_TX_OFFLOAD_OUTER_UDP_CKSUM  0x00100000
-/**
- * Device supports match on metadata Tx offload..
- * Application must set PKT_TX_METADATA and mbuf metadata field.
- */
-#define DEV_TX_OFFLOAD_MATCH_METADATA   0x00200000
 
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /**< Device supports Rx queue setup after device started*/
diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
index f6e050c..51d8292 100644
--- a/lib/librte_ethdev/rte_flow.h
+++ b/lib/librte_ethdev/rte_flow.h
@@ -1268,12 +1268,12 @@  struct rte_flow_item_icmp6_nd_opt_tla_eth {
 /**
  * RTE_FLOW_ITEM_TYPE_META
  *
- * Matches a specified metadata value. On egress, metadata can be set either by
- * mbuf tx_metadata field with PKT_TX_METADATA flag or
- * RTE_FLOW_ACTION_TYPE_SET_META. On ingress, RTE_FLOW_ACTION_TYPE_SET_META sets
- * metadata for a packet and the metadata will be reported via mbuf metadata
- * dynamic field with PKT_RX_DYNF_METADATA flag. The dynamic mbuf field must be
- * registered in advance by rte_flow_dynf_metadata_register().
+ * Matches a specified metadata value. On egress, metadata can be set
+ * either by mbuf dynamic metadata field with PKT_TX_DYNF_METADATA flag or
+ * RTE_FLOW_ACTION_TYPE_SET_META. On ingress, RTE_FLOW_ACTION_TYPE_SET_META
+ * sets metadata for a packet and the metadata will be reported via mbuf
+ * metadata dynamic field with PKT_RX_DYNF_METADATA flag. The dynamic mbuf
+ * field must be registered in advance by rte_flow_dynf_metadata_register().
  */
 struct rte_flow_item_meta {
 	uint32_t data;
@@ -2450,8 +2450,8 @@  struct rte_flow_action_set_mac {
  *
  * RTE_FLOW_ACTION_TYPE_SET_META
  *
- * Set metadata. Metadata set by mbuf tx_metadata field with
- * PKT_TX_METADATA flag on egress will be overridden by this action. On
+ * Set metadata. Metadata set by mbuf metadata dynamic field with
+ * PKT_TX_DYNF_DATA flag on egress will be overridden by this action. On
  * ingress, the metadata will be carried by mbuf metadata dynamic field
  * with PKT_RX_DYNF_METADATA flag if set.  The dynamic mbuf field must be
  * registered in advance by rte_flow_dynf_metadata_register().
@@ -2478,8 +2478,9 @@  struct rte_flow_action_set_meta {
 #define RTE_FLOW_DYNF_METADATA(m) \
 	RTE_MBUF_DYNFIELD((m), rte_flow_dynf_metadata_offs, uint32_t *)
 
-/* Mbuf dynamic flag for metadata. */
+/* Mbuf dynamic flags for metadata. */
 #define PKT_RX_DYNF_METADATA (rte_flow_dynf_metadata_mask)
+#define PKT_TX_DYNF_METADATA (rte_flow_dynf_metadata_mask)
 
 __rte_experimental
 static inline uint32_t
diff --git a/lib/librte_mbuf/rte_mbuf.c b/lib/librte_mbuf/rte_mbuf.c
index 8c51dc1..35df1c4 100644
--- a/lib/librte_mbuf/rte_mbuf.c
+++ b/lib/librte_mbuf/rte_mbuf.c
@@ -670,7 +670,6 @@  const char *rte_get_tx_ol_flag_name(uint64_t mask)
 	case PKT_TX_SEC_OFFLOAD: return "PKT_TX_SEC_OFFLOAD";
 	case PKT_TX_UDP_SEG: return "PKT_TX_UDP_SEG";
 	case PKT_TX_OUTER_UDP_CKSUM: return "PKT_TX_OUTER_UDP_CKSUM";
-	case PKT_TX_METADATA: return "PKT_TX_METADATA";
 	default: return NULL;
 	}
 }
@@ -707,7 +706,6 @@  const char *rte_get_tx_ol_flag_name(uint64_t mask)
 		{ PKT_TX_SEC_OFFLOAD, PKT_TX_SEC_OFFLOAD, NULL },
 		{ PKT_TX_UDP_SEG, PKT_TX_UDP_SEG, NULL },
 		{ PKT_TX_OUTER_UDP_CKSUM, PKT_TX_OUTER_UDP_CKSUM, NULL },
-		{ PKT_TX_METADATA, PKT_TX_METADATA, NULL },
 	};
 	const char *name;
 	unsigned int i;
diff --git a/lib/librte_mbuf/rte_mbuf_core.h b/lib/librte_mbuf/rte_mbuf_core.h
index 3022701..edfc7e9 100644
--- a/lib/librte_mbuf/rte_mbuf_core.h
+++ b/lib/librte_mbuf/rte_mbuf_core.h
@@ -192,11 +192,6 @@ 
 /* add new TX flags here, don't forget to update PKT_LAST_FREE  */
 
 /**
- * Indicate that the metadata field in the mbuf is in use.
- */
-#define PKT_TX_METADATA	(1ULL << 40)
-
-/**
  * Outer UDP checksum offload flag. This flag is used for enabling
  * outer UDP checksum in PMD. To use outer UDP checksum, the user needs to
  * 1) Enable the following in mbuf,
@@ -389,8 +384,7 @@ 
 		PKT_TX_MACSEC |		 \
 		PKT_TX_SEC_OFFLOAD |	 \
 		PKT_TX_UDP_SEG |	 \
-		PKT_TX_OUTER_UDP_CKSUM | \
-		PKT_TX_METADATA)
+		PKT_TX_OUTER_UDP_CKSUM)
 
 /**
  * Mbuf having an external buffer attached. shinfo in mbuf must be filled.
@@ -601,17 +595,6 @@  struct rte_mbuf {
 			/**< User defined tags. See rte_distributor_process() */
 			uint32_t usr;
 		} hash;                   /**< hash information */
-		struct {
-			/**
-			 * Application specific metadata value
-			 * for egress flow rule match.
-			 * Valid if PKT_TX_METADATA is set.
-			 * Located here to allow conjunct use
-			 * with hash.sched.hi.
-			 */
-			uint32_t tx_metadata;
-			uint32_t reserved;
-		};
 	};
 
 	/** Outer VLAN TCI (CPU order), valid if PKT_RX_QINQ is set. */