[dpdk-dev,2/4] net/mlx5: forbid MR registration in secondary process

Message ID 20180115065420.44065-2-xuemingl@mellanox.com (mailing list archive)
State Superseded, archived
Headers

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation fail Compilation issues

Commit Message

Xueming Li Jan. 15, 2018, 6:54 a.m. UTC
  Secondary process are not alloed to access verbs resources, add check to
prevent MR registration in data path.

Signed-off-by: Xueming Li <xuemingl@mellanox.com>
---
 doc/guides/nics/mlx5.rst     | 1 +
 drivers/net/mlx5/mlx5_mr.c   | 3 +++
 drivers/net/mlx5/mlx5_rxtx.h | 3 ---
 3 files changed, 4 insertions(+), 3 deletions(-)
  

Comments

Nélio Laranjeiro Jan. 15, 2018, 3:33 p.m. UTC | #1
Hi Xueming,

On Mon, Jan 15, 2018 at 02:54:18PM +0800, Xueming Li wrote:
> Secondary process are not alloed to access verbs resources, add check to
> prevent MR registration in data path.
> 
> Signed-off-by: Xueming Li <xuemingl@mellanox.com>
> ---
>  doc/guides/nics/mlx5.rst     | 1 +
>  drivers/net/mlx5/mlx5_mr.c   | 3 +++
>  drivers/net/mlx5/mlx5_rxtx.h | 3 ---
>  3 files changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
> index bdc2216c0..4373c7990 100644
> --- a/doc/guides/nics/mlx5.rst
> +++ b/doc/guides/nics/mlx5.rst
> @@ -107,6 +107,7 @@ Limitations
>  - Inner RSS for VXLAN frames is not supported yet.
>  - Hardware checksum RX offloads for VXLAN inner header are not supported yet.
>  - Forked secondary process not supported.
> +- Mempools used in secondary process TX have to be initialized in primary process before rte_eth_dev_start().
>  - Flow pattern without any specific vlan will match for vlan packets as well:
>  
>    When VLAN spec is not specified in the pattern, the matching rule will be created with VLAN as a wild card.
> diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c
> index 6b29eed55..3c80fbb89 100644
> --- a/drivers/net/mlx5/mlx5_mr.c
> +++ b/drivers/net/mlx5/mlx5_mr.c
> @@ -278,6 +278,9 @@ priv_mr_new(struct priv *priv, struct rte_mempool *mp)
>  	unsigned int i;
>  	struct mlx5_mr *mr;
>  
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		rte_panic("Please init mempool before rte_eth_dev_start() in primary process: %s",
> +			  mp->name);

Panic should be avoided unless the situation becomes totally
un-predectible.

Here you can return NULL and stop the Tx burst increasing at the same
time the error statistics.  The panic message can remain as a warning.
This don't deserve a panic.

>  	mr = rte_zmalloc_socket(__func__, sizeof(*mr), 0, mp->socket_id);
>  	if (!mr) {
>  		DEBUG("unable to configure MR, ibv_reg_mr() failed.");
> diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
> index 2eb2f0506..18e1d26f3 100644
> --- a/drivers/net/mlx5/mlx5_rxtx.h
> +++ b/drivers/net/mlx5/mlx5_rxtx.h
> @@ -561,9 +561,6 @@ mlx5_tx_mb2mr(struct mlx5_txq_data *txq, struct rte_mbuf *mb)
>  		}
>  		if (txq->mp2mr[i]->start <= addr &&
>  		    txq->mp2mr[i]->end >= addr) {
> -			assert(txq->mp2mr[i]->lkey != (uint32_t)-1);
> -			assert(rte_cpu_to_be_32(txq->mp2mr[i]->mr->lkey) ==
> -			       txq->mp2mr[i]->lkey);
>  			txq->mr_cache_idx = i;
>  			return txq->mp2mr[i]->lkey;
>  		}
> -- 
> 2.13.3
> 

Thanks,
  

Patch

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index bdc2216c0..4373c7990 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -107,6 +107,7 @@  Limitations
 - Inner RSS for VXLAN frames is not supported yet.
 - Hardware checksum RX offloads for VXLAN inner header are not supported yet.
 - Forked secondary process not supported.
+- Mempools used in secondary process TX have to be initialized in primary process before rte_eth_dev_start().
 - Flow pattern without any specific vlan will match for vlan packets as well:
 
   When VLAN spec is not specified in the pattern, the matching rule will be created with VLAN as a wild card.
diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c
index 6b29eed55..3c80fbb89 100644
--- a/drivers/net/mlx5/mlx5_mr.c
+++ b/drivers/net/mlx5/mlx5_mr.c
@@ -278,6 +278,9 @@  priv_mr_new(struct priv *priv, struct rte_mempool *mp)
 	unsigned int i;
 	struct mlx5_mr *mr;
 
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		rte_panic("Please init mempool before rte_eth_dev_start() in primary process: %s",
+			  mp->name);
 	mr = rte_zmalloc_socket(__func__, sizeof(*mr), 0, mp->socket_id);
 	if (!mr) {
 		DEBUG("unable to configure MR, ibv_reg_mr() failed.");
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 2eb2f0506..18e1d26f3 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -561,9 +561,6 @@  mlx5_tx_mb2mr(struct mlx5_txq_data *txq, struct rte_mbuf *mb)
 		}
 		if (txq->mp2mr[i]->start <= addr &&
 		    txq->mp2mr[i]->end >= addr) {
-			assert(txq->mp2mr[i]->lkey != (uint32_t)-1);
-			assert(rte_cpu_to_be_32(txq->mp2mr[i]->mr->lkey) ==
-			       txq->mp2mr[i]->lkey);
 			txq->mr_cache_idx = i;
 			return txq->mp2mr[i]->lkey;
 		}