diff mbox series

[v2,2/2] vdpa/mlx5: fix large VM memory region registration

Message ID 20210923081122.176735-3-xuemingl@nvidia.com (mailing list archive)
State Accepted, archived
Delegated to: Thomas Monjalon
Headers
Series sched: adds function to get 64 bits greatest common divisor |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK
ci/intel-Testing success Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-aarch64-compile-testing success Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-x86_64-compile-testing success Testing PASS

Commit Message

Xueming Li Sept. 23, 2021, 8:11 a.m. UTC 
  When VM size larger than 4G(u32) and memory region larger than 4G, the
32 bits GCD function overflow and returned wrong value that resulted
memory registration failed.

This patch calls 64 bits GCD function to avoid overflow.

Fixes: cc07a42da250 ("vdpa/mlx5: prepare memory regions")
Cc: matan@mellanox.com
Cc: stable@dpdk.org
Reviewed-by: Matan Azrad <matan@nvidia.com>

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/vdpa/mlx5/mlx5_vdpa_mem.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
diff mbox series

Patch

diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_mem.c b/drivers/vdpa/mlx5/mlx5_vdpa_mem.c
index 59ce4e891c..a06681b494 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa_mem.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa_mem.c
@@ -103,15 +103,15 @@  mlx5_vdpa_vhost_mem_regions_prepare(int vid, uint8_t *mode, uint64_t *mem_size,
 			size = mem->regions[i].guest_phys_addr -
 				(mem->regions[i - 1].guest_phys_addr +
 				 mem->regions[i - 1].size);
-			*gcd = rte_get_gcd(*gcd, size);
+			*gcd = rte_get_gcd64(*gcd, size);
 			klm_entries_num += KLM_NUM_MAX_ALIGN(size);
 		}
 		size = mem->regions[i].size;
-		*gcd = rte_get_gcd(*gcd, size);
+		*gcd = rte_get_gcd64(*gcd, size);
 		klm_entries_num += KLM_NUM_MAX_ALIGN(size);
 	}
 	if (*gcd > MLX5_MAX_KLM_BYTE_COUNT)
-		*gcd = rte_get_gcd(*gcd, MLX5_MAX_KLM_BYTE_COUNT);
+		*gcd = rte_get_gcd64(*gcd, MLX5_MAX_KLM_BYTE_COUNT);
 	if (!RTE_IS_POWER_OF_2(*gcd)) {
 		uint64_t candidate_gcd = rte_align64prevpow2(*gcd);