[v1] app/testpmd: add memory dump command

  Introduce new command to dump memory statistics of each socket,
summary, also show changes since last call.

Usage:
    dump_socket

Signed-off-by: Xueming Li <xuemingl@mellanox.com>
---
 app/test-pmd/cmdline.c                      | 52 +++++++++++++++++++++++++++++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |  6 ++++
 2 files changed, 58 insertions(+)
  

On 4/3/2020 7:53 AM, Xueming Li wrote:
> Introduce new command to dump memory statistics of each socket,
> summary, also show changes since last call.
> 
> Usage:
>     dump_socket
> 
> Signed-off-by: Xueming Li <xuemingl@mellanox.com>
> ---
>  app/test-pmd/cmdline.c                      | 52 +++++++++++++++++++++++++++++
>  doc/guides/testpmd_app_ug/testpmd_funcs.rst |  6 ++++
>  2 files changed, 58 insertions(+)
> 
> diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
> index 274e391..9bb64bb 100644
> --- a/app/test-pmd/cmdline.c
> +++ b/app/test-pmd/cmdline.c
> @@ -9568,6 +9568,55 @@ struct cmd_dump_result {
>  #undef DUMP_SIZE
>  }
>  
> +
> +/* Dump the socket memory statistics on console */
> +static void
> +dump_socket_mem(FILE *f)
> +{
> +	struct rte_malloc_socket_stats socket_stats;
> +	unsigned int i;
> +	int64_t total = 0;
> +	int64_t alloc = 0;
> +	int64_t free = 0;
> +	unsigned int n_alloc = 0;
> +	unsigned int n_free = 0;
> +	static int64_t last_allocs;
> +	static int64_t last_total;
> +
> +
> +	for (i = 0; i < RTE_MAX_NUMA_NODES; i++) {
> +		if (rte_malloc_get_socket_stats(i, &socket_stats) ||
> +		    !socket_stats.heap_totalsz_bytes)
> +			continue;
> +		total += socket_stats.heap_totalsz_bytes;
> +		alloc += socket_stats.heap_allocsz_bytes;
> +		free += socket_stats.heap_freesz_bytes;
> +		n_alloc += socket_stats.alloc_count;
> +		n_free += socket_stats.free_count;
> +		fprintf(f,
> +			"Socket %u: size(M) total: %.6lf alloc: %.6lf(%.3lf%%) free: %.6lf \tcount alloc: %-4u free: %u\n",
> +			i,
> +			socket_stats.heap_totalsz_bytes / 1.0e6,
> +			socket_stats.heap_allocsz_bytes / 1.0e6,
> +			(double)socket_stats.heap_allocsz_bytes * 100 /
> +			(double)socket_stats.heap_totalsz_bytes,
> +			socket_stats.heap_freesz_bytes / 1.0e6,
> +			socket_stats.alloc_count,
> +			socket_stats.free_count);

This gives an output like [1], can you please divide to (1024*1024) to convert
byte to Mb, than it can give more clear numbers.


[1]
Socket 0: size(M) total: 2933.915648 alloc: 1871.655424(63.794%) free:
1062.260224      count alloc: 2137 free: 36
Socket 1: size(M) total: 2923.429888 alloc: 1863.400064(63.740%) free:
1060.029824      count alloc: 177  free: 1
Total   : size(M) total: 5857.345536 alloc: 3735.055488(63.767%) free:
2122.290048      count alloc: 2314 free: 37
Memory total change: 0.000000(M), allocation change: 0.000000(M)


> +	}
> +	fprintf(f,
> +		"Total   : size(M) total: %.6lf alloc: %.6lf(%.3lf%%) free: %.6lf \tcount alloc: %-4u free: %u\n",
> +		total / 1.0e6, alloc / 1.0e6,
> +		(double)alloc * 100 / (double)total, free / 1.0e6,
> +		n_alloc, n_free);
> +	if (last_allocs)
> +		fprintf(stdout, "Memory total change: %.6lf(M), allocation change: %.6lf(M)\n",
> +			(total - last_total) / 1.0e6,
> +			(alloc - last_allocs) / 1.0e6);
> +	last_allocs = alloc;
> +	last_total = total;
> +}
> +
>  static void cmd_dump_parsed(void *parsed_result,
>  			    __attribute__((unused)) struct cmdline *cl,
>  			    __attribute__((unused)) void *data)
> @@ -9576,6 +9625,8 @@ static void cmd_dump_parsed(void *parsed_result,
>  
>  	if (!strcmp(res->dump, "dump_physmem"))
>  		rte_dump_physmem_layout(stdout);
> +	else if (!strcmp(res->dump, "dump_socket"))
> +		dump_socket_mem(stdout);
>  	else if (!strcmp(res->dump, "dump_memzone"))
>  		rte_memzone_dump(stdout);
>  	else if (!strcmp(res->dump, "dump_malloc")) {
> @@ -9604,6 +9655,7 @@ static void cmd_dump_parsed(void *parsed_result,
>  	TOKEN_STRING_INITIALIZER(struct cmd_dump_result, dump,
>  		"dump_physmem#"
>  		"dump_memzone#"
> +		"dump_socket#"
>  		"dump_struct_sizes#"
>  		"dump_ring#"
>  		"dump_mempool#"
> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> index 1a9879f..d248337 100644
> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> @@ -539,6 +539,12 @@ Dumps the layout of all memory zones::
>  
>     testpmd> dump_memzone
>  
> +dump socket
> +~~~~~~~~~~~~
> +
> +Dumps the memory usage of all sockets::
> +
> +   testpmd> dump_socket

'dump_socket' looks like it will list the socket information, what do you think
changing the command name to 'dump_socket_mem' to clarify it will dump the
memory information?

>  
>  dump struct size
>  ~~~~~~~~~~~~~~~~
>
  

On Fri, 3 Apr 2020 14:29:06 +0100
Ferruh Yigit <ferruh.yigit@intel.com> wrote:

> > +	int64_t total = 0;
> > +	int64_t alloc = 0;
> > +	int64_t free = 0;

Should be unsigned (ie uint64_t)
  

On Fri, 3 Apr 2020 14:29:06 +0100
Ferruh Yigit <ferruh.yigit@intel.com> wrote:

> > +			socket_stats.heap_totalsz_bytes / 1.0e6,
> > +			socket_stats.heap_allocsz_bytes / 1.0e6,
> > +			(double)socket_stats.heap_allocsz_bytes * 100 /
> > +			(double)socket_stats.heap_totalsz_bytes,
> > +			socket_stats.heap_freesz_bytes / 1.0e6,
> > +			socket_stats.alloc_count,
> > +			socket_stats.free_count);  
> 
> This gives an output like [1], can you please divide to (1024*1024) to convert
> byte to Mb, than it can give more clear numbers.

Agree, the standard is to use 1024*1024 for memory megabytes.
And 1000*1000 for bytes in networking.
  

> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger
> Sent: Friday, April 3, 2020 6:09 PM
> 
> On Fri, 3 Apr 2020 14:29:06 +0100
> Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> 
> > > +			socket_stats.heap_totalsz_bytes / 1.0e6,
> > > +			socket_stats.heap_allocsz_bytes / 1.0e6,
> > > +			(double)socket_stats.heap_allocsz_bytes * 100 /
> > > +			(double)socket_stats.heap_totalsz_bytes,
> > > +			socket_stats.heap_freesz_bytes / 1.0e6,
> > > +			socket_stats.alloc_count,
> > > +			socket_stats.free_count);
> >
> > This gives an output like [1], can you please divide to (1024*1024)
> to convert
> > byte to Mb, than it can give more clear numbers.
> 
> Agree, the standard is to use 1024*1024 for memory megabytes.

That was the old standard. It has been 1000*1000 so for a decade now.

1000000 is the internationally agreed standard for the MB (megabyte) unit. It has been adopted by all major standardization organs, and is the legally binding definition.

If you want to use 1024*1024, the correct unit is MiB (mebibyte). And in this case, it probably makes sense. Just use the correct MiB unit instead of the incorrect MB unit.

For further information, please also refer to:
https://en.wikipedia.org/wiki/Megabyte

> And 1000*1000 for bytes in networking.

Yes.