mbox series

[v6,0/3] eal: add memory pre-allocation from existing files

Message ID 20211011085644.2716490-1-dkozlyuk@nvidia.com (mailing list archive)
Headers
Series eal: add memory pre-allocation from existing files |

Message

Dmitry Kozlyuk Oct. 11, 2021, 8:56 a.m. UTC 
  Hugepage allocation from the system takes time, resulting in slow
startup or sporadic delays later. Most of the time spent in kernel
is zero-filling memory for security reasons, which may be irrelevant
in a controlled environment. The bottleneck is memory access speed,
so for speeduup the amount of memory cleared must be reduced.
We propose a new EAL option --mem-file FILE1,FILE2,... to quickly
allocate dirty pages from existing files and clean it as necessary.
A new malloc_perf_autotest is provided to estimate the impact.
More details are explained in relevant patches.

v6: fix copyright line (Thomas), add SPDX header for the new test file
    (BTW, why didn't the CI complain in previous versions?)
v5: rebase
v4: getmntent() -> getmntent_r(), better error detection (John Levon)
v3: fix hugepage mount point detection
v2: fix CI failures

Dmitry Kozlyuk (2):
  eal/linux: make hugetlbfs analysis reusable
  app/test: add allocator performance autotest

Viacheslav Ovsiienko (1):
  eal: add memory pre-allocation from existing files

 app/test/meson.build                          |   2 +
 app/test/test_malloc_perf.c                   | 161 +++++++++
 doc/guides/linux_gsg/linux_eal_parameters.rst |  17 +
 lib/eal/common/eal_common_dynmem.c            |   6 +
 lib/eal/common/eal_common_options.c           |  23 ++
 lib/eal/common/eal_internal_cfg.h             |   4 +
 lib/eal/common/eal_memalloc.h                 |   8 +-
 lib/eal/common/eal_options.h                  |   2 +
 lib/eal/common/malloc_elem.c                  |   5 +
 lib/eal/common/malloc_heap.h                  |   8 +
 lib/eal/common/rte_malloc.c                   |  16 +-
 lib/eal/include/rte_memory.h                  |   4 +-
 lib/eal/linux/eal.c                           |  28 ++
 lib/eal/linux/eal_hugepage_info.c             | 158 ++++++---
 lib/eal/linux/eal_hugepage_info.h             |  39 +++
 lib/eal/linux/eal_memalloc.c                  | 328 +++++++++++++++++-
 16 files changed, 739 insertions(+), 70 deletions(-)
 create mode 100644 app/test/test_malloc_perf.c
 create mode 100644 lib/eal/linux/eal_hugepage_info.h

Comments

Thomas Monjalon Oct. 11, 2021, 6:52 p.m. UTC | #1 
11/10/2021 10:56, Dmitry Kozlyuk:
> v6: fix copyright line (Thomas), add SPDX header for the new test file
>     (BTW, why didn't the CI complain in previous versions?)

Probably because the CI doesn't run the script devtools/check-spdx-tag.sh
Cc ci@dpdk.org to add this test.
Lincoln Lavoie Oct. 11, 2021, 9:12 p.m. UTC | #2 
Hi All,

I'm assuming this should be added to the "checkpatch" job / report?
Correct, or do folks feel this should be a separate run / report?

Cheers,
Lincoln

On Mon, Oct 11, 2021 at 2:52 PM Thomas Monjalon <thomas@monjalon.net> wrote:

> 11/10/2021 10:56, Dmitry Kozlyuk:
> > v6: fix copyright line (Thomas), add SPDX header for the new test file
> >     (BTW, why didn't the CI complain in previous versions?)
>
> Probably because the CI doesn't run the script devtools/check-spdx-tag.sh
> Cc ci@dpdk.org to add this test.
>
>
>
>
Thomas Monjalon Oct. 12, 2021, 6:54 a.m. UTC | #3 
11/10/2021 23:12, Lincoln Lavoie:
> Hi All,
> 
> I'm assuming this should be added to the "checkpatch" job / report?

I think yes.

> Correct, or do folks feel this should be a separate run / report?

Please what are the tests already run in this job?
I may suggest more.


> On Mon, Oct 11, 2021 at 2:52 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > 11/10/2021 10:56, Dmitry Kozlyuk:
> > > v6: fix copyright line (Thomas), add SPDX header for the new test file
> > >     (BTW, why didn't the CI complain in previous versions?)
> >
> > Probably because the CI doesn't run the script devtools/check-spdx-tag.sh
> > Cc ci@dpdk.org to add this test.