[v3,00/10] introduce memarea library

  The memarea library is an allocator of variable-size object which based
on a memory region. The main features are as follows:

- The default alignment size is RTE_CACHE_LINE_SIZE.

- The memory region can be initialized from the following memory
  sources:
  1. RTE memory: e.g. invoke rte_malloc_socket to obtain.
  2. System API: e.g. invoke posix_memalign to obtain.
  3. User provided address: it can be from extended memory as long as
     it is available. The address must be aligned to
     RTE_CACHE_LINE_SIZE.
  4. User provided memarea: it can be from another memarea.

- It provides refcnt feature which could be useful in multi-reader
  scenario.

- It provides backup memory mechanism, the memarea could use another
  memarea as a backup.

Note:
a) The memarea is oriented towards the application layer, which could
provides 'region-based memory management' [1] function.
b) The eal library also provide memory zone/heap management, but these
are tied to huge pages management.

[1] https://en.wikipedia.org/wiki/Region-based_memory_management

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Chengwen Feng (10):
  memarea: introduce memarea library
  test/memarea: support memarea test
  memarea: support alloc/free/update-refcnt API
  test/memarea: support alloc/free/update-refcnt test
  memarea: support dump API
  test/memarea: support dump test
  memarea: support backup memory mechanism
  test/memarea: support backup memory test
  memarea: detect memory corruption based on magic
  test/memarea: support no MT-safe test

---
v3:
* add memory source of RTE memory.
* add algorithm field to facilitate the introduction of new algorithms.
* fix memarea log don't output problem.
v2:
* fix compile issues reported by dpdk-test-report.
* address Dimitry and Jerin's comments.
* add no MT-safe test.

 MAINTAINERS                            |   6 +
 app/test/meson.build                   |   2 +
 app/test/test_memarea.c                | 401 +++++++++++++++++++++++
 doc/api/doxy-api-index.md              |   3 +-
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/prog_guide/index.rst        |   1 +
 doc/guides/prog_guide/memarea_lib.rst  |  55 ++++
 doc/guides/rel_notes/release_22_11.rst |   6 +
 lib/eal/common/eal_common_log.c        |   1 +
 lib/eal/include/rte_log.h              |   1 +
 lib/memarea/memarea_private.h          |  37 +++
 lib/memarea/meson.build                |  16 +
 lib/memarea/rte_memarea.c              | 430 +++++++++++++++++++++++++
 lib/memarea/rte_memarea.h              | 229 +++++++++++++
 lib/memarea/version.map                |  16 +
 lib/meson.build                        |   1 +
 16 files changed, 1205 insertions(+), 1 deletion(-)
 create mode 100644 app/test/test_memarea.c
 create mode 100644 doc/guides/prog_guide/memarea_lib.rst
 create mode 100644 lib/memarea/memarea_private.h
 create mode 100644 lib/memarea/meson.build
 create mode 100644 lib/memarea/rte_memarea.c
 create mode 100644 lib/memarea/rte_memarea.h
 create mode 100644 lib/memarea/version.map
  

On Sat, Sep 24, 2022 at 9:56 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
>
> The memarea library is an allocator of variable-size object which based
> on a memory region. The main features are as follows:
>
> - The default alignment size is RTE_CACHE_LINE_SIZE.
>
> - The memory region can be initialized from the following memory
>   sources:
>   1. RTE memory: e.g. invoke rte_malloc_socket to obtain.
>   2. System API: e.g. invoke posix_memalign to obtain.
>   3. User provided address: it can be from extended memory as long as
>      it is available. The address must be aligned to
>      RTE_CACHE_LINE_SIZE.
>   4. User provided memarea: it can be from another memarea.
>
> - It provides refcnt feature which could be useful in multi-reader
>   scenario.
>
> - It provides backup memory mechanism, the memarea could use another
>   memarea as a backup.
>
> Note:
> a) The memarea is oriented towards the application layer, which could
> provides 'region-based memory management' [1] function.
> b) The eal library also provide memory zone/heap management, but these
> are tied to huge pages management.
>
> [1] https://en.wikipedia.org/wiki/Region-based_memory_management
>
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

This series did not pass through the CI, as its patches are spread
over different series in patchwork.
https://patchwork.dpdk.org/project/dpdk/list/?submitter=2146

There was probably something wrong when sending the patches, please
check your setup and repost them.


Thanks.
  

Hi David,

   The v5 (send by datshan@qq.com) is sent to fix it, please have a look.

Thanks.

On 2022/10/3 15:42, David Marchand wrote:
> On Sat, Sep 24, 2022 at 9:56 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
>> The memarea library is an allocator of variable-size object which based
>> on a memory region. The main features are as follows:
>>
>> - The default alignment size is RTE_CACHE_LINE_SIZE.
>>
>> - The memory region can be initialized from the following memory
>>    sources:
>>    1. RTE memory: e.g. invoke rte_malloc_socket to obtain.
>>    2. System API: e.g. invoke posix_memalign to obtain.
>>    3. User provided address: it can be from extended memory as long as
>>       it is available. The address must be aligned to
>>       RTE_CACHE_LINE_SIZE.
>>    4. User provided memarea: it can be from another memarea.
>>
>> - It provides refcnt feature which could be useful in multi-reader
>>    scenario.
>>
>> - It provides backup memory mechanism, the memarea could use another
>>    memarea as a backup.
>>
>> Note:
>> a) The memarea is oriented towards the application layer, which could
>> provides 'region-based memory management' [1] function.
>> b) The eal library also provide memory zone/heap management, but these
>> are tied to huge pages management.
>>
>> [1] https://en.wikipedia.org/wiki/Region-based_memory_management
>>
>> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> This series did not pass through the CI, as its patches are spread
> over different series in patchwork.
> https://patchwork.dpdk.org/project/dpdk/list/?submitter=2146
>
> There was probably something wrong when sending the patches, please
> check your setup and repost them.
>
>
> Thanks.
>