[v9,1/3] Enable ASan for memory detector on DPDK
Checks
Commit Message
From: Zhihong Peng <zhihongx.peng@intel.com>
`AddressSanitizer
<https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
is a widely-used debugging tool to detect memory access errors.
It helps detect issues like use-after-free, various kinds of buffer
overruns in C/C++ programs, and other similar errors, as well as
printing out detailed debug information whenever an error is detected.
DPDK ASan functionality is currently only supported Linux x86_64.
Support other platforms, need to define ASAN_SHADOW_OFFSET value
according to google ASan document, and configure meson
(config/meson.build).
Here is an example of heap-buffer-overflow bug:
......
char *p = rte_zmalloc(NULL, 7, 0);
p[7] = 'a';
......
Here is an example of use-after-free bug:
......
char *p = rte_zmalloc(NULL, 7, 0);
rte_free(p);
*p = 'a';
......
We can enable ASan by adding below compilation options:
-Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
"-Dbuildtype=debug": This is a non-essential option. When this option
is added, if a memory error occurs, ASan can clearly show where the
code is wrong.
"-Db_lundef=false": When use clang to compile DPDK, this option must
be added.
Signed-off-by: Xueqin Lin <xueqin.lin@intel.com>
Signed-off-by: Zhihong Peng <zhihongx.peng@intel.com>
---
v7: 1) Split doc and code into two.
2) Modify asan.rst doc
v8: No change.
v9: 1) Add the check of libasan library.
2) Add release notes.
---
config/meson.build | 10 +++
devtools/words-case.txt | 1 +
doc/guides/prog_guide/asan.rst | 97 ++++++++++++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_21_11.rst | 9 +++
5 files changed, 118 insertions(+)
create mode 100644 doc/guides/prog_guide/asan.rst
Comments
On Tue, Oct 12, 2021 at 11:54 AM <zhihongx.peng@intel.com> wrote:
>
> From: Zhihong Peng <zhihongx.peng@intel.com>
>
> `AddressSanitizer
> <https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> is a widely-used debugging tool to detect memory access errors.
> It helps detect issues like use-after-free, various kinds of buffer
> overruns in C/C++ programs, and other similar errors, as well as
> printing out detailed debug information whenever an error is detected.
>
Comments below concern added feature with patch 2.
From here...
> DPDK ASan functionality is currently only supported Linux x86_64.
> Support other platforms, need to define ASAN_SHADOW_OFFSET value
> according to google ASan document, and configure meson
> (config/meson.build).
>
> Here is an example of heap-buffer-overflow bug:
> ......
> char *p = rte_zmalloc(NULL, 7, 0);
> p[7] = 'a';
> ......
>
> Here is an example of use-after-free bug:
> ......
> char *p = rte_zmalloc(NULL, 7, 0);
> rte_free(p);
> *p = 'a';
> ......
... to here.
>
> We can enable ASan by adding below compilation options:
> -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
> "-Dbuildtype=debug": This is a non-essential option. When this option
> is added, if a memory error occurs, ASan can clearly show where the
> code is wrong.
> "-Db_lundef=false": When use clang to compile DPDK, this option must
> be added.
>
> Signed-off-by: Xueqin Lin <xueqin.lin@intel.com>
> Signed-off-by: Zhihong Peng <zhihongx.peng@intel.com>
More problematic, linking an external (out of meson) application to a
dpdk compiled with ASan is broken.
My environment contains following targets compiled using
./devtools/test-meson-builds.sh:
$ ls $HOME/builds/
build-arm64-bluefield build-arm64-host-clang build-clang-shared
build-gcc-shared build-ppc64le-power8 build-x86-mingw
build-arm64-dpaa build-arm64-octeontx2 build-clang-static
build-gcc-static build-x86-generic
I stopped at patch 1, configured following target to have ASan in
them, like this:
$ meson configure $HOME/builds/build-gcc-static -Db_sanitize=address
$ meson configure $HOME/builds/build-clang-shared -Db_sanitize=address
-Db_lundef=false
$ meson configure $HOME/builds/build-x86-generic -Db_sanitize=address
^^^^^^^^^^^^^^^^^
This is the target for which we test
linking a dpdk application out of meson.
$ meson configure $HOME/builds/build-arm64-bluefield -Db_sanitize=address
Then ran the check:
$ ./devtools/test-meson-builds.sh
...
...
...
## Building cmdline
/usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam' is
dangerous, better use `mkstemp'
/usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tempnam' is
dangerous, better use `mkstemp'
/usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam_r'
is dangerous, better use `mkstemp'
## Building helloworld
/usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam' is
dangerous, better use `mkstemp'
/usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tempnam' is
dangerous, better use `mkstemp'
/usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam_r'
is dangerous, better use `mkstemp'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax_iova_table.c.o):
in function `read_memory_node':
/home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:57:
undefined reference to `__asan_option_detect_stack_use_after_return'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:65:
undefined reference to `__asan_report_store4'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:61:
undefined reference to `__asan_report_store_n'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:60:
undefined reference to `__asan_report_store_n'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:57:
undefined reference to `__asan_stack_malloc_3'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:77:
undefined reference to `__asan_report_load8'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:86:
undefined reference to `__asan_report_load8'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:86:
undefined reference to `__asan_report_load8'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:88:
undefined reference to `__asan_report_load8'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:88:
undefined reference to `__asan_report_load8'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax_iova_table.c.o):/home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:102:
more undefined references to `__asan_report_load8' follow
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax_iova_table.c.o):
in function `read_memory_node':
/home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:119:
undefined reference to `__asan_report_store4'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:145:
undefined reference to `__asan_report_load16'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:135:
undefined reference to `__asan_report_store_n'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:143:
undefined reference to `__asan_report_load8'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax_iova_table.c.o):
in function `rotate_8':
/home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:40:
undefined reference to `__asan_report_store4'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:39:
undefined reference to `__asan_report_store4'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:40:
undefined reference to `__asan_report_store4'
/usr/bin/ld: /home/dmarchan/builds/build-x86-generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax_iova_table.c.o):
in function `read_memory_node':
/home/dmarchan/builds/build-x86-generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:143:
undefined reference to `__asan_report_load8'
*flood of errors about undefined symbols from libasan*
Tried to debug, in-tree compiled testpmd shows:
$ readelf -d $HOME/builds/build-x86-generic/app/dpdk-testpmd | grep asan
0x0000000000000001 (NEEDED) Shared library: [libasan.so.6]
But I see nothing in ninja:
$ grep -i asan $HOME/builds/build-x86-generic/build.ninja
And for external link, looking at an "installed" .pc:
$ PKG_CONFIG_PATH=/home/dmarchan/builds/build-x86-generic/install/usr/local/lib/pkgconfig
pkg-config libdpdk --libs | grep asan
Some more comments about splitting doc with patch2:
> ---
> v7: 1) Split doc and code into two.
> 2) Modify asan.rst doc
> v8: No change.
> v9: 1) Add the check of libasan library.
> 2) Add release notes.
> ---
> config/meson.build | 10 +++
> devtools/words-case.txt | 1 +
> doc/guides/prog_guide/asan.rst | 97 ++++++++++++++++++++++++++
> doc/guides/prog_guide/index.rst | 1 +
> doc/guides/rel_notes/release_21_11.rst | 9 +++
> 5 files changed, 118 insertions(+)
> create mode 100644 doc/guides/prog_guide/asan.rst
>
> diff --git a/config/meson.build b/config/meson.build
> index 4cdf589e20..5170b79fed 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -411,6 +411,16 @@ if get_option('b_lto')
> endif
> endif
>
> +if get_option('b_sanitize') == 'address'
> + if cc.get_id() == 'gcc'
> + asan_dep = cc.find_library('asan', required: true)
> + if (not cc.links('int main(int argc, char *argv[]) { return 0; }',
> + dependencies: asan_dep))
> + error('broken dependency, "libasan"')
> + endif
> + endif
> +endif
> +
> if get_option('default_library') == 'both'
> error( '''
> Unsupported value "both" for "default_library" option.
> diff --git a/devtools/words-case.txt b/devtools/words-case.txt
> index 0bbad48626..ada6910fa0 100644
> --- a/devtools/words-case.txt
> +++ b/devtools/words-case.txt
> @@ -5,6 +5,7 @@ API
> Arm
> armv7
> armv8
> +ASan
> BAR
> CRC
> DCB
> diff --git a/doc/guides/prog_guide/asan.rst b/doc/guides/prog_guide/asan.rst
> new file mode 100644
> index 0000000000..627675ef46
> --- /dev/null
> +++ b/doc/guides/prog_guide/asan.rst
> @@ -0,0 +1,97 @@
> +.. Copyright (c) <2021>, Intel Corporation
> + All rights reserved.
> +
> +Memory error detect standard tool - AddressSanitizer(ASan)
> +==========================================================
> +
> +`AddressSanitizer
> +<https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> +is a widely-used debugging tool to detect memory access errors.
> +It helps detect issues like use-after-free, various kinds of buffer
> +overruns in C/C++ programs, and other similar errors, as well as
> +printing out detailed debug information whenever an error is detected.
> +
> +AddressSanitizer is a part of LLVM (3.1+) and GCC (4.8+).
Same problem, below should be in patch 2, from here...
> +
> +DPDK ASan functionality is currently only supported Linux x86_64.
> +Support other platforms, need to define ASAN_SHADOW_OFFSET value
> +according to google ASan document, and configure meson
> +(config/meson.build).
> +
> +Example heap-buffer-overflow error
> +----------------------------------
> +
> +Following error was reported when ASan was enabled::
> +
> + Applied 9 bytes of memory, but accessed the 10th byte of memory,
> + so heap-buffer-overflow appeared.
> +
> +Below code results in this error::
> +
> + Add code to helloworld:
> + char *p = rte_zmalloc(NULL, 9, 0);
> + if (!p) {
> + printf("rte_zmalloc error.");
> + return -1;
> + }
> + p[9] = 'a';
> +
> +The error log::
> +
> + ==369953==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fb17f465809 at pc 0x5652e6707b84 bp 0x7ffea70eea20 sp 0x7ffea70eea10 WRITE of size 1 at 0x7fb17f465809 thread T0
> + #0 0x5652e6707b83 in main ../examples/helloworld/main.c:47
> + #1 0x7fb94953c0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
> + #2 0x5652e67079bd in _start (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-helloworld+0x8329bd)
> +
> + Address 0x7fb17f465809 is a wild pointer.
> + SUMMARY: AddressSanitizer: heap-buffer-overflow ../examples/helloworld/main.c:47 in main
> +
> +Example use-after-free error
> +----------------------------
> +
> +Following error was reported when ASan was enabled::
> +
> + Applied for 9 bytes of memory, and accessed the first byte after
> + released, so heap-use-after-free appeared.
> +
> +Below code results in this error::
> +
> + Add code to helloworld:
> + char *p = rte_zmalloc(NULL, 9, 0);
> + if (!p) {
> + printf("rte_zmalloc error.");
> + return -1;
> + }
> + rte_free(p);
> + *p = 'a';
> +
> +The error log::
> +
> + ==417048==ERROR: AddressSanitizer: heap-use-after-free on address 0x7fc83f465800 at pc 0x564308a39b89 bp 0x7ffc8c85bf50 sp 0x7ffc8c85bf40 WRITE of size 1 at 0x7fc83f465800 thread T0
> + #0 0x564308a39b88 in main ../examples/helloworld/main.c:48
> + #1 0x7fd0079c60b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
> + #2 0x564308a399bd in _start (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-helloworld+0x8329bd)
> +
> + Address 0x7fc83f465800 is a wild pointer.
> + SUMMARY: AddressSanitizer: heap-use-after-free ../examples/helloworld/main.c:48 in main
... till here.
> +
> +Usage
> +-----
> +
> +meson build
> +^^^^^^^^^^^
> +
> +To enable ASan in meson build system, use following meson build command:
> +
> +Example usage::
> +
> + meson build -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
l_undef unnecessary for gcc.
> + ninja -C build
> +
> +.. Note::
> +
> + a) Some of the features of ASan (for example, 'Display memory application location, currently
> + displayed as a wild pointer') are not currently supported by DPDK's implementation.
This comment should be in patch 2, there is nothing specific in patch
1 about DPDK wrt ASan.
> + b) DPDK test has been completed in ubuntu18.04/ubuntu20.04/redhat8.3. To compile with gcc in
> + centos, libasan needs to be installed separately.
> + c) If the program uses cmdline, when a memory bug occurs, need to execute the "stty echo" command.
> diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
> index 2dce507f46..df8a4b93e1 100644
> --- a/doc/guides/prog_guide/index.rst
> +++ b/doc/guides/prog_guide/index.rst
> @@ -71,3 +71,4 @@ Programmer's Guide
> lto
> profile_app
> glossary
> + asan
> diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
> index 5036641842..58c6377148 100644
> --- a/doc/guides/rel_notes/release_21_11.rst
> +++ b/doc/guides/rel_notes/release_21_11.rst
> @@ -141,6 +141,15 @@ New Features
> * Added tests to validate packets hard expiry.
> * Added tests to verify tunnel header verification in IPsec inbound.
>
> +* **Enable ASan for memory detector on DPDK.**
> +
> + `AddressSanitizer
> + <https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> + is a widely-used debugging tool to detect memory access errors.
> + It helps detect issues like use-after-free, various kinds of buffer
> + overruns in C/C++ programs, and other similar errors, as well as
> + printing out detailed debug information whenever an error is detected.
> +
>
> Removed Items
> -------------
> --
> 2.25.1
>
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Thursday, October 14, 2021 12:45 AM
> To: Peng, ZhihongX <zhihongx.peng@intel.com>
> Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Stephen Hemminger
> <stephen@networkplumber.org>; dev <dev@dpdk.org>; Lin, Xueqin
> <xueqin.lin@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> Thomas Monjalon <thomas@monjalon.net>
> Subject: Re: [PATCH v9 1/3] Enable ASan for memory detector on DPDK
>
> On Tue, Oct 12, 2021 at 11:54 AM <zhihongx.peng@intel.com> wrote:
> >
> > From: Zhihong Peng <zhihongx.peng@intel.com>
> >
> > `AddressSanitizer
> > <https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> > is a widely-used debugging tool to detect memory access errors.
> > It helps detect issues like use-after-free, various kinds of buffer
> > overruns in C/C++ programs, and other similar errors, as well as
> > printing out detailed debug information whenever an error is detected.
> >
>
>
> Comments below concern added feature with patch 2.
> From here...
>
> > DPDK ASan functionality is currently only supported Linux x86_64.
> > Support other platforms, need to define ASAN_SHADOW_OFFSET value
> > according to google ASan document, and configure meson
> > (config/meson.build).
> >
> > Here is an example of heap-buffer-overflow bug:
> > ......
> > char *p = rte_zmalloc(NULL, 7, 0);
> > p[7] = 'a';
> > ......
> >
> > Here is an example of use-after-free bug:
> > ......
> > char *p = rte_zmalloc(NULL, 7, 0);
> > rte_free(p);
> > *p = 'a';
> > ......
> ... to here.
>
>
> >
> > We can enable ASan by adding below compilation options:
> > -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
> > "-Dbuildtype=debug": This is a non-essential option. When this option
> > is added, if a memory error occurs, ASan can clearly show where the
> > code is wrong.
> > "-Db_lundef=false": When use clang to compile DPDK, this option must
> > be added.
> >
> > Signed-off-by: Xueqin Lin <xueqin.lin@intel.com>
> > Signed-off-by: Zhihong Peng <zhihongx.peng@intel.com>
>
>
> More problematic, linking an external (out of meson) application to a dpdk
> compiled with ASan is broken.
>
> My environment contains following targets compiled using
> ./devtools/test-meson-builds.sh:
> $ ls $HOME/builds/
> build-arm64-bluefield build-arm64-host-clang build-clang-shared build-gcc-
> shared build-ppc64le-power8 build-x86-mingw
> build-arm64-dpaa build-arm64-octeontx2 build-clang-static
> build-gcc-static build-x86-generic
>
> I stopped at patch 1, configured following target to have ASan in them, like
> this:
>
> $ meson configure $HOME/builds/build-gcc-static -Db_sanitize=address
> $ meson configure $HOME/builds/build-clang-shared -Db_sanitize=address -
> Db_lundef=false $ meson configure $HOME/builds/build-x86-generic -
> Db_sanitize=address
> ^^^^^^^^^^^^^^^^^
> This is the target for which we test linking a dpdk application
> out of meson.
>
> $ meson configure $HOME/builds/build-arm64-bluefield -
> Db_sanitize=address
I don't know your test platform , arm or x86.
This is our compilation command:
Gcc is 9.3.0 or 10.3.0
CC=gcc meson -Db_sanitize=address x86_64-native-linuxapp-gcc
ninja -C x86_64-native-linuxapp-gcc
meson configure -Dexamples=helloworld x86_64-native-linuxapp-gcc
ninja -C x86_64-native-linuxapp-gcc
I don’t know how you get this parameter $HOME/builds/build-x86-generic.
Can you send me all your configuration, I will reproduce this error in our environment.
Thank you very much for your help!
> Then ran the check:
> $ ./devtools/test-meson-builds.sh
> ...
> ...
> ...
> ## Building cmdline
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tempnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam_r'
> is dangerous, better use `mkstemp'
> ## Building helloworld
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tempnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam_r'
> is dangerous, better use `mkstemp'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `read_memory_node':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:57:
> undefined reference to `__asan_option_detect_stack_use_after_return'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:65:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:61:
> undefined reference to `__asan_report_store_n'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:60:
> undefined reference to `__asan_report_store_n'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:57:
> undefined reference to `__asan_stack_malloc_3'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:77:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:86:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:86:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:88:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:88:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):/home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:102:
> more undefined references to `__asan_report_load8' follow
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `read_memory_node':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:119:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:145:
> undefined reference to `__asan_report_load16'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:135:
> undefined reference to `__asan_report_store_n'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:143:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `rotate_8':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:40:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:39:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:40:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `read_memory_node':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:143:
> undefined reference to `__asan_report_load8'
> *flood of errors about undefined symbols from libasan*
>
>
> Tried to debug, in-tree compiled testpmd shows:
> $ readelf -d $HOME/builds/build-x86-generic/app/dpdk-testpmd | grep asan
> 0x0000000000000001 (NEEDED) Shared library: [libasan.so.6]
> But I see nothing in ninja:
> $ grep -i asan $HOME/builds/build-x86-generic/build.ninja
>
> And for external link, looking at an "installed" .pc:
> $ PKG_CONFIG_PATH=/home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/pkgconfig
> pkg-config libdpdk --libs | grep asan
>
>
>
> Some more comments about splitting doc with patch2:
>
>
> > ---
> > v7: 1) Split doc and code into two.
> > 2) Modify asan.rst doc
> > v8: No change.
> > v9: 1) Add the check of libasan library.
> > 2) Add release notes.
> > ---
> > config/meson.build | 10 +++
> > devtools/words-case.txt | 1 +
> > doc/guides/prog_guide/asan.rst | 97
> ++++++++++++++++++++++++++
> > doc/guides/prog_guide/index.rst | 1 +
> > doc/guides/rel_notes/release_21_11.rst | 9 +++
> > 5 files changed, 118 insertions(+)
> > create mode 100644 doc/guides/prog_guide/asan.rst
> >
> > diff --git a/config/meson.build b/config/meson.build index
> > 4cdf589e20..5170b79fed 100644
> > --- a/config/meson.build
> > +++ b/config/meson.build
> > @@ -411,6 +411,16 @@ if get_option('b_lto')
> > endif
> > endif
> >
> > +if get_option('b_sanitize') == 'address'
> > + if cc.get_id() == 'gcc'
> > + asan_dep = cc.find_library('asan', required: true)
> > + if (not cc.links('int main(int argc, char *argv[]) { return 0; }',
> > + dependencies: asan_dep))
> > + error('broken dependency, "libasan"')
> > + endif
> > + endif
> > +endif
> > +
> > if get_option('default_library') == 'both'
> > error( '''
> > Unsupported value "both" for "default_library" option.
> > diff --git a/devtools/words-case.txt b/devtools/words-case.txt index
> > 0bbad48626..ada6910fa0 100644
> > --- a/devtools/words-case.txt
> > +++ b/devtools/words-case.txt
> > @@ -5,6 +5,7 @@ API
> > Arm
> > armv7
> > armv8
> > +ASan
> > BAR
> > CRC
> > DCB
> > diff --git a/doc/guides/prog_guide/asan.rst
> > b/doc/guides/prog_guide/asan.rst new file mode 100644 index
> > 0000000000..627675ef46
> > --- /dev/null
> > +++ b/doc/guides/prog_guide/asan.rst
> > @@ -0,0 +1,97 @@
> > +.. Copyright (c) <2021>, Intel Corporation
> > + All rights reserved.
> > +
> > +Memory error detect standard tool - AddressSanitizer(ASan)
> >
> +=========================================================
> =
> > +
> > +`AddressSanitizer
> > +<https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> > +is a widely-used debugging tool to detect memory access errors.
> > +It helps detect issues like use-after-free, various kinds of buffer
> > +overruns in C/C++ programs, and other similar errors, as well as
> > +printing out detailed debug information whenever an error is detected.
> > +
> > +AddressSanitizer is a part of LLVM (3.1+) and GCC (4.8+).
>
> Same problem, below should be in patch 2, from here...
>
>
> > +
> > +DPDK ASan functionality is currently only supported Linux x86_64.
> > +Support other platforms, need to define ASAN_SHADOW_OFFSET value
> > +according to google ASan document, and configure meson
> > +(config/meson.build).
> > +
> > +Example heap-buffer-overflow error
> > +----------------------------------
> > +
> > +Following error was reported when ASan was enabled::
> > +
> > + Applied 9 bytes of memory, but accessed the 10th byte of memory,
> > + so heap-buffer-overflow appeared.
> > +
> > +Below code results in this error::
> > +
> > + Add code to helloworld:
> > + char *p = rte_zmalloc(NULL, 9, 0);
> > + if (!p) {
> > + printf("rte_zmalloc error.");
> > + return -1;
> > + }
> > + p[9] = 'a';
> > +
> > +The error log::
> > +
> > + ==369953==ERROR: AddressSanitizer: heap-buffer-overflow on address
> 0x7fb17f465809 at pc 0x5652e6707b84 bp 0x7ffea70eea20 sp 0x7ffea70eea10
> WRITE of size 1 at 0x7fb17f465809 thread T0
> > + #0 0x5652e6707b83 in main ../examples/helloworld/main.c:47
> > + #1 0x7fb94953c0b2 in __libc_start_main (/lib/x86_64-linux-
> gnu/libc.so.6+0x270b2)
> > + #2 0x5652e67079bd in _start
> > + (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-
> hellow
> > + orld+0x8329bd)
> > +
> > + Address 0x7fb17f465809 is a wild pointer.
> > + SUMMARY: AddressSanitizer: heap-buffer-overflow
> > + ../examples/helloworld/main.c:47 in main
> > +
> > +Example use-after-free error
> > +----------------------------
> > +
> > +Following error was reported when ASan was enabled::
> > +
> > + Applied for 9 bytes of memory, and accessed the first byte after
> > + released, so heap-use-after-free appeared.
> > +
> > +Below code results in this error::
> > +
> > + Add code to helloworld:
> > + char *p = rte_zmalloc(NULL, 9, 0);
> > + if (!p) {
> > + printf("rte_zmalloc error.");
> > + return -1;
> > + }
> > + rte_free(p);
> > + *p = 'a';
> > +
> > +The error log::
> > +
> > + ==417048==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x7fc83f465800 at pc 0x564308a39b89 bp 0x7ffc8c85bf50 sp 0x7ffc8c85bf40
> WRITE of size 1 at 0x7fc83f465800 thread T0
> > + #0 0x564308a39b88 in main ../examples/helloworld/main.c:48
> > + #1 0x7fd0079c60b2 in __libc_start_main (/lib/x86_64-linux-
> gnu/libc.so.6+0x270b2)
> > + #2 0x564308a399bd in _start
> > + (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-
> hellow
> > + orld+0x8329bd)
> > +
> > + Address 0x7fc83f465800 is a wild pointer.
> > + SUMMARY: AddressSanitizer: heap-use-after-free
> > + ../examples/helloworld/main.c:48 in main
>
> ... till here.
>
>
> > +
> > +Usage
> > +-----
> > +
> > +meson build
> > +^^^^^^^^^^^
> > +
> > +To enable ASan in meson build system, use following meson build
> command:
> > +
> > +Example usage::
> > +
> > + meson build -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
>
> l_undef unnecessary for gcc.
>
>
> > + ninja -C build
> > +
> > +.. Note::
> > +
> > + a) Some of the features of ASan (for example, 'Display memory
> application location, currently
> > + displayed as a wild pointer') are not currently supported by DPDK's
> implementation.
>
> This comment should be in patch 2, there is nothing specific in patch
> 1 about DPDK wrt ASan.
>
>
> > + b) DPDK test has been completed in
> ubuntu18.04/ubuntu20.04/redhat8.3. To compile with gcc in
> > + centos, libasan needs to be installed separately.
> > + c) If the program uses cmdline, when a memory bug occurs, need to
> execute the "stty echo" command.
> > diff --git a/doc/guides/prog_guide/index.rst
> > b/doc/guides/prog_guide/index.rst index 2dce507f46..df8a4b93e1 100644
> > --- a/doc/guides/prog_guide/index.rst
> > +++ b/doc/guides/prog_guide/index.rst
> > @@ -71,3 +71,4 @@ Programmer's Guide
> > lto
> > profile_app
> > glossary
> > + asan
> > diff --git a/doc/guides/rel_notes/release_21_11.rst
> > b/doc/guides/rel_notes/release_21_11.rst
> > index 5036641842..58c6377148 100644
> > --- a/doc/guides/rel_notes/release_21_11.rst
> > +++ b/doc/guides/rel_notes/release_21_11.rst
> > @@ -141,6 +141,15 @@ New Features
> > * Added tests to validate packets hard expiry.
> > * Added tests to verify tunnel header verification in IPsec inbound.
> >
> > +* **Enable ASan for memory detector on DPDK.**
> > +
> > + `AddressSanitizer
> > + <https://github.com/google/sanitizers/wiki/AddressSanitizer>`
> > + (ASan) is a widely-used debugging tool to detect memory access errors.
> > + It helps detect issues like use-after-free, various kinds of buffer
> > + overruns in C/C++ programs, and other similar errors, as well as
> > + printing out detailed debug information whenever an error is detected.
> > +
> >
> > Removed Items
> > -------------
> > --
> > 2.25.1
> >
>
>
> --
> David Marchand
14/10/2021 08:46, Peng, ZhihongX:
> From: David Marchand <david.marchand@redhat.com>
> > More problematic, linking an external (out of meson) application to a dpdk
> > compiled with ASan is broken.
> >
> > My environment contains following targets compiled using
> > ./devtools/test-meson-builds.sh:
> > $ ls $HOME/builds/
> > build-arm64-bluefield build-arm64-host-clang build-clang-shared build-gcc-
> > shared build-ppc64le-power8 build-x86-mingw
> > build-arm64-dpaa build-arm64-octeontx2 build-clang-static
> > build-gcc-static build-x86-generic
> >
> > I stopped at patch 1, configured following target to have ASan in them, like
> > this:
> >
> > $ meson configure $HOME/builds/build-gcc-static -Db_sanitize=address
> > $ meson configure $HOME/builds/build-clang-shared -Db_sanitize=address -
> > Db_lundef=false $ meson configure $HOME/builds/build-x86-generic -
> > Db_sanitize=address
> > ^^^^^^^^^^^^^^^^^
> > This is the target for which we test linking a dpdk application
> > out of meson.
> >
> > $ meson configure $HOME/builds/build-arm64-bluefield -
> > Db_sanitize=address
>
> I don't know your test platform , arm or x86.
>
> This is our compilation command:
> Gcc is 9.3.0 or 10.3.0
> CC=gcc meson -Db_sanitize=address x86_64-native-linuxapp-gcc
> ninja -C x86_64-native-linuxapp-gcc
> meson configure -Dexamples=helloworld x86_64-native-linuxapp-gcc
> ninja -C x86_64-native-linuxapp-gcc
>
> I don’t know how you get this parameter $HOME/builds/build-x86-generic.
>
> Can you send me all your configuration, I will reproduce this error in our environment.
This is written just below:
> Thank you very much for your help!
>
> > Then ran the check:
> > $ ./devtools/test-meson-builds.sh
Here, this is the command recommended to run in the contributing guide.
https://doc.dpdk.org/guides/contributing/patches.html#checking-compilation
You need, at the very minimum, to install an Arm cross-compiler.
As you are changing stuff in the compilation of the project,
please become familiar with testing compilation in multiple environments.
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Thursday, October 14, 2021 12:45 AM
> To: Peng, ZhihongX <zhihongx.peng@intel.com>
> Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Stephen Hemminger
> <stephen@networkplumber.org>; dev <dev@dpdk.org>; Lin, Xueqin
> <xueqin.lin@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> Thomas Monjalon <thomas@monjalon.net>
> Subject: Re: [PATCH v9 1/3] Enable ASan for memory detector on DPDK
>
> On Tue, Oct 12, 2021 at 11:54 AM <zhihongx.peng@intel.com> wrote:
> >
> > From: Zhihong Peng <zhihongx.peng@intel.com>
> >
> > `AddressSanitizer
> > <https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> > is a widely-used debugging tool to detect memory access errors.
> > It helps detect issues like use-after-free, various kinds of buffer
> > overruns in C/C++ programs, and other similar errors, as well as
> > printing out detailed debug information whenever an error is detected.
> >
>
>
> Comments below concern added feature with patch 2.
> From here...
The v10 version will be fixed.
> > DPDK ASan functionality is currently only supported Linux x86_64.
> > Support other platforms, need to define ASAN_SHADOW_OFFSET value
> > according to google ASan document, and configure meson
> > (config/meson.build).
> >
> > Here is an example of heap-buffer-overflow bug:
> > ......
> > char *p = rte_zmalloc(NULL, 7, 0);
> > p[7] = 'a';
> > ......
> >
> > Here is an example of use-after-free bug:
> > ......
> > char *p = rte_zmalloc(NULL, 7, 0);
> > rte_free(p);
> > *p = 'a';
> > ......
> ... to here.
>
>
> >
> > We can enable ASan by adding below compilation options:
> > -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
> > "-Dbuildtype=debug": This is a non-essential option. When this option
> > is added, if a memory error occurs, ASan can clearly show where the
> > code is wrong.
> > "-Db_lundef=false": When use clang to compile DPDK, this option must
> > be added.
> >
> > Signed-off-by: Xueqin Lin <xueqin.lin@intel.com>
> > Signed-off-by: Zhihong Peng <zhihongx.peng@intel.com>
>
>
> More problematic, linking an external (out of meson) application to a dpdk
> compiled with ASan is broken.
>
> My environment contains following targets compiled using
> ./devtools/test-meson-builds.sh:
> $ ls $HOME/builds/
> build-arm64-bluefield build-arm64-host-clang build-clang-shared build-gcc-
> shared build-ppc64le-power8 build-x86-mingw
> build-arm64-dpaa build-arm64-octeontx2 build-clang-static
> build-gcc-static build-x86-generic
>
> I stopped at patch 1, configured following target to have ASan in them, like
> this:
>
> $ meson configure $HOME/builds/build-gcc-static -Db_sanitize=address
> $ meson configure $HOME/builds/build-clang-shared -Db_sanitize=address -
> Db_lundef=false $ meson configure $HOME/builds/build-x86-generic -
> Db_sanitize=address
> ^^^^^^^^^^^^^^^^^
> This is the target for which we test linking a dpdk application
> out of meson.
>
> $ meson configure $HOME/builds/build-arm64-bluefield -
> Db_sanitize=address
>
> Then ran the check:
> $ ./devtools/test-meson-builds.sh
> ...
> ...
> ...
> ## Building cmdline
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tempnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam_r'
> is dangerous, better use `mkstemp'
> ## Building helloworld
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tempnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam_r'
> is dangerous, better use `mkstemp'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `read_memory_node':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:57:
> undefined reference to `__asan_option_detect_stack_use_after_return'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:65:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:61:
> undefined reference to `__asan_report_store_n'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:60:
> undefined reference to `__asan_report_store_n'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:57:
> undefined reference to `__asan_stack_malloc_3'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:77:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:86:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:86:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:88:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:88:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):/home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:102:
> more undefined references to `__asan_report_load8' follow
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `read_memory_node':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:119:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:145:
> undefined reference to `__asan_report_load16'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:135:
> undefined reference to `__asan_report_store_n'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:143:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `rotate_8':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:40:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:39:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:40:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `read_memory_node':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:143:
> undefined reference to `__asan_report_load8'
> *flood of errors about undefined symbols from libasan*
>
>
> Tried to debug, in-tree compiled testpmd shows:
> $ readelf -d $HOME/builds/build-x86-generic/app/dpdk-testpmd | grep asan
> 0x0000000000000001 (NEEDED) Shared library: [libasan.so.6]
> But I see nothing in ninja:
> $ grep -i asan $HOME/builds/build-x86-generic/build.ninja
>
> And for external link, looking at an "installed" .pc:
> $ PKG_CONFIG_PATH=/home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/pkgconfig
> pkg-config libdpdk --libs | grep asan
Need to link -lasan when using gcc, but don’t need to link when using clang.
You added it in the v6 version comments, but need to restrict when using clang.
All code:
if get_option('b_sanitize') == 'address' or get_option('b_sanitize') == 'address,undefined'
if cc.get_id() == 'gcc'
asan_dep = cc.find_library('asan', required: true)
if (not cc.links('int main(int argc, char *argv[]) { return 0; }',
dependencies: asan_dep))
error('broken dependency, "libasan"')
endif
add_project_link_arguments('-lasan', language: 'c')
dpdk_extra_ldflags += '-lasan'
endif
if exec_env == 'linux' and arch_subdir == 'x86'
dpdk_conf.set10('RTE_MALLOC_ASAN', true)
endif
endif
>
>
> Some more comments about splitting doc with patch2:
>
>
> > ---
> > v7: 1) Split doc and code into two.
> > 2) Modify asan.rst doc
> > v8: No change.
> > v9: 1) Add the check of libasan library.
> > 2) Add release notes.
> > ---
> > config/meson.build | 10 +++
> > devtools/words-case.txt | 1 +
> > doc/guides/prog_guide/asan.rst | 97
> ++++++++++++++++++++++++++
> > doc/guides/prog_guide/index.rst | 1 +
> > doc/guides/rel_notes/release_21_11.rst | 9 +++
> > 5 files changed, 118 insertions(+)
> > create mode 100644 doc/guides/prog_guide/asan.rst
> >
> > diff --git a/config/meson.build b/config/meson.build index
> > 4cdf589e20..5170b79fed 100644
> > --- a/config/meson.build
> > +++ b/config/meson.build
> > @@ -411,6 +411,16 @@ if get_option('b_lto')
> > endif
> > endif
> >
> > +if get_option('b_sanitize') == 'address'
> > + if cc.get_id() == 'gcc'
> > + asan_dep = cc.find_library('asan', required: true)
> > + if (not cc.links('int main(int argc, char *argv[]) { return 0; }',
> > + dependencies: asan_dep))
> > + error('broken dependency, "libasan"')
> > + endif
> > + endif
> > +endif
> > +
> > if get_option('default_library') == 'both'
> > error( '''
> > Unsupported value "both" for "default_library" option.
> > diff --git a/devtools/words-case.txt b/devtools/words-case.txt index
> > 0bbad48626..ada6910fa0 100644
> > --- a/devtools/words-case.txt
> > +++ b/devtools/words-case.txt
> > @@ -5,6 +5,7 @@ API
> > Arm
> > armv7
> > armv8
> > +ASan
> > BAR
> > CRC
> > DCB
> > diff --git a/doc/guides/prog_guide/asan.rst
> > b/doc/guides/prog_guide/asan.rst new file mode 100644 index
> > 0000000000..627675ef46
> > --- /dev/null
> > +++ b/doc/guides/prog_guide/asan.rst
> > @@ -0,0 +1,97 @@
> > +.. Copyright (c) <2021>, Intel Corporation
> > + All rights reserved.
> > +
> > +Memory error detect standard tool - AddressSanitizer(ASan)
> >
> +=========================================================
> =
> > +
> > +`AddressSanitizer
> > +<https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> > +is a widely-used debugging tool to detect memory access errors.
> > +It helps detect issues like use-after-free, various kinds of buffer
> > +overruns in C/C++ programs, and other similar errors, as well as
> > +printing out detailed debug information whenever an error is detected.
> > +
> > +AddressSanitizer is a part of LLVM (3.1+) and GCC (4.8+).
>
> Same problem, below should be in patch 2, from here...
The v10 version will be fixed.
>
> > +
> > +DPDK ASan functionality is currently only supported Linux x86_64.
> > +Support other platforms, need to define ASAN_SHADOW_OFFSET value
> > +according to google ASan document, and configure meson
> > +(config/meson.build).
> > +
> > +Example heap-buffer-overflow error
> > +----------------------------------
> > +
> > +Following error was reported when ASan was enabled::
> > +
> > + Applied 9 bytes of memory, but accessed the 10th byte of memory,
> > + so heap-buffer-overflow appeared.
> > +
> > +Below code results in this error::
> > +
> > + Add code to helloworld:
> > + char *p = rte_zmalloc(NULL, 9, 0);
> > + if (!p) {
> > + printf("rte_zmalloc error.");
> > + return -1;
> > + }
> > + p[9] = 'a';
> > +
> > +The error log::
> > +
> > + ==369953==ERROR: AddressSanitizer: heap-buffer-overflow on address
> 0x7fb17f465809 at pc 0x5652e6707b84 bp 0x7ffea70eea20 sp 0x7ffea70eea10
> WRITE of size 1 at 0x7fb17f465809 thread T0
> > + #0 0x5652e6707b83 in main ../examples/helloworld/main.c:47
> > + #1 0x7fb94953c0b2 in __libc_start_main (/lib/x86_64-linux-
> gnu/libc.so.6+0x270b2)
> > + #2 0x5652e67079bd in _start
> > + (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-
> hellow
> > + orld+0x8329bd)
> > +
> > + Address 0x7fb17f465809 is a wild pointer.
> > + SUMMARY: AddressSanitizer: heap-buffer-overflow
> > + ../examples/helloworld/main.c:47 in main
> > +
> > +Example use-after-free error
> > +----------------------------
> > +
> > +Following error was reported when ASan was enabled::
> > +
> > + Applied for 9 bytes of memory, and accessed the first byte after
> > + released, so heap-use-after-free appeared.
> > +
> > +Below code results in this error::
> > +
> > + Add code to helloworld:
> > + char *p = rte_zmalloc(NULL, 9, 0);
> > + if (!p) {
> > + printf("rte_zmalloc error.");
> > + return -1;
> > + }
> > + rte_free(p);
> > + *p = 'a';
> > +
> > +The error log::
> > +
> > + ==417048==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x7fc83f465800 at pc 0x564308a39b89 bp 0x7ffc8c85bf50 sp 0x7ffc8c85bf40
> WRITE of size 1 at 0x7fc83f465800 thread T0
> > + #0 0x564308a39b88 in main ../examples/helloworld/main.c:48
> > + #1 0x7fd0079c60b2 in __libc_start_main (/lib/x86_64-linux-
> gnu/libc.so.6+0x270b2)
> > + #2 0x564308a399bd in _start
> > + (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-
> hellow
> > + orld+0x8329bd)
> > +
> > + Address 0x7fc83f465800 is a wild pointer.
> > + SUMMARY: AddressSanitizer: heap-use-after-free
> > + ../examples/helloworld/main.c:48 in main
>
> ... till here.
>
>
> > +
> > +Usage
> > +-----
> > +
> > +meson build
> > +^^^^^^^^^^^
> > +
> > +To enable ASan in meson build system, use following meson build
> command:
> > +
> > +Example usage::
> > +
> > + meson build -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
>
> l_undef unnecessary for gcc.
The v10 version will be fixed.
>
> > + ninja -C build
> > +
> > +.. Note::
> > +
> > + a) Some of the features of ASan (for example, 'Display memory
> application location, currently
> > + displayed as a wild pointer') are not currently supported by DPDK's
> implementation.
>
> This comment should be in patch 2, there is nothing specific in patch
> 1 about DPDK wrt ASan.
The v10 version will be fixed.
>
> > + b) DPDK test has been completed in
> ubuntu18.04/ubuntu20.04/redhat8.3. To compile with gcc in
> > + centos, libasan needs to be installed separately.
> > + c) If the program uses cmdline, when a memory bug occurs, need to
> execute the "stty echo" command.
> > diff --git a/doc/guides/prog_guide/index.rst
> > b/doc/guides/prog_guide/index.rst index 2dce507f46..df8a4b93e1 100644
> > --- a/doc/guides/prog_guide/index.rst
> > +++ b/doc/guides/prog_guide/index.rst
> > @@ -71,3 +71,4 @@ Programmer's Guide
> > lto
> > profile_app
> > glossary
> > + asan
> > diff --git a/doc/guides/rel_notes/release_21_11.rst
> > b/doc/guides/rel_notes/release_21_11.rst
> > index 5036641842..58c6377148 100644
> > --- a/doc/guides/rel_notes/release_21_11.rst
> > +++ b/doc/guides/rel_notes/release_21_11.rst
> > @@ -141,6 +141,15 @@ New Features
> > * Added tests to validate packets hard expiry.
> > * Added tests to verify tunnel header verification in IPsec inbound.
> >
> > +* **Enable ASan for memory detector on DPDK.**
> > +
> > + `AddressSanitizer
> > + <https://github.com/google/sanitizers/wiki/AddressSanitizer>`
> > + (ASan) is a widely-used debugging tool to detect memory access errors.
> > + It helps detect issues like use-after-free, various kinds of buffer
> > + overruns in C/C++ programs, and other similar errors, as well as
> > + printing out detailed debug information whenever an error is detected.
> > +
> >
> > Removed Items
> > -------------
> > --
> > 2.25.1
> >
>
>
> --
> David Marchand
@@ -411,6 +411,16 @@ if get_option('b_lto')
endif
endif
+if get_option('b_sanitize') == 'address'
+ if cc.get_id() == 'gcc'
+ asan_dep = cc.find_library('asan', required: true)
+ if (not cc.links('int main(int argc, char *argv[]) { return 0; }',
+ dependencies: asan_dep))
+ error('broken dependency, "libasan"')
+ endif
+ endif
+endif
+
if get_option('default_library') == 'both'
error( '''
Unsupported value "both" for "default_library" option.
@@ -5,6 +5,7 @@ API
Arm
armv7
armv8
+ASan
BAR
CRC
DCB
new file mode 100644
@@ -0,0 +1,97 @@
+.. Copyright (c) <2021>, Intel Corporation
+ All rights reserved.
+
+Memory error detect standard tool - AddressSanitizer(ASan)
+==========================================================
+
+`AddressSanitizer
+<https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
+is a widely-used debugging tool to detect memory access errors.
+It helps detect issues like use-after-free, various kinds of buffer
+overruns in C/C++ programs, and other similar errors, as well as
+printing out detailed debug information whenever an error is detected.
+
+AddressSanitizer is a part of LLVM (3.1+) and GCC (4.8+).
+
+DPDK ASan functionality is currently only supported Linux x86_64.
+Support other platforms, need to define ASAN_SHADOW_OFFSET value
+according to google ASan document, and configure meson
+(config/meson.build).
+
+Example heap-buffer-overflow error
+----------------------------------
+
+Following error was reported when ASan was enabled::
+
+ Applied 9 bytes of memory, but accessed the 10th byte of memory,
+ so heap-buffer-overflow appeared.
+
+Below code results in this error::
+
+ Add code to helloworld:
+ char *p = rte_zmalloc(NULL, 9, 0);
+ if (!p) {
+ printf("rte_zmalloc error.");
+ return -1;
+ }
+ p[9] = 'a';
+
+The error log::
+
+ ==369953==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fb17f465809 at pc 0x5652e6707b84 bp 0x7ffea70eea20 sp 0x7ffea70eea10 WRITE of size 1 at 0x7fb17f465809 thread T0
+ #0 0x5652e6707b83 in main ../examples/helloworld/main.c:47
+ #1 0x7fb94953c0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
+ #2 0x5652e67079bd in _start (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-helloworld+0x8329bd)
+
+ Address 0x7fb17f465809 is a wild pointer.
+ SUMMARY: AddressSanitizer: heap-buffer-overflow ../examples/helloworld/main.c:47 in main
+
+Example use-after-free error
+----------------------------
+
+Following error was reported when ASan was enabled::
+
+ Applied for 9 bytes of memory, and accessed the first byte after
+ released, so heap-use-after-free appeared.
+
+Below code results in this error::
+
+ Add code to helloworld:
+ char *p = rte_zmalloc(NULL, 9, 0);
+ if (!p) {
+ printf("rte_zmalloc error.");
+ return -1;
+ }
+ rte_free(p);
+ *p = 'a';
+
+The error log::
+
+ ==417048==ERROR: AddressSanitizer: heap-use-after-free on address 0x7fc83f465800 at pc 0x564308a39b89 bp 0x7ffc8c85bf50 sp 0x7ffc8c85bf40 WRITE of size 1 at 0x7fc83f465800 thread T0
+ #0 0x564308a39b88 in main ../examples/helloworld/main.c:48
+ #1 0x7fd0079c60b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
+ #2 0x564308a399bd in _start (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-helloworld+0x8329bd)
+
+ Address 0x7fc83f465800 is a wild pointer.
+ SUMMARY: AddressSanitizer: heap-use-after-free ../examples/helloworld/main.c:48 in main
+
+Usage
+-----
+
+meson build
+^^^^^^^^^^^
+
+To enable ASan in meson build system, use following meson build command:
+
+Example usage::
+
+ meson build -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
+ ninja -C build
+
+.. Note::
+
+ a) Some of the features of ASan (for example, 'Display memory application location, currently
+ displayed as a wild pointer') are not currently supported by DPDK's implementation.
+ b) DPDK test has been completed in ubuntu18.04/ubuntu20.04/redhat8.3. To compile with gcc in
+ centos, libasan needs to be installed separately.
+ c) If the program uses cmdline, when a memory bug occurs, need to execute the "stty echo" command.
@@ -71,3 +71,4 @@ Programmer's Guide
lto
profile_app
glossary
+ asan
@@ -141,6 +141,15 @@ New Features
* Added tests to validate packets hard expiry.
* Added tests to verify tunnel header verification in IPsec inbound.
+* **Enable ASan for memory detector on DPDK.**
+
+ `AddressSanitizer
+ <https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
+ is a widely-used debugging tool to detect memory access errors.
+ It helps detect issues like use-after-free, various kinds of buffer
+ overruns in C/C++ programs, and other similar errors, as well as
+ printing out detailed debug information whenever an error is detected.
+
Removed Items
-------------