From patchwork Thu May 5 09:29:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 110687 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B41D0A04FF; Thu, 5 May 2022 11:30:23 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CAC3542827; Thu, 5 May 2022 11:30:18 +0200 (CEST) Received: from us-smtp-delivery-74.mimecast.com (us-smtp-delivery-74.mimecast.com [170.10.133.74]) by mails.dpdk.org (Postfix) with ESMTP id A13BC40E5A for ; Thu, 5 May 2022 11:30:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651743016; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EeFPcH+tMPiIQZlBA7ASG/o6s+hqfs4EkR7j3JUxsYI=; b=bk1wc6aGAHZvWwO2OiH1U2brA4oUYVOXQsw0yJwtwYRG8UIh9KW+OLq6ObZG53zea8kOPi aC1xlx7/9P9gcxu+3W2sbZ1XOfGW2PTurQZOg79+zLq+/UzB/cOJjIVEznP/EYwggdZvzK MFcKvfCeHgfjwtO5ZHb+oRDfdgLy+8w= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-568-J2ZN3YkpNAWBlF7kViFT-w-1; Thu, 05 May 2022 05:30:11 -0400 X-MC-Unique: J2ZN3YkpNAWBlF7kViFT-w-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B2615397968C; Thu, 5 May 2022 09:30:10 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.200]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6BCE954F40A; Thu, 5 May 2022 09:30:09 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: john.mcnamara@intel.com, stable@dpdk.org, Anatoly Burakov , Xueqin Lin , Zhihong Peng Subject: [PATCH v2 1/2] test/mem: disable ASan when accessing unallocated mem Date: Thu, 5 May 2022 11:29:51 +0200 Message-Id: <20220505092952.11818-2-david.marchand@redhat.com> In-Reply-To: <20220505092952.11818-1-david.marchand@redhat.com> References: <20220415173127.3838-1-david.marchand@redhat.com> <20220505092952.11818-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org As described in bugzilla, ASan reports accesses to all memory segment as invalid, since those parts have not been allocated. Move __rte_no_asan to rte_common.h and disable ASan on a part of the test. Bugzilla ID: 880 Fixes: 6cc51b1293ce ("mem: instrument allocator for ASan") Cc: stable@dpdk.org Signed-off-by: David Marchand Acked-by: Anatoly Burakov --- app/test/test_memory.c | 5 +++++ lib/eal/common/malloc_elem.h | 10 ++-------- lib/eal/include/rte_common.h | 13 +++++++++++++ 3 files changed, 20 insertions(+), 8 deletions(-) diff --git a/app/test/test_memory.c b/app/test/test_memory.c index 140ac3f3cf..440e5ef838 100644 --- a/app/test/test_memory.c +++ b/app/test/test_memory.c @@ -25,6 +25,11 @@ * - Try to read all memory; it should not segfault. */ +/* + * ASan complains about accessing unallocated memory. + * See: https://bugs.dpdk.org/show_bug.cgi?id=880 + */ +__rte_no_asan static int check_mem(const struct rte_memseg_list *msl __rte_unused, const struct rte_memseg *ms, void *arg __rte_unused) diff --git a/lib/eal/common/malloc_elem.h b/lib/eal/common/malloc_elem.h index c5f65895e1..952ce7343b 100644 --- a/lib/eal/common/malloc_elem.h +++ b/lib/eal/common/malloc_elem.h @@ -7,6 +7,8 @@ #include +#include + #define MIN_DATA_SIZE (RTE_CACHE_LINE_SIZE) /* dummy definition of struct so we can use pointers to it in malloc_elem struct */ @@ -131,12 +133,6 @@ malloc_elem_cookies_ok(const struct malloc_elem *elem) #define ASAN_MEM_TO_SHADOW(mem) \ RTE_PTR_ADD(ASAN_MEM_SHIFT(mem), ASAN_SHADOW_OFFSET) -#if defined(__clang__) -#define __rte_no_asan __attribute__((no_sanitize("address", "hwaddress"))) -#else -#define __rte_no_asan __attribute__((no_sanitize_address)) -#endif - __rte_no_asan static inline void asan_set_shadow(void *addr, char val) @@ -276,8 +272,6 @@ old_malloc_size(struct malloc_elem *elem) #else /* !RTE_MALLOC_ASAN */ -#define __rte_no_asan - static inline void asan_set_zone(void *ptr __rte_unused, size_t len __rte_unused, uint32_t val __rte_unused) { } diff --git a/lib/eal/include/rte_common.h b/lib/eal/include/rte_common.h index 67587025ab..d56a7570c0 100644 --- a/lib/eal/include/rte_common.h +++ b/lib/eal/include/rte_common.h @@ -267,6 +267,19 @@ static void __attribute__((destructor(RTE_PRIO(prio)), used)) func(void) */ #define __rte_cold __attribute__((cold)) +/** + * Disable AddressSanitizer on some code + */ +#ifdef RTE_MALLOC_ASAN +#ifdef RTE_CC_CLANG +#define __rte_no_asan __attribute__((no_sanitize("address", "hwaddress"))) +#else +#define __rte_no_asan __attribute__((no_sanitize_address)) +#endif +#else /* ! RTE_MALLOC_ASAN */ +#define __rte_no_asan +#endif + /*********** Macros for pointer arithmetic ********/ /** From patchwork Thu May 5 09:29:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 110686 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F08C0A04FF; Thu, 5 May 2022 11:30:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E1FDE40E5A; Thu, 5 May 2022 11:30:17 +0200 (CEST) Received: from us-smtp-delivery-74.mimecast.com (us-smtp-delivery-74.mimecast.com [170.10.133.74]) by mails.dpdk.org (Postfix) with ESMTP id 809BC40042 for ; Thu, 5 May 2022 11:30:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651743016; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IjJsoCNnRjfTpOMJRbDX2M0W6cniAOJ00oiLjbl9BFc=; b=iQg1rrVbmIr1B2q6j29Xh7CUa7OVuCEcMpL7lQljW7Zy6D+ThUmLs0L67hyX1WWomlHfsg qKigGwLe8tirsGzi/uPFNeSPGVh5Yv9wgJ0tsW8xgjg392AnQI15nd1fMwyLrR0YjNEVBi hl1AoM17DcfrjfcaYrzH/GjvolMgNEE= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-216-lR_eUl-PNluvrB-d4z-7iw-1; Thu, 05 May 2022 05:30:14 -0400 X-MC-Unique: lR_eUl-PNluvrB-d4z-7iw-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 987DE29ABA37; Thu, 5 May 2022 09:30:14 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.200]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B1C054F3F5; Thu, 5 May 2022 09:30:13 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: john.mcnamara@intel.com, Aaron Conole , Michael Santana Subject: [PATCH v2 2/2] ci: build some job with ASan Date: Thu, 5 May 2022 11:29:52 +0200 Message-Id: <20220505092952.11818-3-david.marchand@redhat.com> In-Reply-To: <20220505092952.11818-1-david.marchand@redhat.com> References: <20220415173127.3838-1-david.marchand@redhat.com> <20220505092952.11818-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Enable ASan, this can greatly help identify leaks and buffer overflows. Running unit tests relying on multiprocess is unreliable with ASan enabled, so skip them. Signed-off-by: David Marchand Acked-by: Aaron Conole --- Changes since v1: - reinstated table_autotest in "ASan-safe" list of ut, --- .ci/linux-build.sh | 8 ++ .github/workflows/build.yml | 3 +- app/test/meson.build | 208 +++++++++++++++++++----------------- 3 files changed, 118 insertions(+), 101 deletions(-) diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 774a1441bf..93706c0131 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -95,6 +95,14 @@ if [ "$MINI" = "true" ]; then OPTS="$OPTS -Denable_drivers=net/null" OPTS="$OPTS -Ddisable_libs=*" fi + +if [ "$ASAN" = "true" ]; then + OPTS="$OPTS -Db_sanitize=address" + if [ "${CC%%clang}" != "$CC" ]; then + OPTS="$OPTS -Db_lundef=false" + fi +fi + meson build --werror $OPTS ninja -C build diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 22daaabb91..45871e76ed 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -16,6 +16,7 @@ jobs: env: AARCH64: ${{ matrix.config.cross == 'aarch64' }} ABI_CHECKS: ${{ contains(matrix.config.checks, 'abi') }} + ASAN: ${{ contains(matrix.config.checks, 'asan') }} BUILD_32BIT: ${{ matrix.config.cross == 'i386' }} BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }} CC: ccache ${{ matrix.config.compiler }} @@ -47,7 +48,7 @@ jobs: - os: ubuntu-18.04 compiler: clang library: shared - checks: doc+tests + checks: asan+doc+tests - os: ubuntu-18.04 compiler: gcc library: static diff --git a/app/test/meson.build b/app/test/meson.build index 5fc1dd1b7b..bb4621ed2a 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -149,96 +149,97 @@ test_deps = enabled_libs # as well as libs, the pci and vdev bus drivers are needed for a lot of tests test_deps += ['bus_pci', 'bus_vdev'] -# Each test is marked with flag true/false -# to indicate whether it can run in no-huge mode. +# Each test is marked with flags: +# - the first flag indicates whether the test can run in no-huge mode, +# - the second flag indicates whether the test can run with ASan enabled, fast_tests = [ - ['acl_autotest', true], - ['atomic_autotest', false], - ['bitmap_autotest', true], - ['bpf_autotest', true], - ['bpf_convert_autotest', true], - ['bitops_autotest', true], - ['byteorder_autotest', true], - ['cksum_autotest', true], - ['cmdline_autotest', true], - ['common_autotest', true], - ['cpuflags_autotest', true], - ['debug_autotest', true], - ['devargs_autotest', true], - ['eal_flags_c_opt_autotest', false], - ['eal_flags_main_opt_autotest', false], - ['eal_flags_n_opt_autotest', false], - ['eal_flags_hpet_autotest', false], - ['eal_flags_no_huge_autotest', false], - ['eal_flags_a_opt_autotest', false], - ['eal_flags_b_opt_autotest', false], - ['eal_flags_vdev_opt_autotest', false], - ['eal_flags_r_opt_autotest', false], - ['eal_flags_mem_autotest', false], - ['eal_flags_file_prefix_autotest', false], - ['eal_flags_misc_autotest', false], - ['eal_fs_autotest', true], - ['errno_autotest', true], - ['ethdev_link_status', true], - ['event_ring_autotest', true], - ['fib_autotest', true], - ['fib6_autotest', true], - ['func_reentrancy_autotest', false], - ['hash_autotest', true], - ['interrupt_autotest', true], - ['ipfrag_autotest', false], - ['lcores_autotest', true], - ['logs_autotest', true], - ['lpm_autotest', true], - ['lpm6_autotest', true], - ['malloc_autotest', false], - ['mbuf_autotest', false], - ['mcslock_autotest', false], - ['memcpy_autotest', true], - ['memory_autotest', false], - ['mempool_autotest', false], - ['memzone_autotest', false], - ['meter_autotest', true], - ['multiprocess_autotest', false], - ['per_lcore_autotest', true], - ['pflock_autotest', true], - ['prefetch_autotest', true], - ['rcu_qsbr_autotest', true], - ['pie_autotest', true], - ['rib_autotest', true], - ['rib6_autotest', true], - ['ring_autotest', true], - ['rwlock_test1_autotest', true], - ['rwlock_rda_autotest', true], - ['rwlock_rds_wrm_autotest', true], - ['rwlock_rde_wro_autotest', true], - ['sched_autotest', true], - ['security_autotest', false], - ['spinlock_autotest', true], - ['stack_autotest', false], - ['stack_lf_autotest', false], - ['string_autotest', true], - ['tailq_autotest', true], - ['ticketlock_autotest', true], - ['timer_autotest', false], - ['user_delay_us', true], - ['version_autotest', true], - ['crc_autotest', true], - ['distributor_autotest', false], - ['eventdev_common_autotest', true], - ['fbarray_autotest', true], - ['hash_readwrite_func_autotest', false], - ['ipsec_autotest', true], - ['kni_autotest', false], - ['kvargs_autotest', true], - ['member_autotest', true], - ['power_cpufreq_autotest', false], - ['power_autotest', true], - ['power_kvm_vm_autotest', false], - ['reorder_autotest', true], - ['service_autotest', true], - ['thash_autotest', true], - ['trace_autotest', true], + ['acl_autotest', true, true], + ['atomic_autotest', false, true], + ['bitmap_autotest', true, true], + ['bpf_autotest', true, true], + ['bpf_convert_autotest', true, true], + ['bitops_autotest', true, true], + ['byteorder_autotest', true, true], + ['cksum_autotest', true, true], + ['cmdline_autotest', true, true], + ['common_autotest', true, true], + ['cpuflags_autotest', true, true], + ['debug_autotest', true, true], + ['devargs_autotest', true, true], + ['eal_flags_c_opt_autotest', false, false], + ['eal_flags_main_opt_autotest', false, false], + ['eal_flags_n_opt_autotest', false, false], + ['eal_flags_hpet_autotest', false, false], + ['eal_flags_no_huge_autotest', false, false], + ['eal_flags_a_opt_autotest', false, false], + ['eal_flags_b_opt_autotest', false, false], + ['eal_flags_vdev_opt_autotest', false, false], + ['eal_flags_r_opt_autotest', false, false], + ['eal_flags_mem_autotest', false, false], + ['eal_flags_file_prefix_autotest', false, false], + ['eal_flags_misc_autotest', false, false], + ['eal_fs_autotest', true, true], + ['errno_autotest', true, true], + ['ethdev_link_status', true, true], + ['event_ring_autotest', true, true], + ['fib_autotest', true, true], + ['fib6_autotest', true, true], + ['func_reentrancy_autotest', false, true], + ['hash_autotest', true, true], + ['interrupt_autotest', true, true], + ['ipfrag_autotest', false, true], + ['lcores_autotest', true, true], + ['logs_autotest', true, true], + ['lpm_autotest', true, true], + ['lpm6_autotest', true, true], + ['malloc_autotest', false, true], + ['mbuf_autotest', false, true], + ['mcslock_autotest', false, true], + ['memcpy_autotest', true, true], + ['memory_autotest', false, true], + ['mempool_autotest', false, true], + ['memzone_autotest', false, true], + ['meter_autotest', true, true], + ['multiprocess_autotest', false, false], + ['per_lcore_autotest', true, true], + ['pflock_autotest', true, true], + ['prefetch_autotest', true, true], + ['rcu_qsbr_autotest', true, true], + ['pie_autotest', true, true], + ['rib_autotest', true, true], + ['rib6_autotest', true, true], + ['ring_autotest', true, true], + ['rwlock_test1_autotest', true, true], + ['rwlock_rda_autotest', true, true], + ['rwlock_rds_wrm_autotest', true, true], + ['rwlock_rde_wro_autotest', true, true], + ['sched_autotest', true, true], + ['security_autotest', false, true], + ['spinlock_autotest', true, true], + ['stack_autotest', false, true], + ['stack_lf_autotest', false, true], + ['string_autotest', true, true], + ['tailq_autotest', true, true], + ['ticketlock_autotest', true, true], + ['timer_autotest', false, true], + ['user_delay_us', true, true], + ['version_autotest', true, true], + ['crc_autotest', true, true], + ['distributor_autotest', false, true], + ['eventdev_common_autotest', true, true], + ['fbarray_autotest', true, true], + ['hash_readwrite_func_autotest', false, true], + ['ipsec_autotest', true, true], + ['kni_autotest', false, true], + ['kvargs_autotest', true, true], + ['member_autotest', true, true], + ['power_cpufreq_autotest', false, true], + ['power_autotest', true, true], + ['power_kvm_vm_autotest', false, true], + ['reorder_autotest', true, true], + ['service_autotest', true, true], + ['thash_autotest', true, true], + ['trace_autotest', true, true], ] # Tests known to have issues or which don't belong in other tests lists. @@ -345,15 +346,16 @@ endif if dpdk_conf.has('RTE_LIB_FLOW_CLASSIFY') test_sources += 'test_flow_classify.c' - fast_tests += [['flow_classify_autotest', false]] + fast_tests += [['flow_classify_autotest', false, true]] endif if dpdk_conf.has('RTE_LIB_METRICS') test_sources += ['test_metrics.c'] - fast_tests += [['metrics_autotest', true]] + fast_tests += [['metrics_autotest', true, true]] endif if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY') test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c'] - fast_tests += [['telemetry_json_autotest', true], ['telemetry_data_autotest', true]] + fast_tests += [['telemetry_json_autotest', true, true]] + fast_tests += [['telemetry_data_autotest', true, true]] endif if dpdk_conf.has('RTE_LIB_PIPELINE') # pipeline lib depends on port and table libs, so those must be present @@ -366,7 +368,7 @@ if dpdk_conf.has('RTE_LIB_PIPELINE') 'test_table_ports.c', 'test_table_tables.c', ] - fast_tests += [['table_autotest', true]] + fast_tests += [['table_autotest', true, true]] endif # The following linkages of drivers are required because @@ -386,26 +388,26 @@ if dpdk_conf.has('RTE_NET_RING') test_sources += 'test_pmd_ring.c' test_sources += 'test_event_eth_tx_adapter.c' test_sources += 'sample_packet_forward.c' - fast_tests += [['ring_pmd_autotest', true]] + fast_tests += [['ring_pmd_autotest', true, true]] perf_test_names += 'ring_pmd_perf_autotest' - fast_tests += [['event_eth_tx_adapter_autotest', false]] + fast_tests += [['event_eth_tx_adapter_autotest', false, true]] if dpdk_conf.has('RTE_LIB_BITRATESTATS') test_sources += 'test_bitratestats.c' - fast_tests += [['bitratestats_autotest', true]] + fast_tests += [['bitratestats_autotest', true, true]] endif if dpdk_conf.has('RTE_LIB_LATENCYSTATS') test_sources += 'test_latencystats.c' - fast_tests += [['latencystats_autotest', true]] + fast_tests += [['latencystats_autotest', true, true]] endif if dpdk_conf.has('RTE_LIB_PDUMP') test_sources += 'test_pdump.c' - fast_tests += [['pdump_autotest', true]] + fast_tests += [['pdump_autotest', true, false]] endif endif if dpdk_conf.has('RTE_NET_NULL') test_deps += 'net_null' test_sources += 'test_vdev.c' - fast_tests += [['vdev_autotest', true]] + fast_tests += [['vdev_autotest', true, true]] endif if dpdk_conf.has('RTE_HAS_LIBPCAP') @@ -431,7 +433,7 @@ if dpdk_conf.has('RTE_LIB_COMPRESSDEV') if compress_test_dep.found() test_dep_objs += compress_test_dep test_sources += 'test_compressdev.c' - fast_tests += [['compressdev_autotest', false]] + fast_tests += [['compressdev_autotest', false, true]] endif endif @@ -478,6 +480,12 @@ foreach arg : fast_tests endif endif + if get_option('b_sanitize') == 'address' or get_option('b_sanitize') == 'address,undefined' + if not arg[2] + run_test = false + endif + endif + if (get_option('default_library') == 'shared' and arg[0] == 'event_eth_tx_adapter_autotest') foreach drv:dpdk_drivers