From patchwork Tue Sep 15 19:34:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Wojciechowski X-Patchwork-Id: 77812 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7B5C7A04C7; Tue, 15 Sep 2020 21:35:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7E1631C12C; Tue, 15 Sep 2020 21:35:04 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 6B2391C10A for ; Tue, 15 Sep 2020 21:34:59 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20200915193458euoutp010022215fef8fe2fd5ad2df9867b572cb~1DA5wbq9x1761317613euoutp01_ for ; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20200915193458euoutp010022215fef8fe2fd5ad2df9867b572cb~1DA5wbq9x1761317613euoutp01_ DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1600198498; bh=xPWUwWzs8KDd9GRIYyNVFLY6E9FQFwkbk3DM1rTtSlk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=k/kXUkpdkw7riFSrjmtY/fr+J8KnqOU1qk0srN1oaKOLkYI+OSKtsjjckeMHMmqoK p7WRTg62r1NRKXwSE3oVuFKmgF+Sbvd0xSbqQbo0eqqCD9g5K2XWJgbUj3yUS4XCgl 3hUZyyydbarM+/uqgw9O9dT3r4hKqZp/YYVDva/c= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20200915193458eucas1p28e3b5333e24e4c816caf2befb585c59e~1DA5mlBtG0563105631eucas1p2h; Tue, 15 Sep 2020 19:34:58 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 7E.60.05997.267116F5; Tue, 15 Sep 2020 20:34:58 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20200915193457eucas1p2321d28b6abf69f244cd7c1e61ed0620e~1DA5AZzrH3166631666eucas1p29; Tue, 15 Sep 2020 19:34:57 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20200915193457eusmtrp1cc25e5d5d131c08378a5506ffa7e755b~1DA4-2VW41473114731eusmtrp19; Tue, 15 Sep 2020 19:34:57 +0000 (GMT) X-AuditID: cbfec7f4-677ff7000000176d-0c-5f611762be36 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 63.7C.06314.167116F5; Tue, 15 Sep 2020 20:34:57 +0100 (BST) Received: from Padamandas.fritz.box (unknown [106.210.88.70]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20200915193457eusmtip24ddc8686f0599fd010fb3879d3b9e329~1DA4XrXV82767527675eusmtip2p; Tue, 15 Sep 2020 19:34:57 +0000 (GMT) From: Lukasz Wojciechowski To: David Hunt , Bruce Richardson Cc: dev@dpdk.org, l.wojciechow@partner.samsung.com, stable@dpdk.org Date: Tue, 15 Sep 2020 21:34:45 +0200 Message-Id: <20200915193449.13310-3-l.wojciechow@partner.samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrCIsWRmVeSWpSXmKPExsWy7djP87pJ4onxBuuOy1rcWGVv0TfpI5PF u0/bmSye9axjtPjX8YfdgdXj14KlrB6L97xk8jj4bg9TAHMUl01Kak5mWWqRvl0CV8aHhb0s BW91KqZv/c/awLhQpYuRk0NCwERi84ejLF2MXBxCAisYJSaueMYCkhAS+MIosXuHOETiM6PE qw1PmGA6NrWeYoMoWs4osXF3HIT9CajoBVicTcBW4sjMr6wgtohAmERz816goRwczALOEk++ gpUIC7hL/Fh8FGwki4CqxIUlB9hBbF4BV4nr675ArZKXWL3hADOIzSngJnH142xmkHskBC6z Saw4sYQFoshF4vu51+wQtrDEq+NboGwZidOTe1ggGrYxSlz9/ZMRwtnPKHG9dwVUlbXE4X+/ 2SCu05RYv0sfIuwoca1nOiNIWEKAT+LGW0GQMDOQOWnbdGaIMK9ER5sQRLWexNOeqYwwa/+s fcICUeIh0frSCRKEMxkldq3fzDyBUX4Wwq4FjIyrGMVTS4tz01OLjfJSy/WKE3OLS/PS9ZLz czcxAuP+9L/jX3Yw7vqTdIhRgINRiYd3we2EeCHWxLLiytxDjBIczEoivE5nT8cJ8aYkVlal FuXHF5XmpBYfYpTmYFES5zVe9DJWSCA9sSQ1OzW1ILUIJsvEwSnVwDjr0oLu6gkXK6r/1wSW 9jFIKYquZ4+dkTj74t3p1RcnOGWvDniXlfpzw8qI2bf2zq+ct4grPI/7/fpPBsy6FrbOk1r3 7C5nOPZfIaozR+VUwCuxK9HWz0sTwpweiaRK/ru2iquw68fxz5el/JdbCx7IdL3xYPHOE0cL 8vdOyPxcfnCJW976vodKLMUZiYZazEXFiQB/AuSi9wIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrOLMWRmVeSWpSXmKPExsVy+t/xe7qJ4onxBqvazS1urLK36Jv0kcni 3aftTBbPetYxWvzr+MPuwOrxa8FSVo/Fe14yeRx8t4cpgDlKz6Yov7QkVSEjv7jEVina0MJI z9DSQs/IxFLP0Ng81srIVEnfziYlNSezLLVI3y5BL+PDwl6Wgrc6FdO3/mdtYFyo0sXIySEh YCKxqfUUWxcjF4eQwFJGiXsbehm7GDmAEjISHy4JQNQIS/y51gVV84FRYmnLRxaQBJuArcSR mV9ZQepFBMIkTqz0BwkzC7hLbFk8lRnEFgayfyw+ygRiswioSlxYcoAdxOYVcJW4vu4LE8R8 eYnVGw6A1XMKuElc/TgbzBYCqll88jz7BEa+BYwMqxhFUkuLc9Nziw31ihNzi0vz0vWS83M3 MQIDcduxn5t3MF7aGHyIUYCDUYmHd8HthHgh1sSy4srcQ4wSHMxKIrxOZ0/HCfGmJFZWpRbl xxeV5qQWH2I0BTpqIrOUaHI+MErySuINTQ3NLSwNzY3Njc0slMR5OwQOxggJpCeWpGanphak FsH0MXFwSjUwBh47MSH3gaHjra6bFRcNn05Wm77o56mwVyHLDK04Kh8nLqxZysvF9L/6l9Wk t9r5Qi0iXMLXLzxru6I2s3eWrfjBb+nr1zx6HvNgc1p07g6B+1l8Ypp7DA+klXAs6FtwcPHp sjMTwnd+mXjRcO/hPcDg3zFrUnlj8G+b9T1zchfeWDmReX39TiWW4oxEQy3mouJEAL9RtPNa AgAA X-CMS-MailID: 20200915193457eucas1p2321d28b6abf69f244cd7c1e61ed0620e X-Msg-Generator: CA X-RootMTR: 20200915193457eucas1p2321d28b6abf69f244cd7c1e61ed0620e X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20200915193457eucas1p2321d28b6abf69f244cd7c1e61ed0620e References: <20200915193449.13310-1-l.wojciechow@partner.samsung.com> Subject: [dpdk-dev] [PATCH v1 2/6] app/test: synchronize statistics between lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Statistics of handled packets are cleared and read on main lcore, while they are increased in workers handlers on different lcores. Without synchronization occasionally showed invalid values. This patch uses atomic acquire/release mechanisms to synchronize. Fixes: c3eabff124e6 ("distributor: add unit tests") Cc: bruce.richardson@intel.com Cc: stable@dpdk.org Signed-off-by: Lukasz Wojciechowski Acked-by: David Hunt --- app/test/test_distributor.c | 39 ++++++++++++++++++++++++------------- 1 file changed, 26 insertions(+), 13 deletions(-) diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c index 35b25463a..0e49e3714 100644 --- a/app/test/test_distributor.c +++ b/app/test/test_distributor.c @@ -43,7 +43,8 @@ total_packet_count(void) { unsigned i, count = 0; for (i = 0; i < worker_idx; i++) - count += worker_stats[i].handled_packets; + count += __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE); return count; } @@ -52,6 +53,7 @@ static inline void clear_packet_count(void) { memset(&worker_stats, 0, sizeof(worker_stats)); + rte_atomic_thread_fence(__ATOMIC_RELEASE); } /* this is the basic worker function for sanity test @@ -72,13 +74,13 @@ handle_work(void *arg) num = rte_distributor_get_pkt(db, id, buf, buf, num); while (!quit) { __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_RELAXED); + __ATOMIC_ACQ_REL); count += num; num = rte_distributor_get_pkt(db, id, buf, buf, num); } __atomic_fetch_add(&worker_stats[id].handled_packets, num, - __ATOMIC_RELAXED); + __ATOMIC_ACQ_REL); count += num; rte_distributor_return_pkt(db, id, buf, num); return 0; @@ -134,7 +136,8 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); printf("Sanity test with all zero hashes done.\n"); /* pick two flows and check they go correctly */ @@ -159,7 +162,9 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n( + &worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); printf("Sanity test with two hash values done\n"); } @@ -185,7 +190,8 @@ sanity_test(struct worker_params *wp, struct rte_mempool *p) for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); printf("Sanity test with non-zero hashes done\n"); rte_mempool_put_bulk(p, (void *)bufs, BURST); @@ -280,15 +286,17 @@ handle_work_with_free_mbufs(void *arg) buf[i] = NULL; num = rte_distributor_get_pkt(d, id, buf, buf, num); while (!quit) { - worker_stats[id].handled_packets += num; count += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, id, buf, buf, num); } - worker_stats[id].handled_packets += num; count += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); rte_distributor_return_pkt(d, id, buf, num); return 0; } @@ -363,8 +371,9 @@ handle_work_for_shutdown_test(void *arg) /* wait for quit single globally, or for worker zero, wait * for zero_quit */ while (!quit && !(id == zero_id && zero_quit)) { - worker_stats[id].handled_packets += num; count += num; + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); for (i = 0; i < num; i++) rte_pktmbuf_free(buf[i]); num = rte_distributor_get_pkt(d, @@ -379,10 +388,11 @@ handle_work_for_shutdown_test(void *arg) total += num; } - worker_stats[id].handled_packets += num; count += num; returned = rte_distributor_return_pkt(d, id, buf, num); + __atomic_fetch_add(&worker_stats[id].handled_packets, num, + __ATOMIC_ACQ_REL); if (id == zero_id) { /* for worker zero, allow it to restart to pick up last packet * when all workers are shutting down. @@ -394,10 +404,11 @@ handle_work_for_shutdown_test(void *arg) id, buf, buf, num); while (!quit) { - worker_stats[id].handled_packets += num; count += num; rte_pktmbuf_free(pkt); num = rte_distributor_get_pkt(d, id, buf, buf, num); + __atomic_fetch_add(&worker_stats[id].handled_packets, + num, __ATOMIC_ACQ_REL); } returned = rte_distributor_return_pkt(d, id, buf, num); @@ -461,7 +472,8 @@ sanity_test_with_worker_shutdown(struct worker_params *wp, for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); if (total_packet_count() != BURST * 2) { printf("Line %d: Error, not all packets flushed. " @@ -514,7 +526,8 @@ test_flush_with_worker_shutdown(struct worker_params *wp, zero_quit = 0; for (i = 0; i < rte_lcore_count() - 1; i++) printf("Worker %u handled %u packets\n", i, - worker_stats[i].handled_packets); + __atomic_load_n(&worker_stats[i].handled_packets, + __ATOMIC_ACQUIRE)); if (total_packet_count() != BURST) { printf("Line %d: Error, not all packets flushed. "