From patchwork Tue Feb 21 05:52:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kamalakshitha Aligeri X-Patchwork-Id: 124256 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B658B41CF8; Tue, 21 Feb 2023 06:52:27 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 175D643152; Tue, 21 Feb 2023 06:52:27 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id E5EDB43151 for ; Tue, 21 Feb 2023 06:52:25 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 536E5106F; Mon, 20 Feb 2023 21:53:08 -0800 (PST) Received: from ampere-altra-2-1.usa.Arm.com (ampere-altra-2-1.usa.arm.com [10.118.91.158]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 405BC3F881; Mon, 20 Feb 2023 21:52:25 -0800 (PST) From: Kamalakshitha Aligeri To: Yuying.Zhang@intel.com, beilei.xing@intel.com, olivier.matz@6wind.com, andrew.rybchenko@oktetlabs.ru, bruce.richardson@intel.com, mb@smartsharesystems.com, konstantin.ananyev@huawei.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, feifei.wang2@arm.com Cc: dev@dpdk.org, nd@arm.com, Kamalakshitha Aligeri Subject: [PATCH v4 1/2] net/i40e: replace put function Date: Tue, 21 Feb 2023 05:52:04 +0000 Message-Id: <20230221055205.22984-2-kamalakshitha.aligeri@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230221055205.22984-1-kamalakshitha.aligeri@arm.com> References: <20230210065407.209567-2-kamalakshitha.aligeri@arm.com> <20230221055205.22984-1-kamalakshitha.aligeri@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Integrated zero-copy put API in mempool cache in i40e PMD. On Ampere Altra server, l3fwd single core's performance improves by 5% with the new API Signed-off-by: Kamalakshitha Aligeri Reviewed-by: Ruifeng Wang Reviewed-by: Feifei Wang Reviewed-by: Morten Brørup --- Depends-on: series-26984 ("[v9] mempool cache: add zero-copy get and put functions") .mailmap | 1 + drivers/net/i40e/i40e_rxtx_vec_common.h | 27 ++++++++++++++++++++----- 2 files changed, 23 insertions(+), 5 deletions(-) -- 2.25.1 diff --git a/.mailmap b/.mailmap index 75884b6fe2..05a42edbcf 100644 --- a/.mailmap +++ b/.mailmap @@ -670,6 +670,7 @@ Kai Ji Kaiwen Deng Kalesh AP Kamalakannan R +Kamalakshitha Aligeri Kamil Bednarczyk Kamil Chalupnik Kamil Rytarowski diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h b/drivers/net/i40e/i40e_rxtx_vec_common.h index fe1a6ec75e..35cdb31b2e 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_common.h +++ b/drivers/net/i40e/i40e_rxtx_vec_common.h @@ -95,18 +95,35 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq) n = txq->tx_rs_thresh; - /* first buffer to free from S/W ring is at index - * tx_next_dd - (tx_rs_thresh-1) - */ + /* first buffer to free from S/W ring is at index + * tx_next_dd - (tx_rs_thresh-1) + */ txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)]; if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) { + struct rte_mempool *mp = txep[0].mbuf->pool; + struct rte_mempool_cache *cache = rte_mempool_default_cache(mp, rte_lcore_id()); + void **cache_objs; + + if (unlikely(!cache)) + goto fallback; + + cache_objs = rte_mempool_cache_zc_put_bulk(cache, mp, n); + if (unlikely(!cache_objs)) + goto fallback; + for (i = 0; i < n; i++) { - free[i] = txep[i].mbuf; + cache_objs[i] = txep[i].mbuf; /* no need to reset txep[i].mbuf in vector path */ } - rte_mempool_put_bulk(free[0]->pool, (void **)free, n); goto done; + +fallback: + for (i = 0; i < n; i++) + free[i] = txep[i].mbuf; + rte_mempool_generic_put(mp, (void **)free, n, cache); + goto done; + } m = rte_pktmbuf_prefree_seg(txep[0].mbuf); From patchwork Tue Feb 21 05:52:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kamalakshitha Aligeri X-Patchwork-Id: 124257 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E4EC41CF8; Tue, 21 Feb 2023 06:52:34 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4C3BC4315B; Tue, 21 Feb 2023 06:52:30 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 79A6540E5A for ; Tue, 21 Feb 2023 06:52:28 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EC84F1515; Mon, 20 Feb 2023 21:53:10 -0800 (PST) Received: from ampere-altra-2-1.usa.Arm.com (ampere-altra-2-1.usa.arm.com [10.118.91.158]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D6E3B3F881; Mon, 20 Feb 2023 21:52:27 -0800 (PST) From: Kamalakshitha Aligeri To: Yuying.Zhang@intel.com, beilei.xing@intel.com, olivier.matz@6wind.com, andrew.rybchenko@oktetlabs.ru, bruce.richardson@intel.com, mb@smartsharesystems.com, konstantin.ananyev@huawei.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, feifei.wang2@arm.com Cc: dev@dpdk.org, nd@arm.com, Kamalakshitha Aligeri Subject: [PATCH v4 2/2] test/mempool: add zero-copy API's Date: Tue, 21 Feb 2023 05:52:05 +0000 Message-Id: <20230221055205.22984-3-kamalakshitha.aligeri@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230221055205.22984-1-kamalakshitha.aligeri@arm.com> References: <20230210065407.209567-2-kamalakshitha.aligeri@arm.com> <20230221055205.22984-1-kamalakshitha.aligeri@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Added mempool test cases with zero-copy get and put API's Signed-off-by: Kamalakshitha Aligeri Reviewed-by: Ruifeng Wang Reviewed-by: Feifei Wang Acked-by: Morten Brørup --- Depends-on: series-26984 ("[v9] mempool cache: add zero-copy get and put functions") app/test/test_mempool.c | 81 ++++++++++++++++++++++++++++++----------- 1 file changed, 60 insertions(+), 21 deletions(-) -- 2.25.1 diff --git a/app/test/test_mempool.c b/app/test/test_mempool.c index 8e493eda47..6d29f5bc7b 100644 --- a/app/test/test_mempool.c +++ b/app/test/test_mempool.c @@ -74,7 +74,7 @@ my_obj_init(struct rte_mempool *mp, __rte_unused void *arg, /* basic tests (done on one core) */ static int -test_mempool_basic(struct rte_mempool *mp, int use_external_cache) +test_mempool_basic(struct rte_mempool *mp, int use_external_cache, int use_zc_api) { uint32_t *objnum; void **objtable; @@ -84,6 +84,7 @@ test_mempool_basic(struct rte_mempool *mp, int use_external_cache) unsigned i, j; int offset; struct rte_mempool_cache *cache; + void **cache_objs; if (use_external_cache) { /* Create a user-owned mempool cache. */ @@ -100,8 +101,13 @@ test_mempool_basic(struct rte_mempool *mp, int use_external_cache) rte_mempool_dump(stdout, mp); printf("get an object\n"); - if (rte_mempool_generic_get(mp, &obj, 1, cache) < 0) - GOTO_ERR(ret, out); + if (use_zc_api) { + cache_objs = rte_mempool_cache_zc_get_bulk(cache, mp, 1); + obj = *cache_objs; + } else { + if (rte_mempool_generic_get(mp, &obj, 1, cache) < 0) + GOTO_ERR(ret, out); + } rte_mempool_dump(stdout, mp); /* tests that improve coverage */ @@ -123,21 +129,41 @@ test_mempool_basic(struct rte_mempool *mp, int use_external_cache) #endif printf("put the object back\n"); - rte_mempool_generic_put(mp, &obj, 1, cache); + if (use_zc_api) { + cache_objs = rte_mempool_cache_zc_put_bulk(cache, mp, 1); + rte_memcpy(cache_objs, &obj, sizeof(void *)); + } else { + rte_mempool_generic_put(mp, &obj, 1, cache); + } rte_mempool_dump(stdout, mp); printf("get 2 objects\n"); - if (rte_mempool_generic_get(mp, &obj, 1, cache) < 0) - GOTO_ERR(ret, out); - if (rte_mempool_generic_get(mp, &obj2, 1, cache) < 0) { - rte_mempool_generic_put(mp, &obj, 1, cache); - GOTO_ERR(ret, out); + if (use_zc_api) { + cache_objs = rte_mempool_cache_zc_get_bulk(cache, mp, 1); + obj = *cache_objs; + cache_objs = rte_mempool_cache_zc_get_bulk(cache, mp, 1); + obj2 = *cache_objs; + } else { + if (rte_mempool_generic_get(mp, &obj, 1, cache) < 0) + GOTO_ERR(ret, out); + if (rte_mempool_generic_get(mp, &obj2, 1, cache) < 0) { + rte_mempool_generic_put(mp, &obj, 1, cache); + GOTO_ERR(ret, out); + } } rte_mempool_dump(stdout, mp); printf("put the objects back\n"); - rte_mempool_generic_put(mp, &obj, 1, cache); - rte_mempool_generic_put(mp, &obj2, 1, cache); + if (use_zc_api) { + cache_objs = rte_mempool_cache_zc_put_bulk(cache, mp, 1); + rte_memcpy(cache_objs, &obj, sizeof(void *)); + cache_objs = rte_mempool_cache_zc_put_bulk(cache, mp, 1); + rte_memcpy(cache_objs, &obj2, sizeof(void *)); + + } else { + rte_mempool_generic_put(mp, &obj, 1, cache); + rte_mempool_generic_put(mp, &obj2, 1, cache); + } rte_mempool_dump(stdout, mp); /* @@ -149,8 +175,13 @@ test_mempool_basic(struct rte_mempool *mp, int use_external_cache) GOTO_ERR(ret, out); for (i = 0; i < MEMPOOL_SIZE; i++) { - if (rte_mempool_generic_get(mp, &objtable[i], 1, cache) < 0) - break; + if (use_zc_api) { + cache_objs = rte_mempool_cache_zc_get_bulk(cache, mp, 1); + objtable[i] = *cache_objs; + } else { + if (rte_mempool_generic_get(mp, &objtable[i], 1, cache) < 0) + break; + } } /* @@ -170,8 +201,12 @@ test_mempool_basic(struct rte_mempool *mp, int use_external_cache) if (obj_data[j] != 0) ret = -1; } - - rte_mempool_generic_put(mp, &objtable[i], 1, cache); + if (use_zc_api) { + cache_objs = rte_mempool_cache_zc_put_bulk(cache, mp, 1); + rte_memcpy(cache_objs, &objtable[i], sizeof(void *)); + } else { + rte_mempool_generic_put(mp, &objtable[i], 1, cache); + } } free(objtable); @@ -979,15 +1014,19 @@ test_mempool(void) rte_mempool_list_dump(stdout); /* basic tests without cache */ - if (test_mempool_basic(mp_nocache, 0) < 0) + if (test_mempool_basic(mp_nocache, 0, 0) < 0) + GOTO_ERR(ret, err); + + /* basic tests with zero-copy API's */ + if (test_mempool_basic(mp_cache, 0, 1) < 0) GOTO_ERR(ret, err); - /* basic tests with cache */ - if (test_mempool_basic(mp_cache, 0) < 0) + /* basic tests with user-owned cache and zero-copy API's */ + if (test_mempool_basic(mp_nocache, 1, 1) < 0) GOTO_ERR(ret, err); /* basic tests with user-owned cache */ - if (test_mempool_basic(mp_nocache, 1) < 0) + if (test_mempool_basic(mp_nocache, 1, 0) < 0) GOTO_ERR(ret, err); /* more basic tests without cache */ @@ -1008,10 +1047,10 @@ test_mempool(void) GOTO_ERR(ret, err); /* test the stack handler */ - if (test_mempool_basic(mp_stack, 1) < 0) + if (test_mempool_basic(mp_stack, 1, 0) < 0) GOTO_ERR(ret, err); - if (test_mempool_basic(default_pool, 1) < 0) + if (test_mempool_basic(default_pool, 1, 0) < 0) GOTO_ERR(ret, err); /* test mempool event callbacks */