From patchwork Thu Aug 24 13:28:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Santosh Shukla X-Patchwork-Id: 27859 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id B40757CB3; Thu, 24 Aug 2017 15:30:07 +0200 (CEST) Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0043.outbound.protection.outlook.com [104.47.32.43]) by dpdk.org (Postfix) with ESMTP id 8183D568A for ; Thu, 24 Aug 2017 15:30:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=qD5PhOGGIF3LcnohGh8/bpToCS9WUPtV1sYlRPZG4HE=; b=Iuo2aZft5CkfN1Ky+4q7DVR4nuUm2KVUNmkf+6QFhgdghC9YaBRTznLAsi5lkAKT/FnPazvn+ohI0wgxkf0zCUcZ4GSzziIveLgYR2ABpHxAWy3rGwYNzHXuhLUcu2Qxr0tqGnuVsPLuu7puXLwkE17xBH49f5GKJHhOgpzGRfM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Santosh.Shukla@cavium.com; Received: from localhost.localdomain (14.140.2.178) by CY4PR07MB3093.namprd07.prod.outlook.com (10.172.115.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1362.18; Thu, 24 Aug 2017 13:30:02 +0000 From: Santosh Shukla To: olivier.matz@6wind.com, dev@dpdk.org Cc: thomas@monjalon.net, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, Santosh Shukla Date: Thu, 24 Aug 2017 18:58:57 +0530 Message-Id: <20170824132903.32057-6-santosh.shukla@caviumnetworks.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170824132903.32057-1-santosh.shukla@caviumnetworks.com> References: <20170824132903.32057-1-santosh.shukla@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [14.140.2.178] X-ClientProxiedBy: BM1PR01CA0071.INDPRD01.PROD.OUTLOOK.COM (10.174.208.139) To CY4PR07MB3093.namprd07.prod.outlook.com (10.172.115.7) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6b8d1811-a623-49a1-a86b-08d4eaf43b49 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:CY4PR07MB3093; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3093; 3:Z6FVbqayJkzSMpCJKqY3IFcbZ0oRFsOMHEyUd2EwLlVuxXproH+vh9ZdJGyj1AAx2ew2pV7tKobp2KbKNibkhEfBxsw+Y3jXsUGGCN2ABUb8NKfPWGSqNoB5GKXVq7l6pjqIlyArvfCo/EAAQLhCSryTSk2QyN35T6kY4S40PvTOZr/uJGLssp8NjE7p2NmERYKRBXEBp04XafG711SNTepBdLGwR9Hlikobj+qYFH/+svdRpw+hbpwaAZyKTv2h; 25:nPs5an/9HJQfIT2SwfJtOvYIlw2dyBvZHno68ji2eJ6ZUmR0ccz8J8yz1B68U8naGA47FbikukvulBi200B5j2fWoVbslsrqgX3wO0YTRE2ycbtVvfTeebryHxbGwVavi/WXC3TEFTLX4D9VwnK85r7txUolfWmulykLfYWx3OZKalyCwUZ6d9xMrF70M63wFFMlTR3sl948R6IbL0JiwfdJ4E4ZEOQ4BFh0BZ+0hQsdGpH8YLvwvSvv3Gf+RXe4+kODZOP+cOHaQW36kKZ33axIILSGsTZlkFDLq4xHzuLOe0TCd30ZuK+dBIffDGb+Eqyogy1GCy7qtDOoaaYCHg==; 31:ffrLd/0kQKAqqsR2CPUym1K0BsyfHLwJkh0MC8y3bGgvKeOVnEg8LxEXtuDRa7E3hPTV49XuFqYVrFreLYKHNYwHIabnLRzjOobyMTXhIavH2qjrEWt2+CGBVlLFAKS1E9MRWLtwJLv5LusNJ0lvsRD7q0dGFzu26jnPyr53Zte5wIZ7qxlZefaVYxP0WlfwGySjbvBpeH2U0mlI2Ruf6Un2P48SOE3pctuh3WYpFiY= X-MS-TrafficTypeDiagnostic: CY4PR07MB3093: X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3093; 20:/q5mnc9jJZHN8LcqYoWyA9RurVUFN4Uyv4HmJKWaRzuXNgucqZ94uZxTgEk0hd3L6k7XwqjRCPo91SCF+JVQtyR0tRg0VIfnbsZK3V7bAZB/TAlGNwBul5kkgSiTVb61xNlIVAhekPAGhJPa60cbT4ANXC+vOM6yj5XBXvDk07w1wYt9m8O9wF9S95oY4HHTihSn/Y6o3td/WG9zSZ3rI76nkr7sNBeW3ZkQZWROq87N3JLkkE8HBpXkDQMykaWRDw81yXlJlTo7bkXeg1chP+DQfV4tIQU+x5KIutpkezdxJft9DCfmMbVpDDp6Rq617RQEROFTiRRGUe6Zd3GH5LxuilWuSBfWyMf8x2tUf3EUMNXFsFDFXYPaMwNIKL08heQEAEoeCV46r9eENDmtuzVhJT77PC6oyJJVtjYbquj9OMS9ObwZCbnaOJBeBK5eWi/rQEjR5hRN8AFX0HkAbiTmDdj7mgwE/hM1ZufIi1p8iT1ylsRB9qXl6i0NSk9S4WHyFNcRoC3wYAORBZNBLEIzrbesJm7+xu6zfKhTdS65dSZal1yvWJ+E9Uf9bUe2HMfmFDih1Eid9Y7noPIrrgIEgZpe4v1S9am+cSBbTVk=; 4:q0lScV3eexA9joTVfFS8LvKIsDOeN7p05/G8vaVTxB2we8HT6hXCefbGQNR5ZJJH0QNEnNEDkLlj6c+7Imu+GctOnBMTMeNGCr//roTFzsSfeccjuwMZoJ5i4m0MfTleMgYdiU0gmntCv6AVMtHu1M/We/o91UD1sQWZSjY1MYtXQy/6NEma1qvO93S6PLrvQcaOgUKy+rI8B/k1eJUC19JYW48xf1T5WK7mTf0KKFvw72UQfK7B3Xa8ET5/h9KI X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(10201501046)(3002001)(93006095)(6041248)(20161123560025)(20161123558100)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY4PR07MB3093; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY4PR07MB3093; X-Forefront-PRVS: 04097B7F7F X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(7370300001)(4630300001)(6069001)(6009001)(189002)(199003)(110136004)(107886003)(53936002)(105586002)(101416001)(7736002)(50226002)(76176999)(42186005)(68736007)(6486002)(33646002)(50986999)(6512007)(106356001)(1076002)(2906002)(6506006)(3846002)(6116002)(478600001)(97736004)(4326008)(5660300001)(7350300001)(47776003)(5009440100003)(72206003)(575784001)(50466002)(8656003)(48376002)(25786009)(81156014)(81166006)(66066001)(305945005)(6666003)(5003940100001)(42882006)(36756003)(8676002)(189998001)(2950100002)(110426004); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR07MB3093; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR07MB3093; 23:2ITyo5tWhbYk0sS5kl4bi4qSuPfymNRPF9uWKWqg1?= VVdLs1KEg4Lp85Egt7V4OaqM0KkoTgsS010/602sF45Rl0lXJwyqgtZNjYsPQLLYZu+PZTdDnosDssQfl7+t+/RE68nOkgrHNc4UyYGCvD7E58xaWj8HNNOitqmkNDagD7w9hhgn+e+YdAkIBiQCLArlCvtvYTUBqHAUGxjXQsHM5zp/4G4a/vBlGlv2xZDBq8Qo0VV+9epPH+uoper35MHCgzciErsSTywUPUcAxTkrGAUbyIHiBIFkNtreOMZFSenLedqps05crDHj5RT5EZvlJW+b2t4OxAau6sa/wEYtjvUjUK3cGnqxMfNj2ubcWN5t9TtN1VIBqSE9NORDUIzRApLRubkI9epdyG6FaVmtnvbyBhCSxcuZYKROeng1dl321YUKwgZjpC9peQ49nGu1NgJzWoigV99tGVzBKtU6tRmgc5NUFTPDi2RGFOWmkpXb0JUuV3Emx5WvSaOlRDvTf9PB1iucU/TLtQ8zjv6SvC1318JX2hXLQwQx+p+/Oeemoodqyr8fSjZG1IUrlhqN40sdSIVkBrYniSIoUP+M9CvFVlrspIkkYZkgRfHJ+8MRrZMyZ8rdHAvYfO/6REScCxmtaxwwnOqryyyp+cyP1bzjMqEmyhliMsAdDmZCJntmgTzmcfp/dJsdK0sRWQqRHbeeKJ8ZVAlXUCNop73ckG9/ghQmR4NWXa9o6taZpoaC1ufKpYkab1Tdo8tYugSc9OK5pCrek1PCGsZ7Nyt09iINvE2bAxAYnbgFrwUfXp/9rSYqkYcuzYXu8H/AYOYluzRh28FLwbY0ZyoOZNzSG//0gQUC7WnpTdesePQSbQ7sAnrygCn8b+73b8fQq/Ixwl4EOnFY7Pm1Ik4N6JgJSmHbxE9FH5WByPGlVMXIl+jQmkLViJ9TTptao78MEoKjg0WxXJeYbnSY+JccwApEfU/Kv5R6sfmuvwid/IQVvGxOw5rFeZEsUDOs7Fli2ODPwdh1Hpxh89RN4cJoHt0m+n7MDs+5sMFLvIFwV34YCAtaPe3HS+PWXIO5lk9Mrl8hTvr42XP+rd0+xOJXDmFhfozNA4eaJCYNeYjSLVZCYu/ZEz8b8rijrXOEzlaYfAV9APqS4M5ppMWP5ZCo4r2/Hvp3QVPb5CQlv5BWJbXMg0RryzYieQ8yugyJA/jm0sZIJGXniJVImw2u6Y8JfYtC2AoBz/FypYJWmr2TGDGuZw= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3093; 6:j3Gja1TzMuQVBav7qElit2nh+F+D9ity0ehLmXn2IA3kxJJjpHSt28255NNYKS3v0hFIyKOkrzbQexmC41VmxzH3v8/xzN/qXFUNGTxs8tsYyjUTq0MKRzvhmkNw2Meu5JdPgOXByBCwXyeKoWUHzXXhFdHKKWSukoQfII+h9zTNIuPL+8iNB4XUrYUzW6i3NlUk4djg09rb2ZEdDw4YDovf6iWryULOz1FD4gDMXU4P7XtvglXA+iYGEB6dT2Kysq08VBzvi2jxN3WEOW4expwU4KqdSGSsItHY/HfACd+Q09EdBiuDAL0BfjuKNPmrKjbDv5ExPKZV7BqDwIYtwQ==; 5:+XQuXJUbCK+YdY/SCipWUWZh+Js0REyxd6Pe5Ly1iOlF1fL3EJIVCmMC3wJU5ZOR5Aw2Og7NsIIpEf03u86tp2xfrMGINs8tmLoMFh/AKX5/pKtmkZ/TvsouLsMik7b/HN9dwyG/8bzf+tA869TzOgfct+0EDAa+vXS9oC/RyuA=; 24:jcvURYjt7dAosxV4640ZUkqpsTvmGN/WgFe47zYCcTiA11AGxFPOB9X/xqk0UJ4mG9hFJYh9gEC78Fy9TwTxIpXIH3hd3qwHC610k8jLPy0=; 7:WctUUX5z/WIMOFyKBC+MtqvFmBTaRaR8QJS+nYSeeIP7Ph2MryBETD8X4+M4G5BThtE0fdt2B7epo2pC1hEQERQxBxZ3CLqDBO5b7bbvZ9KNOre8s25WzcYdIHj0xNr5VftqaqrHulN5nTOTEXftZdIGZzJ+6uEOgjWxcBZuskoPkl2QnJoF080Q1zMh6WNObZwfhSC8fxyyMm4I/EyX7y0gJ13h+VCpbCdmIzHBCv8= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Aug 2017 13:30:02.7747 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3093 Subject: [dpdk-dev] [PATCH v1 05/11] mempool/octeontx: implement pool free X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Upon pool free request from application, Octeon FPA free does following: - Uses mbox to reset fpapf pool setup. - frees fpavf resources. Signed-off-by: Santosh Shukla Signed-off-by: Jerin Jacob --- drivers/mempool/octeontx/octeontx_fpavf.c | 107 ++++++++++++++++++++++++ drivers/mempool/octeontx/octeontx_fpavf.h | 2 + drivers/mempool/octeontx/rte_mempool_octeontx.c | 12 ++- 3 files changed, 120 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/octeontx/octeontx_fpavf.c b/drivers/mempool/octeontx/octeontx_fpavf.c index 85ddf0a03..bcbbefd7d 100644 --- a/drivers/mempool/octeontx/octeontx_fpavf.c +++ b/drivers/mempool/octeontx/octeontx_fpavf.c @@ -582,6 +582,113 @@ octeontx_fpa_bufpool_create(unsigned int object_size, unsigned int object_count, return (uintptr_t)NULL; } +/* + * Destroy a buffer pool. + */ +int +octeontx_fpa_bufpool_destroy(uintptr_t handle, int node_id) +{ + void **node, **curr, *head = NULL; + uint64_t sz; + uint64_t cnt, avail; + unsigned int gpool; + int ret; + + RTE_SET_USED(node_id); + + /* Wait for all outstanding writes to be comitted */ + rte_smp_wmb(); + + if (unlikely(!octeontx_fpa_handle_valid(handle))) + return -EINVAL; + + /* get pool */ + gpool = octeontx_fpa_handle2gpool(handle); + + /* Check for no outstanding buffers */ + cnt = fpavf_read64((void *)((uintptr_t)handle + + FPA_VF_VHAURA_CNT(gpool))); + if (cnt) { + fpavf_log_dbg("buffer exist in pool cnt %ld\n", cnt); + return -EBUSY; + } + + rte_spinlock_lock(&fpadev.lock); + + avail = fpavf_read64((void *)((uintptr_t)handle + + FPA_VF_VHPOOL_AVAILABLE(gpool))); + + /* Prepare to empty the entire POOL */ + fpavf_write64(avail, (void *)((uintptr_t)handle + + FPA_VF_VHAURA_CNT_LIMIT(gpool))); + fpavf_write64(avail + 1, (void *)((uintptr_t)handle + + FPA_VF_VHAURA_CNT_THRESHOLD(gpool))); + + /* Empty the pool */ + /* Invalidate the POOL */ + octeontx_gpool_free(gpool); + + /* Process all buffers in the pool */ + while (avail--) { + + /* Yank a buffer from the pool */ + node = (void *)(uintptr_t) + fpavf_read64((void *) + (handle + FPA_VF_VHAURA_OP_ALLOC(gpool))); + + if (node == NULL) { + fpavf_log_err("ERROR: GAURA[%u] missing %lu buffers\n", + gpool, avail); + break; + } + + /* Imsert it into an ordered linked list */ + for (curr = &head; curr[0] != NULL; curr = curr[0]) { + if ((uintptr_t)node <= (uintptr_t)curr[0]) + break; + } + node[0] = curr[0]; + curr[0] = node; + } + + /* Verify the linked list to be a perfect series */ + sz = octeontx_fpa_bufpool_block_size(handle) << 7; + for (curr = head; curr != NULL && curr[0] != NULL; + curr = curr[0]) { + if (curr == curr[0] || + (curr != ((void *)((uintptr_t)curr[0] - sz)))) { + fpavf_log_err("POOL# %u buf sequence err (%p vs. %p)\n", + gpool, curr, curr[0]); + } + } + + /* Disable pool operation */ + fpavf_write64(~0ul, (void *)((uintptr_t)handle + + FPA_VF_VHPOOL_START_ADDR(gpool))); + fpavf_write64(~0ul, (void *)((uintptr_t)handle + + FPA_VF_VHPOOL_END_ADDR(gpool))); + + (void)octeontx_fpapf_pool_destroy(gpool); + + /* Deactivate the AURA */ + fpavf_write64(0, (void *)((uintptr_t)handle + + FPA_VF_VHAURA_CNT_LIMIT(gpool))); + fpavf_write64(0, (void *)((uintptr_t)handle + + FPA_VF_VHAURA_CNT_THRESHOLD(gpool))); + + ret = octeontx_fpapf_aura_detach(gpool); + if (ret) { + fpavf_log_err("Failed to dettach gaura %u. error code=%d\n", + gpool, ret); + } + + /* Free VF */ + (void)octeontx_fpavf_free(gpool); + + rte_spinlock_unlock(&fpadev.lock); + return 0; +} + static void octeontx_fpavf_setup(void) { diff --git a/drivers/mempool/octeontx/octeontx_fpavf.h b/drivers/mempool/octeontx/octeontx_fpavf.h index 3e8a2682f..936276715 100644 --- a/drivers/mempool/octeontx/octeontx_fpavf.h +++ b/drivers/mempool/octeontx/octeontx_fpavf.h @@ -135,5 +135,7 @@ octeontx_fpa_bufpool_create(unsigned int object_size, unsigned int object_count, unsigned int buf_offset, char **va_start, int node); int +octeontx_fpa_bufpool_destroy(uintptr_t handle, int node); +int octeontx_fpa_bufpool_block_size(uintptr_t handle); #endif /* __OCTEONTX_FPAVF_H__ */ diff --git a/drivers/mempool/octeontx/rte_mempool_octeontx.c b/drivers/mempool/octeontx/rte_mempool_octeontx.c index 73648aa7f..6754a78c0 100644 --- a/drivers/mempool/octeontx/rte_mempool_octeontx.c +++ b/drivers/mempool/octeontx/rte_mempool_octeontx.c @@ -74,10 +74,20 @@ octeontx_fpavf_alloc(struct rte_mempool *mp) return rc; } +static void +octeontx_fpavf_free(struct rte_mempool *mp) +{ + uintptr_t pool; + + pool = (uintptr_t)mp->pool_id; + + octeontx_fpa_bufpool_destroy(pool, mp->socket_id); +} + static struct rte_mempool_ops octeontx_fpavf_ops = { .name = "octeontx_fpavf", .alloc = octeontx_fpavf_alloc, - .free = NULL, + .free = octeontx_fpavf_free, .enqueue = NULL, .dequeue = NULL, .get_count = NULL,