Message ID | 20220410213550.1733330-1-dkozlyuk@nvidia.com (mailing list archive) |
---|---|
State | Accepted, archived |
Delegated to: | Ferruh Yigit |
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7A5ADA00BE; Sun, 10 Apr 2022 23:36:05 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 12D544068C; Sun, 10 Apr 2022 23:36:05 +0200 (CEST) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2058.outbound.protection.outlook.com [40.107.100.58]) by mails.dpdk.org (Postfix) with ESMTP id 1CD9F40685; Sun, 10 Apr 2022 23:36:03 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dGavWcxj3Je7boX4LQvueVYkA4lxWLtYtjm2ZRZ4RBJQNgFLfWPCh2XBSxHKWcj+4CZYN6Zu2i6adMZqk4Oee7Vw+NwUEmaaXg+QWYlvHrcJmW8xsKXHv6+q0zotNQ+14aMc+VnK+gdWEssO8i71Xu0OYBlR09l0LYumLy2Hkbl3dXF+rahtH7xofjoObMrXBA/TRQP5mYw/UAJP9G1ltHyj4XhzyJt9R+8f2KpLG6VAgWUD4Vo4Aq+oMLYVfIKVHCnWxJZqxaFErcpsl5Cfq1ile0/4vETL/cMddIDWuB8n4CPAPjPcCN+r3Bkmely0St8Ab3ZFh2sD98fyCgVG5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nNQkUW58WtBH/Xu0IZAvfwJvUeclbrE6BZy5//xIl2A=; b=ckPoGyuasaE2hqaWnfdjRALjkhglSRQ1CLSml+nS5q9TA56OQnnOILopNyNJhvv/eXeLyE5UWdwjRVMRhKqtrj0CC2ycJXNVBFQA66k2PHFP2HRgaLB0eiE4vsg+IHTeUdUoW1dP2V5XCx/XTAkA7U2YkxWGEI29lw6gEvoOO5yLeobpz4UquOkcgFsrtlCUuU6OyvcrrMi9rv7A96teg0Rp0KthQ4z/8MMNHJzNpqgTrwc6cGcayIqrAUNT6FA5/4okW1wflC18hybnUFzKejYRafXdUwe3YY79ICRq2lwXtfEewsZCKUTAMEcEV4aWv7aT1i/HMjenhFkKaVVAyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nNQkUW58WtBH/Xu0IZAvfwJvUeclbrE6BZy5//xIl2A=; b=dD15/Awc4g4wdq41aTzcZKbTp2y+yVm4MKXgq90xqRmcsgt5X4Wts7hxkKFgnIXu2FnnaaLX6QYb9Hw8bXouEFB6PocyxBQUBJvixnlsnubNZF8pfdMnoL4bzVkuTzZOSFiwEjkUGaac0vidtwcykEA+GmVprTsSJWNdcW8raQH+bo4+xwqibNYS+O/ZjrAqngPVu0SezbZS9ag0VqXgvCYothm4C1tmbySjqDf6xUzBHu3UXcbbBhvt8nP+2OfairvBme/KFLduASbT30sQEMm2u/qu7TSHIhUwHR9y190E1Ty5OR0ncjxY1SmT6b+t5gvvbDWKp+BCvocpnlbJ7w== Received: from DS7P222CA0007.NAMP222.PROD.OUTLOOK.COM (2603:10b6:8:2e::24) by MW2PR12MB2364.namprd12.prod.outlook.com (2603:10b6:907:b::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.22; Sun, 10 Apr 2022 21:36:00 +0000 Received: from DM6NAM11FT041.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2e:cafe::66) by DS7P222CA0007.outlook.office365.com (2603:10b6:8:2e::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.22 via Frontend Transport; Sun, 10 Apr 2022 21:36:00 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by DM6NAM11FT041.mail.protection.outlook.com (10.13.172.98) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5144.20 via Frontend Transport; Sun, 10 Apr 2022 21:35:59 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Sun, 10 Apr 2022 21:35:59 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Sun, 10 Apr 2022 14:35:58 -0700 Received: from nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Sun, 10 Apr 2022 14:35:56 -0700 From: Dmitry Kozlyuk <dkozlyuk@nvidia.com> To: <dev@dpdk.org> CC: <stable@dpdk.org>, Thomas Monjalon <thomas@monjalon.net>, Ferruh Yigit <ferruh.yigit@intel.com>, Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru> Subject: [PATCH] ethdev: prohibit polling of a stopped queue Date: Mon, 11 Apr 2022 00:35:50 +0300 Message-ID: <20220410213550.1733330-1-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 59a4795e-63fd-4378-a6ca-08da1b3a1ad6 X-MS-TrafficTypeDiagnostic: MW2PR12MB2364:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: <MW2PR12MB236498BC2AF012BBA0313F9DB9EB9@MW2PR12MB2364.namprd12.prod.outlook.com> X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: i+YASAGNetXu+cdfCUEBvBRGB87z5pXLLyJCLwifus7ee86Br0sDafAaOWLs34pc31qS34WU9EYrltnXkr4K0h1RTa0NXf8ajH13L3fpgdHO4THzFtzKeeGgSmfrUN71SlM5L3gE83uGDhh8+qlwmfGLegmsA2zvsH4TK9Xs76f89NjoBND9C90VmKItq5kHvhLST5H/4zq8OVVzcinyh7od7H+X6GAzed3ETA4vG89x0jOzBDGkgXuoN+j67Tpcv48trqWtdP7UQO5Ud8Je4CeLXlfstcxzu4nnHY7E8p+7njSmReh7poRUyxGxNgTKQt1k7bGqlB1nI7nNR/nG2AOc2a6xYHWSLuEgoB+pwlQqIPZAqu+GDdKT2YVc7RUODt5DFFQO4LaioSNCl+uVXIN9twhBpvHc9/57AQb50yCitUOqxwEZeWlTUQIgKNtHU20TjxjqtlRYnZi/khSJCK/3icYh0QEGG/fIjIniW+apsZm0Seojmcgu7AOjwR6KGgdarnV/j+r4gMHubMP219DzE6hWwmSrHENbG+LDIQ9HLdQPyPnjyb8nGy69InCVTUDWeKR7RSXkfT4SplkKmGWGM9f4D9NOBxloFXHJadLpYY2sUhmb+aNnt6/wB3KKB9cBbg4EMDP1XU8SYaVc55Myp3SXcluHPtS86mt1r0rfJY4iW5+1LMnK0+ACAG3RiYQtjoMwsl3q6XYqdNPiQZgo7e/nTh99sWaBu8+TOmHz0Hgm8rkX2CeBOWpz24Fz1u2ti9Y1t+e155tu1Ta09SwWnMKUmnqifyHuEXpIklA= X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(26005)(5660300002)(8936002)(6666004)(426003)(336012)(4326008)(8676002)(6286002)(70586007)(83380400001)(2616005)(86362001)(70206006)(186003)(1076003)(966005)(54906003)(7696005)(316002)(356005)(81166007)(6916009)(2906002)(82310400005)(47076005)(40460700003)(508600001)(36756003)(55016003)(36860700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Apr 2022 21:35:59.7580 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 59a4795e-63fd-4378-a6ca-08da1b3a1ad6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT041.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR12MB2364 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org |
Series |
ethdev: prohibit polling of a stopped queue
|
|
Checks
Context | Check | Description |
---|---|---|
ci/checkpatch | success | coding style OK |
ci/Intel-compilation | success | Compilation OK |
ci/iol-abi-testing | success | Testing PASS |
ci/github-robot: build | success | github build: passed |
ci/iol-aarch64-unit-testing | success | Testing PASS |
ci/iol-x86_64-compile-testing | success | Testing PASS |
ci/iol-x86_64-unit-testing | success | Testing PASS |
ci/iol-aarch64-compile-testing | success | Testing PASS |
ci/intel-Testing | success | Testing PASS |
ci/iol-mellanox-Performance | success | Performance Testing PASS |
ci/iol-intel-Functional | success | Functional Testing PASS |
ci/iol-intel-Performance | success | Performance Testing PASS |
Commit Message
Dmitry Kozlyuk
April 10, 2022, 9:35 p.m. UTC
Whether it is allowed to call Rx/Tx functions for a stopped queue
was undocumented. Some PMDs make this behavior a no-op
either by explicitly checking the queue state
or by the way how their routines are implemented or HW works.
No-op behavior may be convenient for application developers.
But it also means that pollers of stopped queues
would go all the way down to PMD Rx/Tx routines, wasting cycles.
Some PMDs would do a check for the queue state on data path,
even though it may never be needed for a particular application.
Also, use cases for stopping queues or starting them deferred
do not logically require polling stopped queues.
Use case 1: a secondary that was polling the queue has crashed,
the primary is doing a recovery to free all mbufs.
By definition the queue to be restarted is not polled.
Use case 2: deferred queue start or queue reconfiguration.
The polling thread must be synchronized anyway,
because queue start and stop are non-atomic.
Prohibit calling Rx/Tx functions on stopped queues.
Fixes: 0748be2cf9a2 ("ethdev: queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
---
This patch is was originally a part of the series:
http://patchwork.dpdk.org/project/dpdk/patch/20220307125351.697936-3-dkozlyuk@nvidia.com/
The discussion there is summarized in the commit message.
lib/ethdev/rte_ethdev.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Comments
On Mon, Apr 11, 2022 at 12:35:50AM +0300, Dmitry Kozlyuk wrote: > Whether it is allowed to call Rx/Tx functions for a stopped queue > was undocumented. Some PMDs make this behavior a no-op > either by explicitly checking the queue state > or by the way how their routines are implemented or HW works. > > No-op behavior may be convenient for application developers. > But it also means that pollers of stopped queues > would go all the way down to PMD Rx/Tx routines, wasting cycles. > Some PMDs would do a check for the queue state on data path, > even though it may never be needed for a particular application. > Also, use cases for stopping queues or starting them deferred > do not logically require polling stopped queues. > > Use case 1: a secondary that was polling the queue has crashed, > the primary is doing a recovery to free all mbufs. > By definition the queue to be restarted is not polled. > > Use case 2: deferred queue start or queue reconfiguration. > The polling thread must be synchronized anyway, > because queue start and stop are non-atomic. > > Prohibit calling Rx/Tx functions on stopped queues. > > Fixes: 0748be2cf9a2 ("ethdev: queue start and stop") > Cc: stable@dpdk.org > > Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com> Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
10/04/2022 23:35, Dmitry Kozlyuk: > Whether it is allowed to call Rx/Tx functions for a stopped queue > was undocumented. Some PMDs make this behavior a no-op > either by explicitly checking the queue state > or by the way how their routines are implemented or HW works. > > No-op behavior may be convenient for application developers. > But it also means that pollers of stopped queues > would go all the way down to PMD Rx/Tx routines, wasting cycles. > Some PMDs would do a check for the queue state on data path, > even though it may never be needed for a particular application. > Also, use cases for stopping queues or starting them deferred > do not logically require polling stopped queues. > > Use case 1: a secondary that was polling the queue has crashed, > the primary is doing a recovery to free all mbufs. > By definition the queue to be restarted is not polled. > > Use case 2: deferred queue start or queue reconfiguration. > The polling thread must be synchronized anyway, > because queue start and stop are non-atomic. > > Prohibit calling Rx/Tx functions on stopped queues. > > Fixes: 0748be2cf9a2 ("ethdev: queue start and stop") > Cc: stable@dpdk.org > > Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com> > --- > This patch is was originally a part of the series: > http://patchwork.dpdk.org/project/dpdk/patch/20220307125351.697936-3-dkozlyuk@nvidia.com/ > The discussion there is summarized in the commit message. [...] > * rte_eth_rx_queue_setup()), it must call rte_eth_dev_stop() first to stop the > * device and then do the reconfiguration before calling rte_eth_dev_start() > * again. The transmit and receive functions should not be invoked when the > - * device is stopped. > + * device is stopped or when the queue is stopped (for that queue). I think we can make it simpler: The transmit and receive functions should not be invoked when the device or the queue is stopped.
25/04/2022 10:30, Thomas Monjalon: > 10/04/2022 23:35, Dmitry Kozlyuk: > > Whether it is allowed to call Rx/Tx functions for a stopped queue > > was undocumented. Some PMDs make this behavior a no-op > > either by explicitly checking the queue state > > or by the way how their routines are implemented or HW works. > > > > No-op behavior may be convenient for application developers. > > But it also means that pollers of stopped queues > > would go all the way down to PMD Rx/Tx routines, wasting cycles. > > Some PMDs would do a check for the queue state on data path, > > even though it may never be needed for a particular application. > > Also, use cases for stopping queues or starting them deferred > > do not logically require polling stopped queues. > > > > Use case 1: a secondary that was polling the queue has crashed, > > the primary is doing a recovery to free all mbufs. > > By definition the queue to be restarted is not polled. > > > > Use case 2: deferred queue start or queue reconfiguration. > > The polling thread must be synchronized anyway, > > because queue start and stop are non-atomic. > > > > Prohibit calling Rx/Tx functions on stopped queues. > > > > Fixes: 0748be2cf9a2 ("ethdev: queue start and stop") > > Cc: stable@dpdk.org > > > > Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com> > > --- > > This patch is was originally a part of the series: > > http://patchwork.dpdk.org/project/dpdk/patch/20220307125351.697936-3-dkozlyuk@nvidia.com/ > > The discussion there is summarized in the commit message. > [...] > > * rte_eth_rx_queue_setup()), it must call rte_eth_dev_stop() first to stop the > > * device and then do the reconfiguration before calling rte_eth_dev_start() > > * again. The transmit and receive functions should not be invoked when the > > - * device is stopped. > > + * device is stopped or when the queue is stopped (for that queue). > > I think we can make it simpler: > > The transmit and receive functions should not be invoked when the device > or the queue is stopped. No comment after a month. The patch is applied in next-net with the suggested rewording.
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index 04cff8ee10..435720a84e 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -74,7 +74,7 @@ * rte_eth_rx_queue_setup()), it must call rte_eth_dev_stop() first to stop the * device and then do the reconfiguration before calling rte_eth_dev_start() * again. The transmit and receive functions should not be invoked when the - * device is stopped. + * device is stopped or when the queue is stopped (for that queue). * * Please note that some configuration is not stored between calls to * rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration will