From patchwork Tue Jan 11 13:41:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 105736 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 34DC0A034F; Tue, 11 Jan 2022 14:41:41 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2343B42708; Tue, 11 Jan 2022 14:41:41 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 0111441143; Tue, 11 Jan 2022 14:41:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1641908499; x=1673444499; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z0N9Bxnw3A4KEGlkTED7T3U6YuuleNducmrX5TimkxI=; b=YCtpYMNfs9B4E84hAA4Cj88IlnDup1706fefhSuLLnZ1bW903U6OExAo X/qfH9CUQsqAfDiZLjVrBDTs0P369nxb+DOCBYJP2tPuxRrxU3wl/e9oY /4ABXyKtFZiM0bW4FSHgs7p6T3nmSEQC0c1xKQrBcG11yr3bqZNLyPZ/C wlUaKpiQTZqSd8GvpxCYa5UC8ji8T3pDV442dfCFg9+wABHjaGazK5PkR OnHFYjl1fq46TiGcsU1xwecWEYWGGwLalDBnowwLN4djiVE6Q8Df9sEHh l8wJAo3qLzSz5XzxrzvcM3KfgpDb5vlgnT1VOSJN6rdaRfgrQ2fmFLjDC Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10223"; a="306836000" X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="306836000" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2022 05:41:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="690997103" Received: from silpixa00399126.ir.intel.com ([10.237.223.86]) by orsmga005.jf.intel.com with ESMTP; 11 Jan 2022 05:41:36 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , kevin.laatz@intel.com, stable@dpdk.org, Jiayu Hu , Conor Walsh , Chengwen Feng Subject: [PATCH v2 1/4] dma/idxd: fix burst capacity calculation Date: Tue, 11 Jan 2022 13:41:02 +0000 Message-Id: <20220111134105.1007191-2-bruce.richardson@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220111134105.1007191-1-bruce.richardson@intel.com> References: <20211220170514.736732-1-bruce.richardson@intel.com> <20220111134105.1007191-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When the maximum burst size supported by HW is less than the available ring space, incorrect capacity was returned when there was already some jobs queued up for submission. This was because the capacity calculation failed to subtract the number of already-enqueued jobs from the max burst size. After subtraction is done, ensure that any negative values (which should never occur if the user respects the reported limits), are clamped to zero. Fixes: 9459de4edc99 ("dma/idxd: add burst capacity") Cc: kevin.laatz@intel.com Cc: stable@dpdk.org Signed-off-by: Bruce Richardson Acked-by: Kevin Laatz Tested-by: Jiayu Hu --- drivers/dma/idxd/idxd_common.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index fc11b11337..4442d1cbbd 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -485,7 +485,9 @@ idxd_burst_capacity(const void *dev_private, uint16_t vchan __rte_unused) write_idx += idxd->desc_ring_mask + 1; used_space = write_idx - idxd->ids_returned; - return RTE_MIN((idxd->desc_ring_mask - used_space), idxd->max_batch_size); + const int ret = RTE_MIN((idxd->desc_ring_mask - used_space), + (idxd->max_batch_size - idxd->batch_size)); + return ret < 0 ? 0 : (uint16_t)ret; } int From patchwork Tue Jan 11 13:41:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 105737 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5D2A1A034F; Tue, 11 Jan 2022 14:41:55 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 512504270F; Tue, 11 Jan 2022 14:41:55 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id F1B1941143; Tue, 11 Jan 2022 14:41:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1641908513; x=1673444513; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uLS6kz0NEeD9xqA2Bpcqnu94bW5O2uph9CG6Q8zn5b4=; b=AUidnZoDrdH9TTh/N4/w05XXJPIBI/6B9t7vgTe+JaWL5qAsQF3hbxFa VUF3lK6yXHvmtvyiY9bQHA+tLBsT8CqE3T5iOWwgvM0ANtbKo87k2rrtB 7LIOmNGrEHLauusRgZFBGZ/MIS2EIVjjZ4pkpDJeUZH9TXxI13uOkQCrZ 0nZEayGo6w9TTz6G16jq72NFfDDlj4S2bHwLarp68LcRZgNYLXt7sIBmQ sEMv5SldKhB4lUZ1SvppBtfU+OBC1taBYUmBhEZNYoc8S0IVi0RvtqcI7 9q7wncR0UvH2H6D4ESEu0f3FzHhN+k7ih9Ldjfae87zzUEkOyrPlF9G+g Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10223"; a="306836040" X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="306836040" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2022 05:41:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="690997178" Received: from silpixa00399126.ir.intel.com ([10.237.223.86]) by orsmga005.jf.intel.com with ESMTP; 11 Jan 2022 05:41:50 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , stable@dpdk.org, Kevin Laatz , Radu Nicolau Subject: [PATCH v2 2/4] dma/idxd: fix paths to driver sysfs directory Date: Tue, 11 Jan 2022 13:41:03 +0000 Message-Id: <20220111134105.1007191-3-bruce.richardson@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220111134105.1007191-1-bruce.richardson@intel.com> References: <20211220170514.736732-1-bruce.richardson@intel.com> <20220111134105.1007191-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Recent kernel changes[1][2] mean that we cannot guarantee that the paths in sysfs used for creating/binding a DSA or workqueue instance will be as given in the utility script, since they are now "compatibility-mode only". Update script to support both new paths and compatibility ones. [1] https://lore.kernel.org/all/162637445139.744545.6008938867943724701.stgit@djiang5-desk3.ch.intel.com/ [2] https://lore.kernel.org/all/162637468705.744545.4399080971745974435.stgit@djiang5-desk3.ch.intel.com/ Fixes: 01863b9d2354 ("raw/ioat: include example configuration script") Cc: stable@dpdk.org Signed-off-by: Bruce Richardson Acked-by: Kevin Laatz --- drivers/dma/idxd/dpdk_idxd_cfg.py | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/drivers/dma/idxd/dpdk_idxd_cfg.py b/drivers/dma/idxd/dpdk_idxd_cfg.py index fcc27822ef..34537cb980 100755 --- a/drivers/dma/idxd/dpdk_idxd_cfg.py +++ b/drivers/dma/idxd/dpdk_idxd_cfg.py @@ -29,9 +29,17 @@ def write_values(self, values): f.write(str(contents)) +def get_drv_dir(dtype): + "Get the sysfs path for the driver, either 'idxd' or 'user'" + drv_dir = "/sys/bus/dsa/drivers/" + dtype + if not os.path.exists(drv_dir): + return "/sys/bus/dsa/drivers/dsa" + return drv_dir + + def reset_device(dsa_id): "Reset the DSA device and all its queues" - drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") + drv_dir = SysfsDir(get_drv_dir("idxd")) drv_dir.write_values({"unbind": f"dsa{dsa_id}"}) @@ -58,7 +66,6 @@ def get_dsa_id(pci): def configure_dsa(dsa_id, queues, prefix): "Configure the DSA instance with appropriate number of queues" dsa_dir = SysfsDir(f"/sys/bus/dsa/devices/dsa{dsa_id}") - drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") max_groups = dsa_dir.read_int("max_groups") max_engines = dsa_dir.read_int("max_engines") @@ -85,9 +92,12 @@ def configure_dsa(dsa_id, queues, prefix): "size": int(max_work_queues_size / nb_queues)}) # enable device and then queues - drv_dir.write_values({"bind": f"dsa{dsa_id}"}) + idxd_dir = SysfsDir(get_drv_dir("idxd")) + idxd_dir.write_values({"bind": f"dsa{dsa_id}"}) + + user_dir = SysfsDir(get_drv_dir("user")) for q in range(nb_queues): - drv_dir.write_values({"bind": f"wq{dsa_id}.{q}"}) + user_dir.write_values({"bind": f"wq{dsa_id}.{q}"}) def main(args): From patchwork Tue Jan 11 13:41:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 105738 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 69A05A034F; Tue, 11 Jan 2022 14:42:03 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4F46942709; Tue, 11 Jan 2022 14:42:03 +0100 (CET) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id B3D3341143; Tue, 11 Jan 2022 14:42:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1641908521; x=1673444521; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Km6+KRQAQ5Va+gNSXtAiMWR8aiwowM9k/oDsf/gHcaI=; b=N6k8qv8zP7T95xSvFFLB64XIqkqUygnAZUkx97AJ0rhJOFLax8iZjVuR KpwI/vxTLgBQxiWhzzoEEMr0euHtwTVtp7q2+A7sZyftRpqzNOfZG500v cxINCFcla62ySgzKWNhhZiVTj5qGOghmkQpJsJaddfq/GF4Zshb2msuMQ tMdxrJRTJ0jlEnm7bONwb1YwFO2wexwJEpWeGUPKV08JDg1CtWDrJ3XiV UOfMYAcK8Z1uPaaXu6QAvNQPsUcupTHGswnOu5zi9pbIDcJ7F39g5/1Z3 83+2RHu24ZgngNuaabw1GJ/QrVUL9O3EjrQ5Cb1NwwmvcSKo3RqGj37Xx w==; X-IronPort-AV: E=McAfee;i="6200,9189,10223"; a="223467476" X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="223467476" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2022 05:41:59 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="690997229" Received: from silpixa00399126.ir.intel.com ([10.237.223.86]) by orsmga005.jf.intel.com with ESMTP; 11 Jan 2022 05:41:57 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , kevin.laatz@intel.com, stable@dpdk.org, Sunil Pai G , Conor Walsh , Chengwen Feng Subject: [PATCH v2 3/4] dma/idxd: fix wrap-around in burst capacity calculation Date: Tue, 11 Jan 2022 13:41:04 +0000 Message-Id: <20220111134105.1007191-4-bruce.richardson@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220111134105.1007191-1-bruce.richardson@intel.com> References: <20211220170514.736732-1-bruce.richardson@intel.com> <20220111134105.1007191-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The burst capacity calculation code assumes that the write and read (i.e. ids_returned) values both wrap at the ring-size, but the read value instead wraps as UINT16_MAX. Therefore, instead of just adding ring-size to the write value in case the read is greater, we need to just always mask the result to ensure a correct, in-range, value. Fixes: 9459de4edc99 ("dma/idxd: add burst capacity") Cc: kevin.laatz@intel.com Cc: stable@dpdk.org Reported-by: Sunil Pai G Signed-off-by: Bruce Richardson Tested-by: Sunil Pai G Acked-by: Kevin Laatz --- drivers/dma/idxd/idxd_common.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 4442d1cbbd..ea6413cc7a 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -480,10 +480,8 @@ idxd_burst_capacity(const void *dev_private, uint16_t vchan __rte_unused) idxd->batch_idx_write + 1 == idxd->batch_idx_read) return 0; - /* For descriptors, check for wrap-around on write but not read */ - if (idxd->ids_returned > write_idx) - write_idx += idxd->desc_ring_mask + 1; - used_space = write_idx - idxd->ids_returned; + /* Subtract and mask to get in correct range */ + used_space = (write_idx - idxd->ids_returned) & idxd->desc_ring_mask; const int ret = RTE_MIN((idxd->desc_ring_mask - used_space), (idxd->max_batch_size - idxd->batch_size)); From patchwork Tue Jan 11 13:41:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 105739 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E816FA034F; Tue, 11 Jan 2022 14:42:08 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 588D742713; Tue, 11 Jan 2022 14:42:07 +0100 (CET) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 30F2741143 for ; Tue, 11 Jan 2022 14:42:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1641908525; x=1673444525; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mnU0RH389KKJ+xurCwvI5HjJXJyeEO1DGCaKdJwUtKc=; b=FOAFghsVUOek0VMxgELB+TAbEZ982uvDtOppr3hvzl6q+vjzgk9bJ9LB XB/tHwuzwkaR13LyEclXhBXpqPAUdlEwRsdDSc3orQtSZH3shnWyHjPzd nnZiF86unf9qMRgz3fjdsO667JfcHA1t703SUUf8K5y9YzdNx5g1EN7BW lDu9gWXdr8P9OSIn7sPyHY1T2HeXRx2Ow5Ylt2nkJGmevCK/ms6yW2OKu Xq+4IBkDnDudtkjHL9FJ6F8JLVFCL+axjvmfj98WjVubDIYwAyyEaz018 HmZt07hideJuxL72pZr64olfO54u4ePt0ppdDD1J1JHji7rQ+vwdIz0qH w==; X-IronPort-AV: E=McAfee;i="6200,9189,10223"; a="223467492" X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="223467492" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2022 05:42:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="690997282" Received: from silpixa00399126.ir.intel.com ([10.237.223.86]) by orsmga005.jf.intel.com with ESMTP; 11 Jan 2022 05:42:03 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Chengwen Feng , Kevin Laatz Subject: [PATCH v2 4/4] test_dmadev: increase iterations of capacity test case Date: Tue, 11 Jan 2022 13:41:05 +0000 Message-Id: <20220111134105.1007191-5-bruce.richardson@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220111134105.1007191-1-bruce.richardson@intel.com> References: <20211220170514.736732-1-bruce.richardson@intel.com> <20220111134105.1007191-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org To ensure we catch any bugs in calculation due to wrap-around of the id values, increase the number of iterations of the burst_capacity test. Signed-off-by: Bruce Richardson Acked-by: Kevin Laatz --- app/test/test_dmadev.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c index b206db27ae..db5aff701c 100644 --- a/app/test/test_dmadev.c +++ b/app/test/test_dmadev.c @@ -686,10 +686,11 @@ test_burst_capacity(int16_t dev_id, uint16_t vchan) /* to test capacity, we enqueue elements and check capacity is reduced * by one each time - rebaselining the expected value after each burst * as the capacity is only for a burst. We enqueue multiple bursts to - * fill up half the ring, before emptying it again. We do this twice to - * ensure that we get to test scenarios where we get ring wrap-around + * fill up half the ring, before emptying it again. We do this multiple + * times to ensure that we get to test scenarios where we get ring + * wrap-around and wrap-around of the ids returned (at UINT16_MAX). */ - for (iter = 0; iter < 2; iter++) { + for (iter = 0; iter < 2 * (((int)UINT16_MAX + 1) / ring_space); iter++) { for (i = 0; i < (ring_space / (2 * CAP_TEST_BURST_SIZE)) + 1; i++) { cap = rte_dma_burst_capacity(dev_id, vchan);