From patchwork Wed Jul 6 09:24:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: abhimanyu.saini@xilinx.com X-Patchwork-Id: 113748 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EDB96A0540; Wed, 6 Jul 2022 11:24:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9C1BB40A7F; Wed, 6 Jul 2022 11:24:22 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2055.outbound.protection.outlook.com [40.107.93.55]) by mails.dpdk.org (Postfix) with ESMTP id 9731F40691 for ; Wed, 6 Jul 2022 11:24:20 +0200 (CEST) ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=QLXTh0mi0wWHSMmUgbH5TvGPrJ2/swwwzENLaZ+Zo9WTEuUEqs0B7ShrKKhrAOWASO3rrftCnlJOzgJIe5n908KXgISSM8N28p8xGPjsIlAxG8DQl0grCwaMsOGn4jWW2aEFmIhxGhmUkZ397KypM9DrRAKQGRzHm4u9+P4XGmaIjHxO7fbWdCi+9zma7vVPSwx47Jf1AcpgcvJCWYzKWz7e9mLThr9HHhc7RpLljTq0He/l2ZzFm35Dn81+q8ykcqALTjX/6/OS7UpZKajiyp0rbBMUEsF20qbaVb6yWMRK0Ucdtg+MLhYrTpeDj2TVWiEYPgx1q+7U0aOPfGLV8A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0FfaXCz5mpr1AGyzi91P0hSadW2vdyzx53CW+tpIqpA=; b=nN6sF8f+TqyCDfK20oHfgaMiJOdgCYL3km85wDQMuJ1RDq/l9dOTr0tvm/PEff2AgJmT4NDBoeVMk34gnrHYRwNiCke8d5ahaEAfvB0lJWFQC/mT6fOKLus+dNRVyorWEnpyE3yrSXJt468sSen/46o4sdjWvYroBrvGNFmxUsRTKXhhiTMzGtmmns85mBbVy8wplHLPVF97X3OHSOv/C9PYgFqLwz8gZwzZrj689hx6Lky9MZ2ziprM2SgG7Jbs+3VNgH/JXc63qkyptuC6MSUcs4AJMzRwkg8twFxi8z8sPLkvTYL3XZmXl/CMJEQiTnKDQYRUJcuDtdEMshgJow== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 20.83.241.18) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=fail (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=xilinx.com] dmarc=[1,1,header.from=xilinx.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0FfaXCz5mpr1AGyzi91P0hSadW2vdyzx53CW+tpIqpA=; b=DWG9vi4dK8ilemS66S5ojbCh2AnWmXhVvv2Oa0+Jl9KzYiIIxaCXU/gb9unJK8qmElhOwdIn4KZLeAw+KA2AbJRWZwBnHqY8/8/klmGda51Wfp5Ag1PUEa/dcBhbg/MFRQ0tnH2SJNAw4AavLY7glrIAcEcrlRar0XcsjjoD9RbeBAzHW2bj2tCYDBlQwAnqW71ZprbItsOeA+bHogC+QM8X8p01/ru/nFkPXvOfmCW6zecu45sC4ZSWivdaZgxyScGTmZIkSIu+EK6xJmdoA9Y8Ev+6mQZcBgju5oD4yb6IgSbJLCZ6p6jidKItHXZuOPnfyFKEq24MyGNYg9dQ1A== Received: from MWHPR1601CA0002.namprd16.prod.outlook.com (2603:10b6:300:da::12) by MN2PR12MB3503.namprd12.prod.outlook.com (2603:10b6:208:cf::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.15; Wed, 6 Jul 2022 09:24:18 +0000 Received: from CO1NAM11FT051.eop-nam11.prod.protection.outlook.com (2603:10b6:300:da:cafe::c1) by MWHPR1601CA0002.outlook.office365.com (2603:10b6:300:da::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16 via Frontend Transport; Wed, 6 Jul 2022 09:24:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 20.83.241.18) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=fail action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 20.83.241.18 as permitted sender) receiver=protection.outlook.com; client-ip=20.83.241.18; helo=mailrelay000000.14r1f435wfvunndds3vy4cdalc.xx.internal.cloudapp.net; pr=C Received: from mailrelay000000.14r1f435wfvunndds3vy4cdalc.xx.internal.cloudapp.net (20.83.241.18) by CO1NAM11FT051.mail.protection.outlook.com (10.13.174.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Wed, 6 Jul 2022 09:24:17 +0000 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2048.outbound.protection.outlook.com [104.47.66.48]) by mailrelay000000.14r1f435wfvunndds3vy4cdalc.xx.internal.cloudapp.net (Postfix) with ESMTPS id 7775A3F04A; Wed, 6 Jul 2022 09:24:17 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UWhvF96QJz+tDAdo5FaSQaUCy6ahidXoNOMsLPH35xlZGvf1ylqvg2fs4PnjaVp9030ZzXo0rLCKV9elIJlvg3VxMl2F8+9c1G7Ge6FUZC7vOhPlRsl2WUtwYeuU+neF53VGkAJfiJbiwzEo2D5LPcd3pLXT/8W4WlpJ+BtP36tQiuqIyA8XBEA3DPdgiVt72egGUyMIxilKOSwd8JRlAK6ztbyB+4LxT4rvT7/GPE0FlTpJrOdAmELgePGhBdIpD8CcN1VvUib1jifBvL7aGOeCQXgb2y3lv68lyVc+ZhXYbe7IWpdwPW5MGXzYX9zM3IGW5T3e0UGqhtCcqelovg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0FfaXCz5mpr1AGyzi91P0hSadW2vdyzx53CW+tpIqpA=; b=es9sx4rfGz9Is5M4YfER1DV3j25Up2mOO+isTRajqliVsX9WNDO/tMqQpatGZv3URGdNMnHgFcXCsoZo8zuXbkW8i2ENkJTUrYTF6CfY87Pw7n5RfzaxVLxDW5u+gQU7mITswMDjKR5abeA4sZeV6B8yAwdDU5lHrbneyiwjnFypy5XnQN4aoz3f6kkAeJExR1gSlVfCVl3zdrrcwjGHI7htL813jYu0Sg3JBnpBj+LfEr2q0kNZpJYh5pTlh5X2cTlYQPnhyQjCh+D8fb1j4XG12arVAga4C/grYdT0csaK3q3vmQu/KTPJc8vI2PgVtTI8TDyPkmBQkKJXs5pyKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=dpdk.org smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none Received: from BN6PR17CA0022.namprd17.prod.outlook.com (2603:10b6:404:65::32) by BYAPR02MB5928.namprd02.prod.outlook.com (2603:10b6:a03:123::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.17; Wed, 6 Jul 2022 09:24:14 +0000 Received: from BN1NAM02FT043.eop-nam02.prod.protection.outlook.com (2603:10b6:404:65:cafe::9c) by BN6PR17CA0022.outlook.office365.com (2603:10b6:404:65::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.21 via Frontend Transport; Wed, 6 Jul 2022 09:24:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=xilinx.com; Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by BN1NAM02FT043.mail.protection.outlook.com (10.13.2.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Wed, 6 Jul 2022 09:24:13 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.14; Wed, 6 Jul 2022 02:24:07 -0700 Received: from smtp.xilinx.com (172.19.127.95) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.2176.14 via Frontend Transport; Wed, 6 Jul 2022 02:24:07 -0700 Envelope-to: dev@dpdk.org, chenbo.xia@intel.com, maxime.coquelin@redhat.com, andrew.rybchenko@oktetlabs.ru, absaini@amd.com Received: from [10.170.66.118] (port=46582 helo=xndengvm004118.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1o91GR-000396-6a; Wed, 06 Jul 2022 02:24:07 -0700 From: To: CC: , , , Abhimanyu Saini Subject: [PATCH] vdpa/sfc: resolve race between libvhost and dev_conf Date: Wed, 6 Jul 2022 14:54:01 +0530 Message-ID: <20220706092401.36815-1-asaini@xilinx.com> X-Mailer: git-send-email 2.25.0 MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: df202e23-f65a-443c-bb99-08da5f314d0c X-MS-TrafficTypeDiagnostic: BYAPR02MB5928:EE_|CO1NAM11FT051:EE_|MN2PR12MB3503:EE_ X-MS-Exchange-SenderADCheck: 0 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: YQyh/mB681jg41IctqMOxOZJH6RlQQz3Qq5R7S34Zp6LI613ypi3hz5VhlERjl7l9JF/8ZAxyNX6gg5SjYH5TFkunnAO2/JpC6j6QsiFwIJ1wIh9IK4o47xz7uqGL00ivWxLMnGM8vLjEQXdZCOdtD5eOFgBSmPrq9bdvPTFauvVyY6xxj0CLhNWVtzB8kezJQcxtgVx3ULqIVa2/rUp1V2VT8qgh6bEkFaRinkVSv4IPqlTwrXKivmtyqRtIXOg1l3l5px6HiszPGasxW+Dq8j/nk5vwoQ6ub4SBRwwiHHlcGkJMWIaw2zjzUlCVjiVZBI7HLf3yQq8hbNPeAfoGV6iXsQd+vytn8SNw/hNM0MIWxXwKQTTPDXD83dvkAM8rQpMXaek9ApFYsmKKNcogAVWhEgr+84CB3sR4g0qbOfJPo677Dw6hdEtv8Bln2L+YeVoEXUNLzXJWEkKbDYYgGA+8yDOw0s3eIkJeqrZw0AGia1bL5Ed651lHPFlQbXy4k29tSRRoNuOmUsUMB5xuc3gNoR6JIXOeA9Iomn1H0Va9S+dIoKzNFtkd5WmkqRH5uviv5zThe68IGIZV/5Srx4OqXcVdGAdYiLnc8u3LySz1JerpGcO65az/k43T/GHyzfKs7vdf+W0VTYsckNdcOvWvaaouvKPMvu9oZL2ugR8i0TIEINDo/aT4qWARMQXLHIpsusgUiAjI/P5bLc04vKhXqCaZItL20NbAB9a8kHlALj8P97i/Ktofx6gg2J774Q/Pa7Q5Q2QvskyVmhv1UvH8yw6FhHVzc5fN4tnuozJn2pmzU9xAW7A6vYC3TjrL+35AQHT04Dgv3/ZNPKtZDu/Ox7nWtsOJ58yHndNSCM= X-Forefront-Antispam-Report-Untrusted: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch01.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(13230016)(4636009)(136003)(346002)(376002)(396003)(39860400002)(40470700004)(36840700001)(46966006)(47076005)(6916009)(40460700003)(426003)(336012)(26005)(54906003)(2876002)(82310400005)(186003)(36756003)(316002)(83380400001)(1076003)(9786002)(2616005)(5660300002)(70586007)(7636003)(2906002)(40480700001)(36860700001)(478600001)(7696005)(8936002)(6666004)(8676002)(4326008)(41300700001)(70206006)(356005)(82740400003)(102446001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR02MB5928 X-MS-Exchange-Transport-CrossTenantHeadersStripped: CO1NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: e9aeb428-07f7-40f8-436a-08da5f314ab4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZL15kdjkA6mP1xRKxSDKm7kiK2sy1Xn6m25uHCxpMvOAv8/YNDg1+X1AAHJm8u72rn3Wj0Fcs1g19lMXrYijtB7Lct1G2l1K+HPJochjnh0rVmWnzO5DmkJ2o7OaZOjftAZvAvRo1xwFPq5ioG/sOI7PYF2kqm8GaLZ2gQb8oyqWJflfILJTd6gcWzbUtlxjpWbuVwAnNukS9hEqB5Tr7I7YEe/uYSHfg+msqulm0ue2VNnNBudxs9uBrbjSUfD0L4ChySmRfk0PSIY/LfKLv+XOh5nB3cXDLoFJekMNeLvcoqlVxlc844pCn5WF9q1AYnVrNuZmEp8zPZt9XtXeqebS80r8FR8BuG0nObRaxzYG/a+XbsdC9D5M51Fk+ojY7MqF6P8lYNpdRnaJsTvKhL/FKDZok0vX393FfTvIqkTg+FbdOd9WE6pabh+NRTmoxnxuar/JaUp+SRi+A/zHvkkhSwHPV/shL9nnDvKJXf3x8IZkSlz6AsFJ3p6yu+PM/lM2C94adlWspV5EbBTaakCdEdc1y2UbLhgplhPeibrxNb1NZ1xdL3Bl//fBIV9rGC5JGVLQ0L1kABimJPkUN8U3iVL932PW0lLDwdNVA8KfpevQEhucjexsxcG1G1YEcUKbq2wcN3R9H0HvZJCItw05UT8igZ2IIgueEZfzQCvvNAoDZ0DtO9z97AohQUg3t23i7txzUvtc4m9P54SBHmn0u5lKKstiU7gAD21cTbJ2Dr2dSfgyiaAGnPZ2xzQZkV6Igbqsou67xjgT8jLsjvO32mdFVF8o78dy8mtN4v+aD3HQA0ohd1DV8oUvcJIEDPhbtLsPdCYc+P0AfBuq8wHG7i2Kb+NGhJnmgGWonjOj6Aeqnaos6j9IbdBcRuDgpCuBnS5OeEIkFoSPNDur4pGfRIZm/qNFm5oXPUPDc4g= X-Forefront-Antispam-Report: CIP:20.83.241.18; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mailrelay000000.14r1f435wfvunndds3vy4cdalc.xx.internal.cloudapp.net; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230016)(4636009)(376002)(39860400002)(346002)(396003)(136003)(46966006)(36840700001)(2906002)(36860700001)(41300700001)(34020700004)(2616005)(6666004)(40480700001)(1076003)(82310400005)(316002)(36756003)(2876002)(426003)(54906003)(6916009)(7696005)(70206006)(8676002)(4326008)(9786002)(47076005)(478600001)(186003)(42882007)(83380400001)(336012)(82740400003)(81166007)(83170400001)(26005)(8936002)(5660300002)(102446001)(84790400001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2022 09:24:17.6594 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: df202e23-f65a-443c-bb99-08da5f314d0c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[20.83.241.18]; Helo=[mailrelay000000.14r1f435wfvunndds3vy4cdalc.xx.internal.cloudapp.net] X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TreatMessagesAsInternal-CO1NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3503 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Abhimanyu Saini libvhost calls dev_conf() before prosessing the VHOST_USER_SET_VRING_CALL message for the last VQ. So this message is processed after dev_conf() returns. However, the dev_conf() function spawns a thread to set rte_vhost_host_notifier_ctrl() before returning control to libvhost. This parallel thread in turn invokes get_notify_area(). To get the notify_area, the vdpa driver needs to query the HW and for this query it needs an enabled VQ. But at the same time libvhost is processing the last VHOST_USER_SET_VRING_CALL, and to do that it disables the last VQ. Hence there is a race b/w the libvhost and the vdpa driver. To resolve this race condition, query the HW and cache notify_area inside dev_conf() instead of doing it the parallel thread. Signed-off-by: Abhimanyu Saini Reviewed-by: Maxime Coquelin Acked-by: Vijay Srivastava --- drivers/vdpa/sfc/sfc_vdpa_ops.c | 36 ++++++++++++++++++------------------ drivers/vdpa/sfc/sfc_vdpa_ops.h | 1 + 2 files changed, 19 insertions(+), 18 deletions(-) diff --git a/drivers/vdpa/sfc/sfc_vdpa_ops.c b/drivers/vdpa/sfc/sfc_vdpa_ops.c index 63aa52d..b84699d 100644 --- a/drivers/vdpa/sfc/sfc_vdpa_ops.c +++ b/drivers/vdpa/sfc/sfc_vdpa_ops.c @@ -222,6 +222,7 @@ sfc_vdpa_virtq_start(struct sfc_vdpa_ops_data *ops_data, int vq_num) { int rc; + uint32_t doorbell; efx_virtio_vq_t *vq; struct sfc_vdpa_vring_info vring; efx_virtio_vq_cfg_t vq_cfg; @@ -270,22 +271,35 @@ /* Start virtqueue */ rc = efx_virtio_qstart(vq, &vq_cfg, &vq_dyncfg); if (rc != 0) { - /* destroy virtqueue */ sfc_vdpa_err(ops_data->dev_handle, "virtqueue start failed: %s", rte_strerror(rc)); - efx_virtio_qdestroy(vq); goto fail_virtio_qstart; } sfc_vdpa_info(ops_data->dev_handle, "virtqueue started successfully for vq_num %d", vq_num); + rc = efx_virtio_get_doorbell_offset(vq, &doorbell); + if (rc != 0) { + sfc_vdpa_err(ops_data->dev_handle, + "failed to get doorbell offset: %s", + rte_strerror(rc)); + goto fail_doorbell; + } + + /* + * Cache the bar_offset here for each VQ here, it will come + * in handy when sfc_vdpa_get_notify_area() is invoked. + */ + ops_data->vq_cxt[vq_num].doorbell = (void *)(uintptr_t)doorbell; ops_data->vq_cxt[vq_num].enable = B_TRUE; return rc; +fail_doorbell: fail_virtio_qstart: + efx_virtio_qdestroy(vq); fail_vring_info: return rc; } @@ -792,8 +806,6 @@ int ret; efx_nic_t *nic; int vfio_dev_fd; - efx_rc_t rc; - unsigned int bar_offset; volatile void *doorbell; struct rte_pci_device *pci_dev; struct rte_vdpa_device *vdpa_dev; @@ -824,19 +836,6 @@ return -1; } - if (ops_data->vq_cxt[qid].enable != B_TRUE) { - sfc_vdpa_err(dev, "vq is not enabled"); - return -1; - } - - rc = efx_virtio_get_doorbell_offset(ops_data->vq_cxt[qid].vq, - &bar_offset); - if (rc != 0) { - sfc_vdpa_err(dev, "failed to get doorbell offset: %s", - rte_strerror(rc)); - return rc; - } - reg.index = sfc_vdpa_adapter_by_dev_handle(dev)->mem_bar.esb_rid; ret = ioctl(vfio_dev_fd, VFIO_DEVICE_GET_REGION_INFO, ®); if (ret != 0) { @@ -845,7 +844,8 @@ return ret; } - *offset = reg.offset + bar_offset; + /* Use bar_offset that was cached during sfc_vdpa_virtq_start() */ + *offset = reg.offset + (uint64_t)ops_data->vq_cxt[qid].doorbell; len = (1U << encp->enc_vi_window_shift) / 2; if (len >= sysconf(_SC_PAGESIZE)) { diff --git a/drivers/vdpa/sfc/sfc_vdpa_ops.h b/drivers/vdpa/sfc/sfc_vdpa_ops.h index 6d790fd..9dbd5b8 100644 --- a/drivers/vdpa/sfc/sfc_vdpa_ops.h +++ b/drivers/vdpa/sfc/sfc_vdpa_ops.h @@ -35,6 +35,7 @@ struct sfc_vdpa_vring_info { }; typedef struct sfc_vdpa_vq_context_s { + volatile void *doorbell; uint8_t enable; uint32_t pidx; uint32_t cidx;