From patchwork Mon Jan 24 12:37:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumara Parameshwaran X-Patchwork-Id: 106346 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B6E5AA04A7; Mon, 24 Jan 2022 13:37:52 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 26074427BE; Mon, 24 Jan 2022 13:37:52 +0100 (CET) Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by mails.dpdk.org (Postfix) with ESMTP id 0709740E0F; Mon, 24 Jan 2022 13:37:50 +0100 (CET) Received: by mail-pf1-f170.google.com with SMTP id u130so11580552pfc.2; Mon, 24 Jan 2022 04:37:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ybKbmSLO3k7ujXGRm+Ope+TvpWfTfqiA5eneYmKsJT4=; b=Bok1StKCr5awkx65DL+cUTYFtJbgg6pQK38ikDtaBRNaHgrm6eUsL7iyKjihWA8Cqd 448XePTkDGN9vlhvyv+8kyzvlJ4FnWgpteJ25CQyFfWU/kkBVY64HJXD/ynbMNKw2o02 OESSGspfD0UcyzEwbMOn1KXzPeuhMQRoE49W2OBwcK4LGFuL1aJ896vnRpNy6Xg8Pla9 k3hg17huNMgIEFIvkXeJfKYXxuAUHx0gxk6vz9gPR/SMcrLwk2ML+pxRrWpYe0ipHMIE 4Cfp0mH/OmhwdGeFrOaOsAHiEU9qivX2KrWDG00G3pwvs4jjkvdx0f7WwAKKp5+KC2Kk rlyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ybKbmSLO3k7ujXGRm+Ope+TvpWfTfqiA5eneYmKsJT4=; b=UUv00SSZnVms+wHWO0GdWq8yMNa213hbSdxF5RF2q40IwRFiPQYT6XBFyx+yw6Stg7 RYqZEdq39Escso3ulwR4zTBQmxIHQL0bZsGZtbQ0MqUW6oPTBDx8oWDUvYpEup/+mHmf xHCg4oCfLBS1slOULuYkhzYTqUN/GAf1E4ePgZN0apvehbmlRRLLE+FBSVOVomoq9FZT bxT7JGRupR3vh60+qBtO7MEyDrfCX8c6FpAaGr37zqOJNzKIUno2+Xo+TPuGhQ0LPESH ik2E2clf01ZFi8dxPnnlDm8xc6NjqgwWGfx9C9oCXP8wWQ+AghiZmP5Vq32gBstd+/YK x4bQ== X-Gm-Message-State: AOAM530w9ZKvgYv79uT9B6Mk8a19tr39TfnBYDicovRCyt0S3ghKEHo9 xfUzVxVADkGLvDEneNAdFf3duQmmvsl5g9MyD3U= X-Google-Smtp-Source: ABdhPJwAf4gfqRB1P5qYlrDs5GPFsRVtS6g5jzRbS8fRSXsd9noIUX09iFQDN5ozumau8L9ciA+glg== X-Received: by 2002:a62:79c7:0:b0:4c7:f17e:3ad8 with SMTP id u190-20020a6279c7000000b004c7f17e3ad8mr8975944pfc.21.1643027868753; Mon, 24 Jan 2022 04:37:48 -0800 (PST) Received: from kparameshwa-a01.vmware.com.com ([49.206.13.78]) by smtp.gmail.com with ESMTPSA id h6sm4998895pfc.35.2022.01.24.04.37.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jan 2022 04:37:48 -0800 (PST) From: Kumara Parameshwaran To: dev@dpdk.org Cc: ferruh.yigit@intel.com, Kumara Parameshwaran , stable@dpdk.org Subject: [PATCH v2] net/tap: fix to populate fds in secondary process Date: Mon, 24 Jan 2022 18:07:43 +0530 Message-Id: <20220124123743.70471-1-kumaraparamesh92@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20220121042944.23929-1-kumaraparamesh92@gmail.com> References: <20220121042944.23929-1-kumaraparamesh92@gmail.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Kumara Parameshwaran When a tap device is hotplugged to primary process which in turn adds the device to all secondary process, the secondary process does a tap_mp_attach_queues, but the fds are not populated in the primary during the probe they are populated during the queue_setup, added a fix to sync the queues during rte_eth_dev_start Fixes: 4852aa8f6e21 ("drivers/net: enable hotplug on secondary process") Cc: stable@dpdk.org Signed-off-by: Kumara Parameshwaran --- drivers/net/tap/rte_eth_tap.c | 196 +++++++++++++--------------------- lib/ethdev/ethdev_driver.h | 18 ++++ lib/ethdev/rte_ethdev.c | 11 ++ lib/ethdev/version.map | 1 + 4 files changed, 102 insertions(+), 124 deletions(-) diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index f1b48cae82..f6c25d7e21 100644 --- a/drivers/net/tap/rte_eth_tap.c +++ b/drivers/net/tap/rte_eth_tap.c @@ -66,7 +66,7 @@ (TAP_GSO_MBUFS_PER_CORE * TAP_GSO_MBUF_CACHE_SIZE) /* IPC key for queue fds sync */ -#define TAP_MP_KEY "tap_mp_sync_queues" +#define TAP_MP_REQ_START_RXTX "tap_mp_req_start_rxtx" #define TAP_IOV_DEFAULT_MAX 1024 @@ -880,11 +880,48 @@ tap_link_set_up(struct rte_eth_dev *dev) return tap_ioctl(pmd, SIOCSIFFLAGS, &ifr, 1, LOCAL_AND_REMOTE); } +static int tap_mp_req_on_rxtx(struct rte_eth_dev *dev) +{ + struct rte_mp_msg msg; + struct ipc_queues *request_param = (struct ipc_queues *)msg.param; + int err; + int fd_iterator = 0; + struct pmd_process_private *process_private = dev->process_private; + int i; + + memset(&msg, 0, sizeof(msg)); + strlcpy(msg.name, TAP_MP_REQ_START_RXTX, sizeof(msg.name)); + strlcpy(request_param->port_name, dev->data->name, sizeof(request_param->port_name)); + msg.len_param = sizeof(*request_param); + for (i = 0; i < dev->data->nb_tx_queues; i++) { + msg.fds[fd_iterator++] = process_private->txq_fds[i]; + msg.num_fds++; + request_param->txq_count++; + } + for (i = 0; i < dev->data->nb_rx_queues; i++) { + msg.fds[fd_iterator++] = process_private->rxq_fds[i]; + msg.num_fds++; + request_param->rxq_count++; + } + + err = rte_mp_sendmsg(&msg); + if (err < 0) { + TAP_LOG(ERR, "Failed to send start req to secondary %d", + rte_errno); + return -1; + } + + return 0; +} + static int tap_dev_start(struct rte_eth_dev *dev) { int err, i; + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + tap_mp_req_on_rxtx(dev); + err = tap_intr_handle_set(dev, 1); if (err) return err; @@ -901,6 +938,34 @@ tap_dev_start(struct rte_eth_dev *dev) return err; } +static int +tap_mp_req_start_rxtx(const struct rte_mp_msg *request, __rte_unused const void *peer) +{ + struct rte_eth_dev *dev; + const struct ipc_queues *request_param = + (const struct ipc_queues *)request->param; + int fd_iterator; + int queue; + struct pmd_process_private *process_private; + + dev = rte_get_eth_dev_by_name(request_param->port_name); + if (!dev) { + TAP_LOG(ERR, "Failed to get dev for %s", + request_param->port_name); + return -1; + } + process_private = dev->process_private; + fd_iterator = 0; + TAP_LOG(DEBUG, "tap_attach rx_q:%d tx_q:%d\n", request_param->rxq_count, + request_param->txq_count); + for (queue = 0; queue < request_param->txq_count; queue++) + process_private->txq_fds[queue] = request->fds[fd_iterator++]; + for (queue = 0; queue < request_param->rxq_count; queue++) + process_private->rxq_fds[queue] = request->fds[fd_iterator++]; + + return 0; +} + /* This function gets called when the current port gets stopped. */ static int @@ -1084,6 +1149,7 @@ tap_dev_close(struct rte_eth_dev *dev) if (rte_eal_process_type() != RTE_PROC_PRIMARY) { rte_free(dev->process_private); + rte_mp_action_unregister(TAP_MP_REQ_START_RXTX); return 0; } @@ -1140,8 +1206,6 @@ tap_dev_close(struct rte_eth_dev *dev) internals->ioctl_sock = -1; } rte_free(dev->process_private); - if (tap_devices_count == 1) - rte_mp_action_unregister(TAP_MP_KEY); tap_devices_count--; /* * Since TUN device has no more opened file descriptors @@ -2292,113 +2356,6 @@ rte_pmd_tun_probe(struct rte_vdev_device *dev) return ret; } -/* Request queue file descriptors from secondary to primary. */ -static int -tap_mp_attach_queues(const char *port_name, struct rte_eth_dev *dev) -{ - int ret; - struct timespec timeout = {.tv_sec = 1, .tv_nsec = 0}; - struct rte_mp_msg request, *reply; - struct rte_mp_reply replies; - struct ipc_queues *request_param = (struct ipc_queues *)request.param; - struct ipc_queues *reply_param; - struct pmd_process_private *process_private = dev->process_private; - int queue, fd_iterator; - - /* Prepare the request */ - memset(&request, 0, sizeof(request)); - strlcpy(request.name, TAP_MP_KEY, sizeof(request.name)); - strlcpy(request_param->port_name, port_name, - sizeof(request_param->port_name)); - request.len_param = sizeof(*request_param); - /* Send request and receive reply */ - ret = rte_mp_request_sync(&request, &replies, &timeout); - if (ret < 0 || replies.nb_received != 1) { - TAP_LOG(ERR, "Failed to request queues from primary: %d", - rte_errno); - return -1; - } - reply = &replies.msgs[0]; - reply_param = (struct ipc_queues *)reply->param; - TAP_LOG(DEBUG, "Received IPC reply for %s", reply_param->port_name); - - /* Attach the queues from received file descriptors */ - if (reply_param->rxq_count + reply_param->txq_count != reply->num_fds) { - TAP_LOG(ERR, "Unexpected number of fds received"); - return -1; - } - - dev->data->nb_rx_queues = reply_param->rxq_count; - dev->data->nb_tx_queues = reply_param->txq_count; - fd_iterator = 0; - for (queue = 0; queue < reply_param->rxq_count; queue++) - process_private->rxq_fds[queue] = reply->fds[fd_iterator++]; - for (queue = 0; queue < reply_param->txq_count; queue++) - process_private->txq_fds[queue] = reply->fds[fd_iterator++]; - free(reply); - return 0; -} - -/* Send the queue file descriptors from the primary process to secondary. */ -static int -tap_mp_sync_queues(const struct rte_mp_msg *request, const void *peer) -{ - struct rte_eth_dev *dev; - struct pmd_process_private *process_private; - struct rte_mp_msg reply; - const struct ipc_queues *request_param = - (const struct ipc_queues *)request->param; - struct ipc_queues *reply_param = - (struct ipc_queues *)reply.param; - uint16_t port_id; - int queue; - int ret; - - /* Get requested port */ - TAP_LOG(DEBUG, "Received IPC request for %s", request_param->port_name); - ret = rte_eth_dev_get_port_by_name(request_param->port_name, &port_id); - if (ret) { - TAP_LOG(ERR, "Failed to get port id for %s", - request_param->port_name); - return -1; - } - dev = &rte_eth_devices[port_id]; - process_private = dev->process_private; - - /* Fill file descriptors for all queues */ - reply.num_fds = 0; - reply_param->rxq_count = 0; - if (dev->data->nb_rx_queues + dev->data->nb_tx_queues > - RTE_MP_MAX_FD_NUM){ - TAP_LOG(ERR, "Number of rx/tx queues exceeds max number of fds"); - return -1; - } - - for (queue = 0; queue < dev->data->nb_rx_queues; queue++) { - reply.fds[reply.num_fds++] = process_private->rxq_fds[queue]; - reply_param->rxq_count++; - } - RTE_ASSERT(reply_param->rxq_count == dev->data->nb_rx_queues); - - reply_param->txq_count = 0; - for (queue = 0; queue < dev->data->nb_tx_queues; queue++) { - reply.fds[reply.num_fds++] = process_private->txq_fds[queue]; - reply_param->txq_count++; - } - RTE_ASSERT(reply_param->txq_count == dev->data->nb_tx_queues); - - /* Send reply */ - strlcpy(reply.name, request->name, sizeof(reply.name)); - strlcpy(reply_param->port_name, request_param->port_name, - sizeof(reply_param->port_name)); - reply.len_param = sizeof(*reply_param); - if (rte_mp_reply(&reply, peer) < 0) { - TAP_LOG(ERR, "Failed to reply an IPC request to sync queues"); - return -1; - } - return 0; -} - /* Open a TAP interface device. */ static int @@ -2442,9 +2399,11 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) return -1; } - ret = tap_mp_attach_queues(name, eth_dev); - if (ret != 0) - return -1; + ret = rte_mp_action_register(TAP_MP_REQ_START_RXTX, tap_mp_req_start_rxtx); + if (ret < 0 && rte_errno != ENOTSUP) + TAP_LOG(ERR, "tap: Failed to register IPC callback: %s", + strerror(rte_errno)); + rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -2492,15 +2451,6 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) TAP_LOG(DEBUG, "Initializing pmd_tap for %s", name); - /* Register IPC feed callback */ - if (!tap_devices_count) { - ret = rte_mp_action_register(TAP_MP_KEY, tap_mp_sync_queues); - if (ret < 0 && rte_errno != ENOTSUP) { - TAP_LOG(ERR, "tap: Failed to register IPC callback: %s", - strerror(rte_errno)); - goto leave; - } - } tap_devices_count++; tap_devices_count_increased = 1; ret = eth_dev_tap_create(dev, tap_name, remote_iface, &user_mac, @@ -2511,8 +2461,6 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) TAP_LOG(ERR, "Failed to create pmd for %s as %s", name, tap_name); if (tap_devices_count_increased == 1) { - if (tap_devices_count == 1) - rte_mp_action_unregister(TAP_MP_KEY); tap_devices_count--; } } diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index d95605a355..04b3a9f933 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1629,6 +1629,24 @@ rte_eth_hairpin_queue_peer_bind(uint16_t cur_port, uint16_t cur_queue, struct rte_hairpin_peer_info *peer_info, uint32_t direction); +/** + * @internal + * Get rte_eth_dev from device name. The device name should be specified + * as below: + * - PCIe address (Domain:Bus:Device.Function), for example- 0000:2:00.0 + * - SoC device name, for example- fsl-gmac0 + * - vdev dpdk name, for example- net_[pcap0|null0|tap0] + * + * @param name + * pci address or name of the device + * @return + * - rte_eth_dev if successful + * - NULL on failure + */ +__rte_internal +struct rte_eth_dev* +rte_get_eth_dev_by_name(const char *name); + /** * @internal * Reset the current queue state and configuration to disconnect (unbind) it diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index a1d475a292..9192b0d664 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -894,6 +894,17 @@ rte_eth_dev_get_port_by_name(const char *name, uint16_t *port_id) return -ENODEV; } +struct rte_eth_dev * +rte_get_eth_dev_by_name(const char *name) +{ + uint16_t pid; + + if (rte_eth_dev_get_port_by_name(name, &pid)) + return NULL; + + return &rte_eth_devices[pid]; +} + static int eth_err(uint16_t port_id, int ret) { diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index c2fb0669a4..23ca0e8865 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -282,4 +282,5 @@ INTERNAL { rte_eth_representor_id_get; rte_eth_switch_domain_alloc; rte_eth_switch_domain_free; + rte_get_eth_dev_by_name; };