From patchwork Wed Jun 9 14:08:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heinrich Kuhn X-Patchwork-Id: 94061 X-Patchwork-Delegate: andrew.rybchenko@oktetlabs.ru Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2651DA0C47; Wed, 9 Jun 2021 16:08:55 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9C22D410F6; Wed, 9 Jun 2021 16:08:52 +0200 (CEST) Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by mails.dpdk.org (Postfix) with ESMTP id E4696410F4 for ; Wed, 9 Jun 2021 16:08:50 +0200 (CEST) Received: by mail-wr1-f46.google.com with SMTP id a20so25722125wrc.0 for ; Wed, 09 Jun 2021 07:08:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ejctfz2RMcw9m+AwKpX1GBwXouSToARLo8QOSGvSlRg=; b=ZqppbB8oBq/xNR/3R4UAtXx722O6JwUCVURIIiqTkOjOkqDHCM5aytZwbWlPwSEYAB F3SPe+2gHvyP9rO2G5y6LL2JkoDSZa50JV8TZUSK4zGx6uh/yYM92Pxs/NeV0I/7cYOS 2180sCyb4zje8kAek3eLrCirPIWAQynmE9apftJHgaJAMIZHTQvB/fnw4fNtAIWiiiTX kCytS77jHTeBkX+ZL6JZx3j8osL8IThdQU37pOx3xy5nyeGbU7jvGNYIQKV3WsjihxQx +QIsY0c6pmHLQqKSEqHwPgElMYaiYG0aRYFcfPLpsUqTTCtTHknQu/B0pg8gXLkRaE5C aA2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ejctfz2RMcw9m+AwKpX1GBwXouSToARLo8QOSGvSlRg=; b=BihoReUmsWyZerUc2VUq3PjopBXUvIei5fWZvVkwFgGGQqu0ojOj9uWI8XHnCzGmuL HwrXS/zgEc7OIdZ5yQMcJL3d7dPqSB3MnFXSILJA60ggbyR62suiHM0dsjkGk2jG2Aye 3NZUoudS4c9o51TaxocomDus2d89MGlhfDeSga/iqYoUL6rwvxUrK1ldIClpoAMimd+g 0vzulZuTa2XBq6AXkVjO7Cq/t2D1ThuLzeVJCNHd8KdUxOzacZ8rdZtGTly2AmHc/mbm P/gFNOY8/LO7DYfas6/uwS4K5EJR8yaDEqlAJ//YveSSlqqlfddEGllVNb+vqOpJybMg /Bsw== X-Gm-Message-State: AOAM531YYi7xGkWiWEIpLQIIofib5R/QahGXA1ZtH9Rjyz17+eBBONih NyLADHyb32DOBxywwoL0+YTp162ly/yQS6RTunTjbcOJ+Q1GvpvlnA0UeuSQCFbhRfA9c6XEt8Z FFJgd7in+ioIOCGWksBEspjpssmx6S1365Qwr3Ghk26HdH8JmKp8ZJAqjiEz9N4Qe X-Google-Smtp-Source: ABdhPJxmc6c/FEerI6q9sUJyT7Sgt8L09VxMJWt8OlVTcqzXQZ+ocPdOPq83wUIxcKWMECAqhzlIww== X-Received: by 2002:adf:fe8c:: with SMTP id l12mr28471wrr.26.1623247730338; Wed, 09 Jun 2021 07:08:50 -0700 (PDT) Received: from localhost.localdomain (dsl-197-245-55-60.voxdsl.co.za. [197.245.55.60]) by smtp.gmail.com with ESMTPSA id w13sm10012810wmi.48.2021.06.09.07.08.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 Jun 2021 07:08:50 -0700 (PDT) From: Heinrich Kuhn To: dev@dpdk.org Cc: Heinrich Kuhn , Simon Horman Date: Wed, 9 Jun 2021 16:08:14 +0200 Message-Id: <20210609140814.76977-2-heinrich.kuhn@netronome.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20210609140814.76977-1-heinrich.kuhn@netronome.com> References: <20210609140814.76977-1-heinrich.kuhn@netronome.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 2/2] net/nfp: fix PF secondary process probing X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch creates a new function for handling PF probing of a secondary process. A CPP handle is obtained for the CPP bridge service and the service itself is also registered during secondary process initialization. DPDK services aren't shared between processes so it is not enough to only have the primary register the service if it is also needed in a secondary process. This implies that both the primary and secondary will have their own copy of the bridge service. Fixes: 5e15e799d697 ("net/nfp: create separate entity for PF device") Signed-off-by: Heinrich Kuhn Signed-off-by: Simon Horman --- drivers/net/nfp/nfp_net.c | 147 +++++++++++++++++++++++++++----------- 1 file changed, 105 insertions(+), 42 deletions(-) diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c index d94c8dc727..b18edd8c7b 100644 --- a/drivers/net/nfp/nfp_net.c +++ b/drivers/net/nfp/nfp_net.c @@ -59,6 +59,7 @@ static int nfp_net_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info); static int nfp_net_init(struct rte_eth_dev *eth_dev); static int nfp_pf_init(struct rte_pci_device *pci_dev); +static int nfp_pf_secondary_init(struct rte_pci_device *pci_dev); static int nfp_pci_uninit(struct rte_eth_dev *eth_dev); static int nfp_init_phyports(struct nfp_pf_dev *pf_dev); static int nfp_net_link_update(struct rte_eth_dev *dev, int wait_to_complete); @@ -98,6 +99,7 @@ static int nfp_net_rss_hash_write(struct rte_eth_dev *dev, static int nfp_set_mac_addr(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr); static int32_t nfp_cpp_bridge_service_func(void *args); +static void nfp_register_cpp_service(struct nfp_cpp *cpp); static int nfp_fw_setup(struct rte_pci_device *dev, struct nfp_cpp *cpp, struct nfp_eth_table *nfp_eth_table, @@ -3516,20 +3518,6 @@ static int nfp_init_phyports(struct nfp_pf_dev *pf_dev) snprintf(port_name, sizeof(port_name), "%s_port%d", pf_dev->pci_dev->device.name, i); - if (rte_eal_process_type() != RTE_PROC_PRIMARY) { - eth_dev = rte_eth_dev_attach_secondary(port_name); - if (!eth_dev) { - RTE_LOG(ERR, EAL, - "secondary process attach failed, " - "ethdev doesn't exist"); - ret = -ENODEV; - goto error; - } - - eth_dev->process_private = pf_dev->cpp; - goto nfp_net_init; - } - /* Allocate a eth_dev for this phyport */ eth_dev = rte_eth_dev_allocate(port_name); if (!eth_dev) { @@ -3559,7 +3547,6 @@ static int nfp_init_phyports(struct nfp_pf_dev *pf_dev) hw->nfp_idx = nfp_eth_table->ports[i].index; hw->is_phyport = true; -nfp_net_init: eth_dev->device = &pf_dev->pci_dev->device; /* ctrl/tx/rx BAR mappings and remaining init happens in @@ -3593,6 +3580,23 @@ static int nfp_init_phyports(struct nfp_pf_dev *pf_dev) return ret; } +static void nfp_register_cpp_service(struct nfp_cpp *cpp) +{ + uint32_t *cpp_service_id = NULL; + struct rte_service_spec service; + + memset(&service, 0, sizeof(struct rte_service_spec)); + snprintf(service.name, sizeof(service.name), "nfp_cpp_service"); + service.callback = nfp_cpp_bridge_service_func; + service.callback_userdata = (void *)cpp; + + if (rte_service_component_register(&service, + cpp_service_id)) + RTE_LOG(WARNING, PMD, "NFP CPP bridge service register() failed"); + else + RTE_LOG(DEBUG, PMD, "NFP CPP bridge service registered"); +} + static int nfp_pf_init(struct rte_pci_device *pci_dev) { struct nfp_pf_dev *pf_dev = NULL; @@ -3600,8 +3604,6 @@ static int nfp_pf_init(struct rte_pci_device *pci_dev) struct nfp_hwinfo *hwinfo; struct nfp_rtsym_table *sym_tbl; struct nfp_eth_table *nfp_eth_table = NULL; - struct rte_service_spec service; - uint32_t *nfp_cpp_service_id = NULL; char name[RTE_ETH_NAME_MAX_LEN]; int total_ports; int ret = -ENODEV; @@ -3642,12 +3644,10 @@ static int nfp_pf_init(struct rte_pci_device *pci_dev) goto hwinfo_cleanup; } - if (rte_eal_process_type() == RTE_PROC_PRIMARY) { - if (nfp_fw_setup(pci_dev, cpp, nfp_eth_table, hwinfo)) { - PMD_INIT_LOG(ERR, "Error when uploading firmware"); - ret = -EIO; - goto eth_table_cleanup; - } + if (nfp_fw_setup(pci_dev, cpp, nfp_eth_table, hwinfo)) { + PMD_INIT_LOG(ERR, "Error when uploading firmware"); + ret = -EIO; + goto eth_table_cleanup; } /* Now the symbol table should be there */ @@ -3726,24 +3726,8 @@ static int nfp_pf_init(struct rte_pci_device *pci_dev) goto hwqueues_cleanup; } - /* - * The rte_service needs to be created just once per PMD. - * And the cpp handler needs to be linked to the service. - * Secondary processes will be used for debugging DPDK apps - * when requiring to use the CPP interface for accessing NFP - * components. And the cpp handler for secondary processes is - * available at this point. - */ - memset(&service, 0, sizeof(struct rte_service_spec)); - snprintf(service.name, sizeof(service.name), "nfp_cpp_service"); - service.callback = nfp_cpp_bridge_service_func; - service.callback_userdata = (void *)cpp; - - if (rte_service_component_register(&service, - nfp_cpp_service_id)) - RTE_LOG(ERR, PMD, "NFP CPP bridge service register() failed"); - else - RTE_LOG(DEBUG, PMD, "NFP CPP bridge service registered"); + /* register the CPP bridge service here for primary use */ + nfp_register_cpp_service(pf_dev->cpp); return 0; @@ -3763,10 +3747,89 @@ static int nfp_pf_init(struct rte_pci_device *pci_dev) return ret; } +/* + * When attaching to the NFP4000/6000 PF on a secondary process there + * is no need to initialize the PF again. Only minimal work is required + * here + */ +static int nfp_pf_secondary_init(struct rte_pci_device *pci_dev) +{ + struct nfp_cpp *cpp; + struct nfp_rtsym_table *sym_tbl; + int total_ports; + int i; + int err; + + if (!pci_dev) + return -ENODEV; + + /* + * When device bound to UIO, the device could be used, by mistake, + * by two DPDK apps, and the UIO driver does not avoid it. This + * could lead to a serious problem when configuring the NFP CPP + * interface. Here we avoid this telling to the CPP init code to + * use a lock file if UIO is being used. + */ + if (pci_dev->kdrv == RTE_PCI_KDRV_VFIO) + cpp = nfp_cpp_from_device_name(pci_dev, 0); + else + cpp = nfp_cpp_from_device_name(pci_dev, 1); + + if (!cpp) { + PMD_INIT_LOG(ERR, "A CPP handle can not be obtained"); + return -EIO; + } + + /* + * We don't have access to the PF created in the primary process + * here so we have to read the number of ports from firmware + */ + sym_tbl = nfp_rtsym_table_read(cpp); + if (!sym_tbl) { + PMD_INIT_LOG(ERR, "Something is wrong with the firmware" + " symbol table"); + return -EIO; + } + + total_ports = nfp_rtsym_read_le(sym_tbl, "nfd_cfg_pf0_num_ports", &err); + + for (i = 0; i < total_ports; i++) { + struct rte_eth_dev *eth_dev; + char port_name[RTE_ETH_NAME_MAX_LEN]; + + snprintf(port_name, sizeof(port_name), "%s_port%d", + pci_dev->device.name, i); + + PMD_DRV_LOG(DEBUG, "Secondary attaching to port %s", + port_name); + eth_dev = rte_eth_dev_attach_secondary(port_name); + if (!eth_dev) { + RTE_LOG(ERR, EAL, + "secondary process attach failed, " + "ethdev doesn't exist"); + return -ENODEV; + } + eth_dev->process_private = cpp; + eth_dev->dev_ops = &nfp_net_eth_dev_ops; + eth_dev->rx_queue_count = nfp_net_rx_queue_count; + eth_dev->rx_pkt_burst = &nfp_net_recv_pkts; + eth_dev->tx_pkt_burst = &nfp_net_xmit_pkts; + rte_eth_dev_probing_finish(eth_dev); + } + + /* Register the CPP bridge service for the secondary too */ + nfp_register_cpp_service(cpp); + + return 0; +} + static int nfp_pf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, struct rte_pci_device *dev) { - return nfp_pf_init(dev); + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + return nfp_pf_init(dev); + else + return nfp_pf_secondary_init(dev); } static const struct rte_pci_id pci_id_nfp_pf_net_map[] = {