From patchwork Tue Dec 26 07:28:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 135580 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 154C643795; Tue, 26 Dec 2023 08:30:13 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E367240A72; Tue, 26 Dec 2023 08:29:07 +0100 (CET) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2096.outbound.protection.outlook.com [40.107.212.96]) by mails.dpdk.org (Postfix) with ESMTP id E0F1440DCA for ; Tue, 26 Dec 2023 08:29:06 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UnG3qEiiYPWS/sC0hAfIdTzYOqbRJl7vufD7lq1lAXocDIPVEkpAlLOLIawO2yEKuWfti2GUkpJTA2KM9DG7lnU+aqPhckaoyBV7jU77ITTVOoWr48dRh8LcP9Q7Wz+rNuGUmxxYCf6Op2OIXhxI7Rd2FNFoGcsfBAf2aqKHqWeBfTRxANnV7YNig44ruQtH5YrLSUBgtuSJyPbzX1ZU9nuLejMmPyXxmldIM+/o2Ndla1+tlOJjreoy58WCiFEuN/DxYG2nB4i29QEDgrHKVMcVhPj7jSccXpwDgBvJY6Wbgbic1OU59ZgZu7FymjA4btz2t/e3Q4scjL0D9hozEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ij8QBNgBP72k6ZRgpa33f2G29FEr1Y+m+yfbZ+QHxBk=; b=EE1JdFL5yrjk4gC7MQdrjxGxrW7vR857S2mYkQaDUbz6b0CrPUj20YLPc97m4fwAqi2MTonabj7Q14Ygpq+DbsVgEY6Ze+FBVg7r0sJjnSxQ7w5hR2LkceefD343PJrckoduyewG2kdqtQU6bZVeZDKCWJuS5KdondMSeBDPpZFrtZ3WLmaNoVaDa8IjuEHkuag764dplk9vpYVf/sy+benfwjP1xkrldPvtlizqQiAnHhHpgadto3BSd9HIcPY3eG3JE5SaIW9h0n05rB+/tdTbG+DwTXaSuDvZ+neMUmwWdE5/cpisWkV/DI7iF/kckJRhtNw/E892ohWMSTOHRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ij8QBNgBP72k6ZRgpa33f2G29FEr1Y+m+yfbZ+QHxBk=; b=AJU0c5UpGAhyf+44gEUNCieCrW+4aS71qn3aWT2HzSm6uBWEhrCuK1lzMSNJAnwbxp+VuzPVQexMicfrWSpqbFVEGd6vNL0VyMXJTE1mrUD3h01MVDfS4EAnAP7k1YhSht3l4iiw9WRNJYenh2gZr/W1IUzjMheJ4WOgOLo317g= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by CH3PR13MB6506.namprd13.prod.outlook.com (2603:10b6:610:1a3::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.27; Tue, 26 Dec 2023 07:29:05 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b%6]) with mapi id 15.20.7113.027; Tue, 26 Dec 2023 07:29:05 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Long Wu , Peng Zhang , Chaoyong He Subject: [PATCH v6 11/14] net/nfp: support bond member notification Date: Tue, 26 Dec 2023 15:28:21 +0800 Message-Id: <20231226072824.3163121-12-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231226072824.3163121-1-chaoyong.he@corigine.com> References: <20231226023745.3144143-1-chaoyong.he@corigine.com> <20231226072824.3163121-1-chaoyong.he@corigine.com> X-ClientProxiedBy: SJ0PR03CA0239.namprd03.prod.outlook.com (2603:10b6:a03:39f::34) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|CH3PR13MB6506:EE_ X-MS-Office365-Filtering-Correlation-Id: 343d4728-9b52-4608-9a57-08dc05e45703 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Zbo85fK/+lymgJBBp3XYt53bu/lXoQUjUy6jFUvjgzR47WORVy5LDaUPeZi7b/Es0cwbi7vx8vGEFG/wmo4kB+MjuqiOPnS1v+wi7u3ArvKrmLVeqzIvuMQMGlM73yc9W5se6vYTA2FEWUZKSqhxG6+pbq907TVuVPcm4BIQVQxel9HpvK7OVkJQ4cMGaQcmX11Na9COsikKdXlVkOIBahcv8I19gl9hwpGb7/WirczlgQIWcyKaLCHfgADQA7KBLeXj7Fq+3PzzRTTfgVo3WZ+TXEeOkszHH4m+1CqLAaBWJCrh6f9ttnpAi6rKrIGAYsep1ulf93lik74edlDRU0Obl+lZk4U2GUTV5K4WmQuAGIOIdhD9hsBWNuVMpPLkbrwEJPZHUmp/1YiY7VpMVlcyhyH9fdQQ+oDt2woNqn2ABjsLfGgx38v7+7M4qsk4sL/X7tLYJBJbXstxUzoKiAeLH/HeS1eN7UMuPBOyDUX88lV3xW9DJR6/yVQie0GTCQqE7f8VPIc3/8Dy6s6z+iG0/SZFOEL0M3WL8tDZSc6nWYAu3zHxW6V0byV7SnyXq0+20dNLlbovnLuow0GA2mYYwDgzU0Qj7LKNbO7KJBlxpzPfxE9h1NnPQ5TTT6wKK1pVVgKOLxE8Q11eQuz8TNTYXXNL9DS6i98G4kngeKMw/4Ebr+7IadNYmHA7gn5MJSi4gs7NezXQTurXq7uTwA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376002)(346002)(136003)(396003)(366004)(39840400004)(230273577357003)(230173577357003)(230922051799003)(451199024)(186009)(64100799003)(1800799012)(107886003)(26005)(1076003)(52116002)(6666004)(2616005)(6512007)(83380400001)(5660300002)(44832011)(15650500001)(4326008)(8676002)(41300700001)(2906002)(8936002)(6486002)(478600001)(316002)(6916009)(54906003)(66476007)(66556008)(66946007)(6506007)(86362001)(36756003)(38100700002)(38350700005); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 5sN8SXacNfxPc3NoAvYaRVjaIDaB+QI/GhZzfaZx/LIqoShvoZF7PSWaWtoR/WMLE/8Zw5orzVMqUF8eOiS6p10orumiyHBEtR8VCTU6ZSd+n5RM8XjkzksLGlLXEmiiAE6iKNk/mVOOj6dJhMYkQhZdtPdDye5r+F45kXUmzsyn4IqYyywuJsQZSTkA4KV3SpNBb52AHOVrA2EyQUAx9vcJt/kOlmraxi9dDrHF66ovgEhUDFXYwsAhCmdpbYYf9586iitlD1gOqEA1CRBMfdR++yjACnb06+adsaBr161sTS6VzFQL9L3nVbxYjbaE/xuo7JcLsaZwK2Qt9qNpx371TDVdTNnmqiaau2zUNMcnfRicl4nfTKGha97pY+/U4Y0bXIxrz8PJm5ZLvDDSjePUdqMLd/PluKXg+FULC4i640EXfBnP853+uCMBlSnfgdKwTbU9BTjYUIi7k9XqxBw0D2fXalmpbvtdBRwiLS81fkXlAzUIE2+UDDprT7GRp8F5OYuh7hlPuA1uxFn1aQgaYR58tujJUq9V+YfJoCkfO6d+llXn+HXAkCI67EyBJPKjJjJfykT+k+PtTZiYd+TBgWVeBdzLBr+ec4TF6/Vb5ykrlRi2LHLsxiOJiEMHNs6umGKyT5qLa9o7B0IOGGRFwu9gq9LltDWPgENwOp61trZlv7s0OtdGenUzjiDR3LdT/TzPW2TlEoT4hV/LVei88JizK5tRirFMnDzQEx9thcIlAGiTJIEWJOhFXy/UDDBeF+S+9bJNrM1Eji6fHerB+QpuIVOlZN36v7WI72v+7XPRU3inWGrIkexocQjuVoHTuko/7xr9DRNMk7JEf+AgOtVMy4I/jQ2LaGGJ9oxAZuv6dZgIFKwzKEYZMixTTXAKIyZrZX6k1cAeYCAlzff96cSJgTmjXe1h6lQH/dgDy2D8PKfXRVY3Z2sGaPxfOCYG1/iHuf4YgbK+MES8rROeds+sokFLhmwa8n1fvDbLCDu69XV0qwv0lJ8KBtZQ4n67AYvwO9rtv/PJPdRKPqLA8XTLc7X70N3CrbcPxJgGlXF+dr36+wiL+k5ElG6HhWITWdozTpWj8p8zI4HeYXNsFa94lxIQ+AhN4qY2Pzng5xtK7aw6O3JNb+s/DSNYdTqfBkLxqBcsJuzYo5L2QQt9abJRGFJFzPCJrw2O6tBUZxJHZ6iAHZbTu6IwL/ygeIY8ruvCMC4YEFLzSjhsTuY7L326Kd849cGKZz6ytPY8UO3ZHPkhXlzR0S1H3mGqEybgyIHsaiJTKqojmkQYQut8DYM0OSlpx6iDwBIXl12lF1TIzhDwy7XaZWk/T0aiTNiAZ4H7pnYsS2Q8ifhyw+lbpX5ayRQI9H+RF/AvNxoDCtrU5LNismEXHED4QU+MfJ1EsyGd1RlJ0A9xpCQ0m6wALA1jqugeBHJY2yUiA9aHPysb127u8cTVCPvjWJW/LsAh9qphgq4uhXiaLmyG2nosAxGi8zPr/+e4ySkgBZo8h49Tne4EMZGr2HWAFAKCEWCCLWe6p51aNXMmlYgG9ByapC3fIuo6iVFmYbJjW9PbsT32+5k7OnvAbxHzrW3TutQx80/Vv/Eg+lx5SU6bRA== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 343d4728-9b52-4608-9a57-08dc05e45703 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Dec 2023 07:29:05.2933 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: i1WxO5B3CWeJ38oyMdjHe2jes3IJ/8JgOqQgLLBABSo39lsSV21tnBpwEnoBTxij9XvzHNbsSao/uQYqxgDLyhbRxCQSf6LwiIZZ0aEZrMI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR13MB6506 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Long Wu A NFP representor port can receive a bond notification. Then driver will parse this notification into one of these two events: 1. Bonding port configuration may have changed. This includes creation of a bonding port, removal/addition of a member port, changing the bond mode, etc. 2. Bonding port is deleted. Signed-off-by: Long Wu Reviewed-by: Peng Zhang Reviewed-by: Chaoyong He --- drivers/net/nfp/flower/nfp_flower_bond.c | 183 +++++++++++++++++- drivers/net/nfp/flower/nfp_flower_bond.h | 2 + .../net/nfp/flower/nfp_flower_representor.c | 21 ++ 3 files changed, 205 insertions(+), 1 deletion(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index c814c0d4ea..523e0025ad 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -46,6 +46,39 @@ nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) nfp_bond->batch_ver += 2; } +static void +nfp_flower_bond_group_id_clear(struct nfp_flower_bond *nfp_bond, + uint32_t id_clear) +{ + bool *group_id_map = nfp_bond->group_id_map; + + if (id_clear >= NFP_FL_BOND_GROUP_MAX || id_clear < NFP_FL_BOND_GROUP_MIN) { + PMD_DRV_LOG(ERR, "Try to clear invalid group id %u.", id_clear); + return; + } + + if (group_id_map[id_clear]) + group_id_map[id_clear] = false; +} + +static int +nfp_flower_bond_group_id_get(struct nfp_flower_bond *nfp_bond, + uint32_t *id_ret) +{ + uint32_t id; + bool *group_id_map = nfp_bond->group_id_map; + + for (id = NFP_FL_BOND_GROUP_MIN; id < NFP_FL_BOND_GROUP_MAX; id++) { + if (!group_id_map[id]) { + group_id_map[id] = true; + *id_ret = id; + return 0; + } + } + + return -ENOSPC; +} + static enum nfp_flower_bond_batch nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, struct nfp_bond_group *entry, @@ -65,6 +98,7 @@ nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, entry->offloaded = false; if (entry->to_destroy) { + nfp_flower_bond_group_id_clear(nfp_bond, entry->group_id); LIST_REMOVE(entry, next); rte_free(entry); } @@ -139,6 +173,7 @@ nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) pthread_mutex_lock(&nfp_bond->mutex); LIST_FOREACH(entry, &nfp_bond->group_list, next) { + nfp_flower_bond_group_id_clear(nfp_bond, entry->group_id); LIST_REMOVE(entry, next); rte_free(entry); } @@ -271,16 +306,162 @@ nfp_flower_bond_change_linkstatus_event(struct nfp_flower_bond *nfp_bond, *bond_port_flags |= NFP_FL_BOND_PORT_CHANGED; } +static struct nfp_bond_group * +nfp_flower_bond_group_create(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + uint32_t id; + unsigned int numa_node; + struct nfp_bond_group *group = NULL; + + if (nfp_flower_bond_group_id_get(nfp_bond, &id) < 0) + return NULL; + + numa_node = rte_socket_id(); + + group = rte_zmalloc_socket(NULL, sizeof(struct nfp_bond_group), + RTE_CACHE_LINE_SIZE, numa_node); + if (group == NULL) { + PMD_DRV_LOG(ERR, "Unable malloc memory for nfp bond group"); + nfp_flower_bond_group_id_clear(nfp_bond, id); + return NULL; + } + + group->group_id = id; + group->main_dev = bond_dev; + group->dirty = true; + group->offloaded = false; + group->to_remove = false; + group->to_destroy = false; + group->member_cnt = 0; + group->group_inst = ++nfp_bond->global_inst; + LIST_INSERT_HEAD(&nfp_bond->group_list, group, next); + + return group; +} + +static int +nfp_flower_bond_changeupper_event(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + uint32_t i; + uint16_t port_id; + bool can_offload = true; + uint16_t nfp_member_count; + struct rte_eth_dev *eth_dev; + struct nfp_bond_group *group; + struct bond_dev_private *internals; + struct nfp_flower_representor *repr; + struct nfp_app_fw_flower *app_flower; + + internals = bond_dev->data->dev_private; + app_flower = nfp_bond->app_fw_flower; + nfp_member_count = 0; + + for (i = 0; i < internals->member_count; i++) { + port_id = internals->members[i].port_id; + eth_dev = &rte_eth_devices[port_id]; + if (!nfp_flower_is_phy_repr(eth_dev)) { + can_offload = false; + break; + } + + repr = eth_dev->data->dev_private; + + if (repr->app_fw_flower != app_flower) { + can_offload = false; + break; + } + + if (internals->current_primary_port == port_id || + internals->mode != BONDING_MODE_ACTIVE_BACKUP) + nfp_member_count++; + } + + if (internals != NULL && + internals->mode != BONDING_MODE_ACTIVE_BACKUP && + ((internals->mode != BONDING_MODE_BALANCE && + internals->mode != BONDING_MODE_8023AD) || + internals->balance_xmit_policy != BALANCE_XMIT_POLICY_LAYER34)) { + can_offload = false; + PMD_DRV_LOG(WARNING, "Unable to offload mode %u hash %u.", + internals->mode, + internals->balance_xmit_policy); + } + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (nfp_member_count == 0 || !can_offload) { + if (group != NULL && group->offloaded) + /* Set remove flag */ + group->to_remove = true; + + pthread_mutex_unlock(&nfp_bond->mutex); + return 0; + } + + if (group == NULL) { + group = nfp_flower_bond_group_create(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + return -1; + } + } + + group->dirty = true; + group->member_cnt = nfp_member_count; + group->to_remove = false; + + pthread_mutex_unlock(&nfp_bond->mutex); + + return 0; +} + +static void +nfp_flower_bond_group_delete(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + struct nfp_bond_group *group; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + PMD_DRV_LOG(WARNING, "Untracked bond got unregistered %s", + bond_dev->device->name); + return; + } + + group->to_remove = true; + group->to_destroy = true; + + pthread_mutex_unlock(&nfp_bond->mutex); +} + int nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, struct rte_eth_dev *bond_dev, struct rte_eth_dev *nfp_dev, enum nfp_flower_bond_event event) { + int ret = 0; + switch (event) { + case NFP_FLOWER_CHANGEUPPER: + ret = nfp_flower_bond_changeupper_event(nfp_bond, bond_dev); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Change upper event can not work."); + return ret; + } + break; case NFP_FLOWER_CHANGELINKSTATUS: nfp_flower_bond_change_linkstatus_event(nfp_bond, bond_dev, nfp_dev); break; + case NFP_FLOWER_UNREGISTER: + nfp_flower_bond_group_delete(nfp_bond, bond_dev); + break; default: PMD_DRV_LOG(ERR, "Invalid bond offload event."); return -1; @@ -288,7 +469,7 @@ nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, nfp_flower_bond_do_work(nfp_bond); - return 0; + return ret; } enum nfp_flower_bond_batch diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index e4a09b3427..12e61ff1ce 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -109,6 +109,8 @@ struct nfp_flower_bond { uint32_t global_inst; /** Incremented for each config packet sent */ uint32_t pkt_num; + /** ID of bond group in driver, true is used */ + bool group_id_map[NFP_FL_BOND_GROUP_MAX]; /** Pointer to the flower app */ struct nfp_app_fw_flower *app_fw_flower; }; diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 2810a7a271..43106da90e 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -390,6 +390,25 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev) return 0; } +static int +nfp_flower_repr_bond_notify_member(struct rte_eth_dev *dev, + struct rte_eth_dev *bond_dev) +{ + struct nfp_flower_representor *repr; + enum nfp_flower_bond_event event = NFP_FLOWER_CHANGEUPPER; + + if (!nfp_flower_bond_all_member_are_phyrepr(bond_dev)) + event = NFP_FLOWER_UNREGISTER; + + repr = dev->data->dev_private; + + if (nfp_flower_support_bond_offload(repr->app_fw_flower)) + return nfp_flower_bond_event_handle(repr->app_fw_flower->nfp_bond, + bond_dev, dev, event); + + return 0; +} + static const struct eth_dev_ops nfp_flower_pf_repr_dev_ops = { .dev_infos_get = nfp_flower_repr_dev_infos_get, @@ -437,6 +456,8 @@ static const struct eth_dev_ops nfp_flower_repr_dev_ops = { .flow_ops_get = nfp_flow_ops_get, .mtr_ops_get = nfp_net_mtr_ops_get, + + .bond_notify_member = nfp_flower_repr_bond_notify_member, }; static uint32_t