From patchwork Sat Nov 10 09:59:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 47981 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CBED74CB5; Sat, 10 Nov 2018 10:59:32 +0100 (CET) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140051.outbound.protection.outlook.com [40.107.14.51]) by dpdk.org (Postfix) with ESMTP id 1ECDA378B for ; Sat, 10 Nov 2018 10:59:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yvk8wkU6T+e1igF+MSGZCZS1avZT65bSgiMJ8QPjeAc=; b=Llqn+zxFaQb4N1tEl0HhT+PecrLEJBXQ3pCzVSts4CfpED3AkRVlQk5xr9Hp1clSrEhACp9RtR7uK/80FXfnfWK0VA54hAw3KEnRB7GWyoNfqRF+lsKVLtRC0EpjQfAr/MFFyS10kDGRZWEBgp71snY6WPDG0o2lhJAjQMPN+KY= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (10.171.186.150) by AM4PR05MB3250.eurprd05.prod.outlook.com (10.171.186.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1294.21; Sat, 10 Nov 2018 09:59:27 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::544b:a68d:e6a5:ba6e]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::544b:a68d:e6a5:ba6e%2]) with mapi id 15.20.1294.039; Sat, 10 Nov 2018 09:59:27 +0000 From: Slava Ovsiienko To: Shahaf Shuler , Yongseok Koh CC: "dev@dpdk.org" , Slava Ovsiienko Thread-Topic: [PATCH 3/3] net/mlx5: fix rule cleanup Netlink command sending Thread-Index: AQHUeNwQ2EdvUZRzrESa8460Z/OZZQ== Date: Sat, 10 Nov 2018 09:59:26 +0000 Message-ID: <1541843951-31708-4-git-send-email-viacheslavo@mellanox.com> References: <1541843951-31708-1-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1541843951-31708-1-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: CWLP265CA0342.GBRP265.PROD.OUTLOOK.COM (2603:10a6:401:5a::18) To AM4PR05MB3265.eurprd05.prod.outlook.com (2603:10a6:205:4::22) authentication-results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [37.142.13.130] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM4PR05MB3250; 6:qvupDOSD27sagaFc9QlC5BvcdTHVIDvrWv2z4639NDlXzr5gV1wjvEDWH1bLXUc9RQz/6fmOdJtp0nlO0sMAhoUh4zmhAbV7ywEA/r1IQhfTT2WpyrGOlchyG/gdguPXD/1d6+6lWCmcaW6W6//JN3xEmMCmv/6LMOt9iHnPinAuJulTBT6zi5kUmV2fVhdwcwxgPgTIAX81uLAmhQMZeyu+bb3JbbWziVW1LrIezeTq15CcoqfGeZCYvXDVzPlDG11eJxCFlWOrUpLOG4f4Q5XoRUB/NSVyQwH8e1JKyj1WCsKmYpPu99WocTAGrrW2PGlrF77BVVNFGqZgHGZ9vHVvcVkZ1p/IPe6x5qcQDtPg8/HheTVFJlgEQ4B6IqtsgI7CCJHjxdAb+mBQ7Yyy5yUUJaRKz8ZjxMybwBBg1pc0bUB73HanHbTjNjPHiyYfPK1x9DbXfeIecedK/R0duA==; 5:KvZ0N5hmbVB2JqhQ73um3umDU+6EZ3mQ/Mh6oxoxbjLw0MWVDjPSv5uGclD5rPPlSS97i1V9XP+LI86wkUKylpPuVOgowVC+6cDUPmyGAtzN9PjoaFKlF+NS4YQEdkEaxaEGhe+snA8nZpkbPRFBp3F5Ibj2TupwuLrXZQ1V2q4=; 7:7morvGPu948ihbXARG2DucPgXCm/eIUxucoeYUZNg+PcPwbaZNV07pl51j0pEG1jqXR0fx36l23Bgf05ANLo35CEhFH/gRYyAuQAjvRjce2K7W8RZ2gosSksgMt9NEqG9U9aoqmyC20lapbWsKgIDA== x-ms-office365-filtering-correlation-id: e4f67ed0-71a7-4018-e837-08d646f3330f x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390040)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM4PR05MB3250; x-ms-traffictypediagnostic: AM4PR05MB3250: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231382)(944501410)(52105095)(3002001)(10201501046)(93006095)(93001095)(6055026)(148016)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123564045)(20161123558120)(201708071742011)(7699051)(76991095); SRVR:AM4PR05MB3250; BCL:0; PCL:0; RULEID:; SRVR:AM4PR05MB3250; x-forefront-prvs: 0852EB6797 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(376002)(39860400002)(346002)(136003)(366004)(199004)(189003)(2616005)(11346002)(6436002)(476003)(486006)(4326008)(7736002)(2906002)(3846002)(76176011)(186003)(54906003)(105586002)(6486002)(6116002)(446003)(305945005)(25786009)(110136005)(66066001)(386003)(6506007)(14454004)(102836004)(36756003)(52116002)(81156014)(97736004)(81166006)(6512007)(106356001)(8676002)(2900100001)(53936002)(107886003)(316002)(478600001)(71200400001)(99286004)(8936002)(68736007)(14444005)(6636002)(256004)(5660300001)(71190400001)(86362001)(85306007)(26005); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3250; H:AM4PR05MB3265.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: LCghoz5yAkXpaI8W92O/GFFfcuIPkG1sjXOIazrlx3qKyW2UVAxrlAQLPpKHDGBV9A+72v69Sgmj+1so9PHxh4WlnD0T94/ReL6UXKRH2s7xypXLno8KO0ixIenvfe5dS/+IG/ixJ8pfxKYbatdqHdQ4NuOAGjWr2qbCnO5hfxnA/CbuSVteGzrcqbcGGC1Peyd7nVjelztT0nh2KCVJIwhEXNw6P9uqnM8jz5eyQlQJqn/wyCLZK3arvWD6dROQNQro9hPLk5qGp9Pi0H0wmXcclrNrrTSTc5nh4ajXqAm16AngqhGMw0MLllJOjXBvXUanB9CO6fUIHtmbnNhDyba4/V24784fqX3XcAB+1dY= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: e4f67ed0-71a7-4018-e837-08d646f3330f X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Nov 2018 09:59:27.0093 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3250 Subject: [dpdk-dev] [PATCH 3/3] net/mlx5: fix rule cleanup Netlink command sending X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The VXLAN related rule cleanup routine queries and gathers all existing local IP and neigh rules into buffer list. One buffer may contain multiple rule deletetion commands and is prepared to send into Netlink as single message. But, if error occurs for some deletion commands in the buffer, the multiple ACK message with errors can be send back by the kernel. It breaks the Netlink communication sequence numbers, because we expect only one ACK message and it smashes out futher Netlik communication. The workaround of this problem is to send rule deletion commands from buffer in one-by-one fashion and get ACK message for every command sent. We do not expect too may rules preexist, so there should not be critical performance degradation at VXLAN outer interface initialization. Fixes: f420f03d6772 ("net/mlx5: add E-switch VXLAN rule cleanup routines") Signed-off-by: Viacheslav Ovsiienko Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow_tcf.c | 58 +++++++++++++++++----------------------- 1 file changed, 24 insertions(+), 34 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_tcf.c b/drivers/net/mlx5/mlx5_flow_tcf.c index bba8aed..21eb99e 100644 --- a/drivers/net/mlx5/mlx5_flow_tcf.c +++ b/drivers/net/mlx5/mlx5_flow_tcf.c @@ -3847,30 +3847,6 @@ struct tcf_nlcb_context { } /** - * Set NLM_F_ACK flags in the last netlink command in buffer. - * Only last command in the buffer will be acked by system. - * - * @param[in, out] buf - * Pointer to buffer with netlink commands. - */ -static void -flow_tcf_setack_nlcmd(struct tcf_nlcb_buf *buf) -{ - struct nlmsghdr *nlh; - uint32_t size = 0; - - assert(buf->size); - do { - nlh = (struct nlmsghdr *)&buf->msg[size]; - size += NLMSG_ALIGN(nlh->nlmsg_len); - if (size >= buf->size) { - nlh->nlmsg_flags |= NLM_F_ACK; - break; - } - } while (true); -} - -/** * Send the buffers with prepared netlink commands. Scans the list and * sends all found buffers. Buffers are sent and freed anyway in order * to prevent memory leakage if some every message in received packet. @@ -3888,21 +3864,35 @@ struct tcf_nlcb_context { flow_tcf_send_nlcmd(struct mlx5_flow_tcf_context *tcf, struct tcf_nlcb_context *ctx) { - struct tcf_nlcb_buf *bc, *bn; - struct nlmsghdr *nlh; + struct tcf_nlcb_buf *bc = LIST_FIRST(&ctx->nlbuf); int ret = 0; - bc = LIST_FIRST(&ctx->nlbuf); while (bc) { + struct tcf_nlcb_buf *bn = LIST_NEXT(bc, next); + struct nlmsghdr *nlh; + uint32_t msg = 0; int rc; - bn = LIST_NEXT(bc, next); - if (bc->size) { - flow_tcf_setack_nlcmd(bc); - nlh = (struct nlmsghdr *)&bc->msg; - rc = flow_tcf_nl_ack(tcf, nlh, bc->size, NULL, NULL); - if (rc && !ret) - ret = rc; + while (msg < bc->size) { + /* + * Send Netlink commands from buffer in one by one + * fashion. If we send multiple rule deletion commands + * in one Netlink message and some error occurs it may + * cause multiple ACK error messages and break sequence + * numbers of Netlink communication, because we expect + * the only one ACK reply. + */ + assert((bc->size - msg) >= sizeof(struct nlmsghdr)); + nlh = (struct nlmsghdr *)&bc->msg[msg]; + assert((bc->size - msg) >= nlh->nlmsg_len); + msg += nlh->nlmsg_len; + rc = flow_tcf_nl_ack(tcf, nlh, 0, NULL, NULL); + if (rc) { + DRV_LOG(WARNING, + "netlink: cleanup error %d", rc); + if (!ret) + ret = rc; + } } rte_free(bc); bc = bn;