From patchwork Thu Oct 5 23:00:31 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongseok Koh X-Patchwork-Id: 29788 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1FAD01B212; Fri, 6 Oct 2017 01:01:06 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0045.outbound.protection.outlook.com [104.47.1.45]) by dpdk.org (Postfix) with ESMTP id D601E1B1EC; Fri, 6 Oct 2017 01:01:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=2fFzUwwDkd7qCVZqacCwXsgN/821X9TvBpo8SbsUM2s=; b=vAxeF05x5k/8E+NZQidgfjIo/098WgpIIgz+wa9MIthDFcm2x/JTuU4/s0YOXdkYtm5+9S9BDnK6b3ncNqe+v+jHAruo2iGLR2BXWPk0kea98wA5hBzRPTnkySZVPaO3azWJpcHpqqYLh5MX7CJw5D6VjL8rHy2RCtmLKQzSSFo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from mellanox.com (209.116.155.178) by HE1PR0501MB2041.eurprd05.prod.outlook.com (2603:10a6:3:35::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Thu, 5 Oct 2017 23:00:59 +0000 From: Yongseok Koh To: adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com Cc: dev@dpdk.org, Yongseok Koh , stable@dpdk.org Date: Thu, 5 Oct 2017 16:00:31 -0700 Message-Id: <20171005230032.7548-7-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171005230032.7548-1-yskoh@mellanox.com> References: <20171005230032.7548-1-yskoh@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: DM5PR2001CA0013.namprd20.prod.outlook.com (2603:10b6:4:16::23) To HE1PR0501MB2041.eurprd05.prod.outlook.com (2603:10a6:3:35::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 20b15a16-6287-4065-41f3-08d50c44f239 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:HE1PR0501MB2041; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2041; 3:SquO/W+n6iW7uqpwL1n0z5XEQtrBVK0iAJDOeOdao7ugPit5FJtAfeHf2NgAkyKuQ7/c1pWEgy3as+bexs8ItaVppyMJnBR2A8NXlue9XFbL524GJppSi+sC3bGxE5Am8K8C6HMg7RChAdRVTGmj8KY7IkHmhtMi0NV5V2DUlzMP3uyGLgOLjXQKvfDzXg93m23oTwts1wftrWW67IoKX/OyJEbXwaKD7qufQ3DqwyGsiDUBrmDaSbpLRh7yYMP8; 25:m5km2UJ/8qT2id+ishOpl1sFaF74hm1+pogdcySqHPtw866P04c+QKUxAQ3rVUNH9MGzxAMxY0KZar9bdLpYe4sbXBSkz5PmjZt0r2GqUVJWblFE+W9iIVFzESS42hPc6d16metcqSdmzrp/KmXBm3XNqAKJpUPwmcfqSemB/dkUIenEhEADDwMsgeOYjl1Qh0ICyq/zomh9uGDVxXF81rhYCnBreUZ4lDYBBUhK4g68eTwG9CWnYU2Zc47t85RCWzA8jxptyDLR9klo7mQoOiijbqga4YgcqkhaUE3jHoWgIVwj1wDGmHNDkxqQydAAKauSbh7giTDuCdsptCmcAQ==; 31:6XkkNSvJYbkQaeRXEmg6ztu8yrQWcpPohMapiuz29pS/Qi5ILloQ91uuXFlADnJJLISrORmQcq4MrNFK/GI+9YEAJQYN+9GcMpdRrPolPrcbR1AGyD7baa0ZkitMTN+ntuBZRPTQQ+vTQLm4AIPXqO/KP44KzM7d1XF/YLGT92kc0DLeZw1WeKWVCayTWhVuI+T7YBtPQT29EvipLNHqUH0imc+KuLUAXYNdjJtF3ng= X-MS-TrafficTypeDiagnostic: HE1PR0501MB2041: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2041; 20:J2s57HLhmIddVhaN6Su8T5ksRZcjAWk9tMi3Hp+q4RzE2O7OnfSXiwHU9I27slrgEY/YRS2GmwM151T5Zgb9oM9S9PCS8Wj7RL7QEyxN/b+zrIJ9aPMVsosNn07ocOlIgdzoiIgDAW58CeK94twAMPkJ6g3lUbEnZaIodAS0ukPfVhyU+Uh35ZIBBk/iApKhg42evCGfuAZMGchR4fX0WHk1Xt72+/X2vjISjU4tjxgGU1hLrCXEBh53v4GHWp7TO0LZz84NvokSilsmKYHjfYibd5bO5uKetIQ/SAGSC6U+9vKi0IPXXFRYOfPW+P62YuRwiyP/Lp2c3bZqwz9l9Vw3KIBtgXHUnaPBC0X74BKq0NSvIcmFWET8VrrpGFX9IWcz+z72OP6uwbd7iJPm3mvt6LwESdplk+cuTB3sQsDimxaudImP7cskG62qDLmuOEi9E54lg6zFx1HlniK/XsdSvydVmKgI77gIqmqZuKFNoY619Y3+lcI/m/2Oqm/7; 4:HR8TFx+TlDHP3E+tN6gy+vWgRsKCAZtj2zhZ6PvKN7PhUvhBpTpAZ8cT1hSGZuBUETHbF4q9C9zgEuvgVEc2hy8/y2t1SbDMujPLPonqhmyBEVy+8bXxJ4PhFSkOTGkGi+3mrd1veyKNy/1MK8w7uHZ5PbNm7ilpnNDREq2lQ+M8O9V+8/jpLNq+LyCeuf0+RZWjpqOOCki3bm4cUaEHwp2ZO596Sk2dTGlQ190FsW1SEIHqEj7HAot+tqdkM8x/ X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(100000703101)(100105400095)(3002001)(6055026)(6041248)(20161123562025)(20161123564025)(20161123560025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:HE1PR0501MB2041; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:HE1PR0501MB2041; X-Forefront-PRVS: 04519BA941 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(979002)(6009001)(376002)(346002)(189002)(199003)(8676002)(50466002)(2950100002)(81166006)(66066001)(81156014)(7736002)(478600001)(47776003)(305945005)(189998001)(16526018)(76176999)(97736004)(50986999)(86362001)(2906002)(101416001)(16586007)(33646002)(5003940100001)(6666003)(1076002)(105586002)(69596002)(316002)(6116002)(3846002)(50226002)(55016002)(25786009)(53936002)(48376002)(36756003)(5660300001)(106356001)(21086003)(4326008)(68736007)(8936002)(19627235001)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0501MB2041; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR0501MB2041; 23:pWf4BvwunRLqSkCF5H+bTfDhk4MhBKtiVZR8R5o?= aiMmw3Pb9T7vMBHEbYCdjEeYepTKFMZNHjITt5xNDOE638d/c33pXx3RqIx54rdIm7B6AM2pOrhsU9nWKj85/+tssSIn38+0oc000Lct2YhAlUrL/q5j3f00Fi0SvH4ggW/MIDVMQJEw1zMi9IC3Xie9RUKsZ23M6C9x+TQHHeQezq9VFAY8FzawWyddbiaRU2+p75v+3fXTL1WvrIs1Q/BbQFZHqxNzuQDaJ6rj7YcJsju1lV//O+Oa6cUHJtnYqR/pss6Smsr/qC/VMfrJTfiiZhTw+mKVELMs4N9eeTXqtg7Ms9YUcD0YE1nYTPBTXYt5HTqjl+W7wU2XUvmWi9+/niB3bFHnMrq5/vM8PO0iz/NKH86zTRdOGBEAd5FDK6L5PzK5bnoue2Vy1pHe8rtcukj35HZTu2RBaGCHaQyRrpSbkHP7RSZCtB126HkCwOM3Zk2eos8czfRVzzHIQY1TlgHMkO+m3aG500ji6bHIjFmQCuq4TeH3WPZx6nv4C69TWKwheiQgTa3S2AVs+uYEX1FJQQza9GdpF+33eRO813EvQu0N7dd3eoEcClQj/mJB0inP75udT0Z9e7K94mvw2r2G58yADvhjZhPcjT2g1UPoaxHbsWOP/0lbRvL/MglTMpfwiKzCFJOXCKo5QKKVyKYi8Sajz3fn++pEUJ8XbijmjnsxlEZEd+ALAewS9ATUyUiQUcZfuDWKFS6RjHnD+JUk+jtkKaV5Vo2TfDBLKYujNxulYf/ENFqAHqE/hXDiWOx++L4Xdq/4/iC2/KCTxLW72MmS3ALb4B0bqHi03qMTi0XzRkGoSnMbzLUaycnRSp2BPzUXRNE5nxH6XUL/tqjVI0dJhd/bdhUCkFlQth9MhQypSLW5cs00GrEkrO3hmpzBpzBxMht7YVsv5p2HViVyFN1/JEYiYPyvoHuQKVZZpyZqJOeKyLkmIKNZr21LbYXX9UHA98Ko6cYsbbeRS/K0HtAHy3Pn+Cq1SiKkftMd/3RbGVDr/euypl2lerNLMDFfTdTkbfn5KHBKospx//ULL62Cn3ary4F8Bk756SRF+6oaHiedowujb/yaNMiegDkXeoxJbOjfSyNySz1l4hnHMKeLNQEgaQEylOVdrayp/lSm6bwZ3TGka6S8UTB8= X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2041; 6:lPGmxlTz41+LL8wAX9xSphA5baxe6NHqGd262CBc8qEhFI9ztM4EaL1hh9ep7OZcY0U9p5PM5YCg3+0wweLNdgkwm0KnDsYBthGOTOlF7jQigDkuoOHMjcrOjWAON2NAJFeElmx4tMUGvkS+w4/SaP0zpJtdgYPqYO6WJ1qSkC9idDGCTawlRx4n4SbMHbG3cgSO3HFLQZgdE9ThK/LKpYqTSLrXsDzwkHI4RtAwzmjOmdtcMrBILEcRew6pPTC/cAtLbRgMwEWMIh0IKfeQGVmEAKAfugDuSizi+8p+m8I0sAZiyvD+N2eCqq17e1jHbLKUf4rIjytzJM7YmHezoA==; 5:3ycqSNRRvbzp+LZXNZbdTkyA8hfnXDtFXa4PhXG30kX3xCRGVL+i1IzCZXmTBpO6FtmW7L2edKYIC0ryJWMS07oi8nl7M+XreGEoW0lXTLaIUAj0PLbAHATB6oAwccLfVRqMt6oQe9VzSXvo8MhjUg==; 24:1KGOLusydBcW/UhWmc4Ary2IwiIIS67Qpza+1NeRjzYjOUPnevfh6+nGg2Kt7yM3uNxIXZIb8wSwhDZZtBHSXDu0zNTwtgvtfHBLq7ADHmg=; 7:jhdnETtOjkOxciOkV0mt0VwJ0wTEV5CbHnNS92+mN8wdRRWmYPZQgDQcPUHmHVBB/m9zb4TjSOwmb5lpNoc74JnERP0kr44hJi0hwlazzMymu2lWZfb16DlYBrsYyeMQqUjjKGAIW/rgi6mkEHVuOAEMsytuGi6Iov+hYchOeJWs32xciLMVJ97qKPAdAnU3EOnhp/DZTkR/fmSiXVxf7WyhyUl8m/XKgNG4rbGNMXc= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2017 23:00:59.0659 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0501MB2041 Subject: [dpdk-dev] [PATCH v1 6/7] net/mlx5: fix configuration of Rx CQE compression X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" With the upstream rdma-core, to enable Rx CQE compression, mlx5dv_create_cq() in Direct Verbs has to be used instead of regular Verbs call (ibv_create_cq()). And if the size of CQE is 128 bytes, compression is supported only by certain devices. Thus, it has to be decided by checking the capabilitiy bits. Fixes: bba710e6b99b ("net/mlx5: support upstream rdma-core") Cc: stable@dpdk.org Signed-off-by: Yongseok Koh Acked-by: Nelio Laranjeiro --- drivers/net/mlx5/Makefile | 5 +++++ drivers/net/mlx5/mlx5.c | 16 +++++++++++++++- drivers/net/mlx5/mlx5_rxq.c | 20 +++++++++++++++----- 3 files changed, 35 insertions(+), 6 deletions(-) diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index a38b55608..a7c7a6c51 100644 --- a/drivers/net/mlx5/Makefile +++ b/drivers/net/mlx5/Makefile @@ -119,6 +119,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED \ $(AUTOCONF_OUTPUT) $Q sh -- '$<' '$@' \ + HAVE_IBV_MLX5_MOD_CQE_128B_COMP \ + infiniband/mlx5dv.h \ + enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ HAVE_ETHTOOL_LINK_MODE_25G \ /usr/include/linux/ethtool.h \ enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT \ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index c23ce11f7..20a9300d3 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -101,6 +101,10 @@ #define MLX5DV_CONTEXT_FLAGS_ENHANCED_MPW (1 << 3) #endif +#ifndef HAVE_IBV_MLX5_MOD_CQE_128B_COMP +#define MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP (1 << 4) +#endif + struct mlx5_args { int cqe_comp; int txq_inline; @@ -539,6 +543,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) struct ibv_device_attr_ex device_attr; unsigned int sriov; unsigned int mps; + unsigned int cqe_comp; unsigned int tunnel_en = 0; int idx; int i; @@ -642,6 +647,11 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) INFO("MPW is disabled\n"); mps = MLX5_MPW_DISABLED; } + if (RTE_CACHE_LINE_SIZE == 128 && + !(attrs_out.flags & MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP)) + cqe_comp = 0; + else + cqe_comp = 1; if (ibv_query_device_ex(attr_ctx, NULL, &device_attr)) goto error; INFO("%u port(s) detected", device_attr.orig_attr.phys_port_cnt); @@ -758,7 +768,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) priv->pd = pd; priv->mtu = ETHER_MTU; priv->mps = mps; /* Enable MPW by default if supported. */ - priv->cqe_comp = 1; /* Enable compression by default. */ + priv->cqe_comp = cqe_comp; priv->tunnel_en = tunnel_en; /* Enable vector by default if supported. */ priv->tx_vec_en = 1; @@ -847,6 +857,10 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) priv->txq_inline = MLX5_WQE_SIZE_MAX - MLX5_WQE_SIZE; } + if (priv->cqe_comp && !cqe_comp) { + WARN("Rx CQE compression isn't supported"); + priv->cqe_comp = 0; + } /* Configure the first MAC address by default. */ if (priv_get_mac(priv, &mac.addr_bytes)) { ERROR("cannot get MAC address, is mlx5_en loaded?" diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index e7ec1dae3..e1867cb60 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -558,7 +558,10 @@ mlx5_priv_rxq_ibv_new(struct priv *priv, uint16_t idx) container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); struct ibv_wq_attr mod; union { - struct ibv_cq_init_attr_ex cq; + struct { + struct ibv_cq_init_attr_ex ibv; + struct mlx5dv_cq_init_attr mlx5; + } cq; struct ibv_wq_init_attr wq; struct ibv_cq_ex cq_attr; } attr; @@ -597,12 +600,18 @@ mlx5_priv_rxq_ibv_new(struct priv *priv, uint16_t idx) goto error; } } - attr.cq = (struct ibv_cq_init_attr_ex){ + attr.cq.ibv = (struct ibv_cq_init_attr_ex){ + .cqe = cqe_n, + .channel = tmpl->channel, + .comp_mask = 0, + }; + attr.cq.mlx5 = (struct mlx5dv_cq_init_attr){ .comp_mask = 0, }; if (priv->cqe_comp) { - attr.cq.comp_mask |= IBV_CQ_INIT_ATTR_MASK_FLAGS; - attr.cq.flags |= MLX5DV_CQ_INIT_ATTR_MASK_COMPRESSED_CQE; + attr.cq.mlx5.comp_mask |= + MLX5DV_CQ_INIT_ATTR_MASK_COMPRESSED_CQE; + attr.cq.mlx5.cqe_comp_res_format = MLX5DV_CQE_RES_FORMAT_HASH; /* * For vectorized Rx, it must not be doubled in order to * make cq_ci and rq_ci aligned. @@ -610,7 +619,8 @@ mlx5_priv_rxq_ibv_new(struct priv *priv, uint16_t idx) if (rxq_check_vec_support(rxq_data) < 0) cqe_n *= 2; } - tmpl->cq = ibv_create_cq(priv->ctx, cqe_n, NULL, tmpl->channel, 0); + tmpl->cq = ibv_cq_ex_to_cq(mlx5dv_create_cq(priv->ctx, &attr.cq.ibv, + &attr.cq.mlx5)); if (tmpl->cq == NULL) { ERROR("%p: CQ creation failure", (void *)rxq_ctrl); goto error;