From patchwork Tue Oct 23 10:04:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 47231 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 76E3D1B4A9; Tue, 23 Oct 2018 12:04:37 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80087.outbound.protection.outlook.com [40.107.8.87]) by dpdk.org (Postfix) with ESMTP id 3DC361B3BA for ; Tue, 23 Oct 2018 12:04:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yf061I6nkCo9wGSJMu2YCgv/PxrQUt7DAsgo6ziMR6Y=; b=StOdN4SJ8zTvjJWAq6ZF/kOVLOZPYcJ/TJEf1FkM3rIOBktNV+Bah+eu+OV/rfKwuViQVhawL4rs3sREQcDnjaDmhToChHd2bw6qXsmJsglhsC7U2zPPf3kjR7ayU5qQRHbrn/Malj3h25FS4QP/J/Ruh5tW5Tihkbov5leWq7c= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (10.171.186.150) by AM4PR05MB1506.eurprd05.prod.outlook.com (10.164.79.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1250.30; Tue, 23 Oct 2018 10:04:16 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::477:2091:24b4:439f]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::477:2091:24b4:439f%4]) with mapi id 15.20.1250.028; Tue, 23 Oct 2018 10:04:16 +0000 From: Slava Ovsiienko To: Slava Ovsiienko CC: "dev@dpdk.org" Thread-Topic: [PATCH v4 8/8] net/mlx5: support new flow counter API Thread-Index: AQHUarfB8W50QOpHwk6axRxR+W5H2w== Date: Tue, 23 Oct 2018 10:04:16 +0000 Message-ID: <1540289032-29628-9-git-send-email-viacheslavo@mellanox.com> References: <1539962470-10950-1-git-send-email-viacheslavo@mellanox.com> <1540289032-29628-1-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1540289032-29628-1-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: LO2P265CA0442.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:e::22) To AM4PR05MB3265.eurprd05.prod.outlook.com (2603:10a6:205:4::22) authentication-results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [37.142.13.130] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM4PR05MB1506; 6:rhAnA9+BKQFzVXaswFRcPx+81+ppDCwZUNomSRtha1UFEs1OGv8fySUE/OEE3AuYcMwjSDSazbqXteRnHhjWKHZsFSOS1yr70Dk+77aSMhx9fAZPAQ0IdtAzQEkdjcEFo3JhzL95TA9kxP8+An5WLcqxqQux1xa3BepZBROljZW6EOggVY/lH8suk9nGl9k+rSSWNYwzfdzJMM3wotjbgJyGHbijtS9BkEH1zLoaiM9osiAmwwLck1Ynhht7GBGaahiXjITfWzfu+TWrPwcS9bHrVbtJEwaAI55b/qKXbi4tF/IQNJU+8VoiEV/88mF14p1p/YNBAnG5InhlhdrJZ8eUlF+WT9deo1DiwMZPPLHWGa/yE1lx5HusyEVr5EZBr4DaovRcJasrgAJzPl3RPn0NjbrbF4m9N2wRiNTnr/HZSMcqtLwWrfJRdBOd9j5kon/SFgw5wCSc0zL1Nj7pqw==; 5:FKYcSzW3c4vkeFwzCbfEzi063aiEO3yEprbXA1ViFnWT3B9qYVp1B0e7mpcL0dwoZtalKkLnEUCbEngP4ZXaPjmt0iv8eKbcB2X/rdcj0tgbEVV1pKCE9lLISvlIQiADSZI7Oa9U5odhEG791RobSFS9KuPdm1pqBhlGLdM2JIo=; 7:mKaSi6kTW/2cqV8ZLCPPER/x9v8H8IIPqCFqADCmeL5juHhtT0ULxkRvgGyOPar25WIvGytsGdMrLu23A7fSMLFKK7KvkZ5JgcBvxM5EZ1/HwxNetu6uUNTFXXszlLH0IqojwOU3rWsIaHP4/OjKBLfps+sclx52qVy7C+kBhlOq04hJ1MRB8BllKwJGeMCKdmImJOQur5GMfg9CSz5NkyipPklkOZXxlXpKbkXohoQFgZyf4s5UG/ZwN51LKEHX x-ms-office365-filtering-correlation-id: 6c525036-5d49-470d-e7c4-08d638cee3fe x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM4PR05MB1506; x-ms-traffictypediagnostic: AM4PR05MB1506: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(3002001)(3231355)(944501410)(52105095)(6055026)(148016)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123560045)(20161123562045)(201708071742011)(7699051)(76991095); SRVR:AM4PR05MB1506; BCL:0; PCL:0; RULEID:; SRVR:AM4PR05MB1506; x-forefront-prvs: 0834BAF534 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(346002)(396003)(39860400002)(136003)(366004)(199004)(189003)(478600001)(386003)(476003)(6512007)(6506007)(256004)(53936002)(5024004)(14444005)(486006)(102836004)(86362001)(575784001)(6200100001)(5660300001)(14454004)(5250100002)(97736004)(71190400001)(6436002)(6486002)(105586002)(66066001)(8676002)(305945005)(2906002)(316002)(81166006)(4326008)(2900100001)(68736007)(8936002)(186003)(81156014)(37006003)(11346002)(99286004)(7049001)(76176011)(6116002)(52116002)(446003)(26005)(7736002)(2616005)(36756003)(106356001)(6862004)(25786009)(3846002)(71200400001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB1506; H:AM4PR05MB3265.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: KoP9cqtDh/8K5k4TdKCaFCQbYSVvg7ZCE/0S3u+0HgGp/60UQGov9QcJ8y0T5o9Cwzp0cJbI7bO0LGW8sov0mOYRaGfmKBGU6E78LAJFkZ7W6VEeuzNaJdVVLiikGqntCo8p/HImJHe+Vu3adH06u/JGSxvtEpql6n3bxTNvTLmiIqXDRXorcdzQ7djySK/iJDT2mX+KRegd+Jq62jUH/OOY/mdF6LKH+c/ZdhkuuFG6ghQ8OjXKMqx0/CeQWcXoZKXyEWMhyERRiXGiF9RuV114FAZ6OScR2/JzUdiwcck6Uosbj4u89rwwIE9kSqQbS2Ap7tfRzI2q9CcUDhtSZtPVJFPaDI2BJWuWg3N0uJ8= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6c525036-5d49-470d-e7c4-08d638cee3fe X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Oct 2018 10:04:16.2438 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB1506 Subject: [dpdk-dev] [PATCH v4 8/8] net/mlx5: support new flow counter API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch updates the functions performing the Verbs ibrary calls in order to support different versions of the library. The functions: - flow_verbs_counter_new() - flow_verbs_counter_release() - flow_verbs_counter_query() now have the several compilation branches, depending on the counters support found in the system at compile time. The flow_verbs_counter_create() function is introduced as helper for flow_verbs_counter_new(), actually this helper create the counters with Verbs. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_flow.h | 4 ++ drivers/net/mlx5/mlx5_flow_verbs.c | 135 ++++++++++++++++++++++++++++--------- 2 files changed, 107 insertions(+), 32 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index af0a125..69ef0ef 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -228,7 +228,11 @@ struct mlx5_flow_counter { uint32_t shared:1; /**< Share counter ID with other flow rules. */ uint32_t ref_cnt:31; /**< Reference counter. */ uint32_t id; /**< Counter ID. */ +#if defined(HAVE_IBV_DEVICE_COUNTERS_SET_V42) struct ibv_counter_set *cs; /**< Holds the counters for the rule. */ +#elif defined(HAVE_IBV_DEVICE_COUNTERS_SET_V45) + struct ibv_counters *cs; /**< Holds the counters for the rule. */ +#endif uint64_t hits; /**< Number of packets matched by the rule. */ uint64_t bytes; /**< Number of bytes matched by the rule. */ }; diff --git a/drivers/net/mlx5/mlx5_flow_verbs.c b/drivers/net/mlx5/mlx5_flow_verbs.c index f720c35..2547fb8 100644 --- a/drivers/net/mlx5/mlx5_flow_verbs.c +++ b/drivers/net/mlx5/mlx5_flow_verbs.c @@ -34,6 +34,70 @@ #include "mlx5_flow.h" /** + * Create Verbs flow counter with Verbs library. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in, out] counter + * mlx5 flow counter object, contains the counter id, + * handle of created Verbs flow counter is returned + * in cs field (if counters are supported). + * + * @return + * 0 On success else a negative errno value is returned + * and rte_errno is set. + */ +static int +flow_verbs_counter_create(struct rte_eth_dev *dev, + struct mlx5_flow_counter *counter) +{ +#if defined(HAVE_IBV_DEVICE_COUNTERS_SET_V42) + struct priv *priv = dev->data->dev_private; + struct ibv_counter_set_init_attr init = { + .counter_set_id = counter->id}; + + counter->cs = mlx5_glue->create_counter_set(priv->ctx, &init); + if (!counter->cs) { + rte_errno = ENOTSUP; + return -ENOTSUP; + } + return 0; +#elif defined(HAVE_IBV_DEVICE_COUNTERS_SET_V45) + struct priv *priv = dev->data->dev_private; + struct ibv_counters_init_attr init = {0}; + struct ibv_counter_attach_attr attach = {0}; + int ret; + + counter->cs = mlx5_glue->create_counters(priv->ctx, &init); + if (!counter->cs) { + rte_errno = ENOTSUP; + return -ENOTSUP; + } + attach.counter_desc = IBV_COUNTER_PACKETS; + attach.index = 0; + ret = mlx5_glue->attach_counters(counter->cs, &attach, NULL); + if (!ret) { + attach.counter_desc = IBV_COUNTER_BYTES; + attach.index = 1; + ret = mlx5_glue->attach_counters + (counter->cs, &attach, NULL); + } + if (ret) { + claim_zero(mlx5_glue->destroy_counters(counter->cs)); + counter->cs = NULL; + rte_errno = ret; + return -ret; + } + return 0; +#else + (void)dev; + (void)counter; + rte_errno = ENOTSUP; + return -ENOTSUP; +#endif +} + +/** * Get a flow counter. * * @param[in] dev @@ -51,6 +115,7 @@ { struct priv *priv = dev->data->dev_private; struct mlx5_flow_counter *cnt; + int ret; LIST_FOREACH(cnt, &priv->flow_counters, next) { if (!cnt->shared || cnt->shared != shared) @@ -60,36 +125,25 @@ cnt->ref_cnt++; return cnt; } -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_V42 - - struct mlx5_flow_counter tmpl = { - .shared = shared, - .id = id, - .cs = mlx5_glue->create_counter_set - (priv->ctx, - &(struct ibv_counter_set_init_attr){ - .counter_set_id = id, - }), - .hits = 0, - .bytes = 0, - .ref_cnt = 1, - }; - - if (!tmpl.cs) { - rte_errno = errno; - return NULL; - } cnt = rte_calloc(__func__, 1, sizeof(*cnt), 0); if (!cnt) { - claim_zero(mlx5_glue->destroy_counter_set(tmpl.cs)); rte_errno = ENOMEM; return NULL; } - *cnt = tmpl; - LIST_INSERT_HEAD(&priv->flow_counters, cnt, next); - return cnt; -#endif - rte_errno = ENOTSUP; + cnt->id = id; + cnt->shared = shared; + cnt->ref_cnt = 1; + cnt->hits = 0; + cnt->bytes = 0; + /* Create counter with Verbs. */ + ret = flow_verbs_counter_create(dev, cnt); + if (!ret) { + LIST_INSERT_HEAD(&priv->flow_counters, cnt, next); + return cnt; + } + /* Some error occurred in Verbs library. */ + rte_free(cnt); + rte_errno = -ret; return NULL; } @@ -103,7 +157,11 @@ flow_verbs_counter_release(struct mlx5_flow_counter *counter) { if (--counter->ref_cnt == 0) { +#if defined(HAVE_IBV_DEVICE_COUNTERS_SET_V42) claim_zero(mlx5_glue->destroy_counter_set(counter->cs)); +#elif defined(HAVE_IBV_DEVICE_COUNTERS_SET_V45) + claim_zero(mlx5_glue->destroy_counters(counter->cs)); +#endif LIST_REMOVE(counter, next); rte_free(counter); } @@ -117,14 +175,15 @@ */ static int flow_verbs_counter_query(struct rte_eth_dev *dev __rte_unused, - struct rte_flow *flow __rte_unused, - void *data __rte_unused, + struct rte_flow *flow, void *data, struct rte_flow_error *error) { -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_V42 +#if defined(HAVE_IBV_DEVICE_COUNTERS_SET_V42) || \ + defined(HAVE_IBV_DEVICE_COUNTERS_SET_V45) if (flow->actions & MLX5_FLOW_ACTION_COUNT) { struct rte_flow_query_count *qc = data; uint64_t counters[2] = {0, 0}; +#if defined(HAVE_IBV_DEVICE_COUNTERS_SET_V42) struct ibv_query_counter_set_attr query_cs_attr = { .cs = flow->counter->cs, .query_flags = IBV_COUNTER_SET_FORCE_UPDATE, @@ -135,7 +194,12 @@ }; int err = mlx5_glue->query_counter_set(&query_cs_attr, &query_out); - +#elif defined(HAVE_IBV_DEVICE_COUNTERS_SET_V45) + int err = mlx5_glue->query_counters + (flow->counter->cs, counters, + RTE_DIM(counters), + IBV_READ_COUNTERS_ATTR_PREFER_CACHED); +#endif if (err) return rte_flow_error_set (error, err, @@ -157,6 +221,8 @@ NULL, "flow does not have counter"); #else + (void)flow; + (void)data; return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, @@ -993,7 +1059,8 @@ { const struct rte_flow_action_count *count = action->conf; struct rte_flow *flow = dev_flow->flow; -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_V42 +#if defined(HAVE_IBV_DEVICE_COUNTERS_SET_V42) || \ + defined(HAVE_IBV_DEVICE_COUNTERS_SET_V45) unsigned int size = sizeof(struct ibv_flow_spec_counter_action); struct ibv_flow_spec_counter_action counter = { .type = IBV_FLOW_SPEC_ACTION_COUNT, @@ -1012,9 +1079,12 @@ " context."); } *action_flags |= MLX5_FLOW_ACTION_COUNT; -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_V42 +#if defined(HAVE_IBV_DEVICE_COUNTERS_SET_V42) counter.counter_set_handle = flow->counter->cs->handle; flow_verbs_spec_add(dev_flow, &counter, size); +#elif defined(HAVE_IBV_DEVICE_COUNTERS_SET_V45) + counter.counters = flow->counter->cs; + flow_verbs_spec_add(dev_flow, &counter, size); #endif return 0; } @@ -1277,7 +1347,8 @@ detected_actions |= MLX5_FLOW_ACTION_RSS; break; case RTE_FLOW_ACTION_TYPE_COUNT: -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_V42 +#if defined(HAVE_IBV_DEVICE_COUNTERS_SET_V42) || \ + defined(HAVE_IBV_DEVICE_COUNTERS_SET_V45) size += sizeof(struct ibv_flow_spec_counter_action); #endif detected_actions |= MLX5_FLOW_ACTION_COUNT;