From patchwork Wed Mar 27 11:45:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil Goyal X-Patchwork-Id: 51790 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4EEED1B1E3; Wed, 27 Mar 2019 12:45:47 +0100 (CET) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130041.outbound.protection.outlook.com [40.107.13.41]) by dpdk.org (Postfix) with ESMTP id 32C171B0FB for ; Wed, 27 Mar 2019 12:45:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W6LfzuQk4kkvU5JbSLdXTeNz1N0lXqLOdzNa/Qbfepg=; b=wsNJbTY5Qy7A5cudpZGa3n1bgbnWwPn6DdpCQon+f5Jdj0mgdDvJ+UPXt8Rh6D7c4sSyaDi5Q+MdhkJ0B6Bqkhnb8V1TwMfD1SisIqN66rZ2hXUlJ8Hs3OPtwJJECFZWwH4+v0aC52lVFvRxvM3l4HfmXKsxLGZFzV/R11VxsDw= Received: from VI1PR04MB4893.eurprd04.prod.outlook.com (20.177.49.154) by VI1PR04MB3072.eurprd04.prod.outlook.com (10.170.228.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.19; Wed, 27 Mar 2019 11:45:43 +0000 Received: from VI1PR04MB4893.eurprd04.prod.outlook.com ([fe80::d81a:c2cb:6bce:30bb]) by VI1PR04MB4893.eurprd04.prod.outlook.com ([fe80::d81a:c2cb:6bce:30bb%4]) with mapi id 15.20.1750.014; Wed, 27 Mar 2019 11:45:43 +0000 From: Akhil Goyal To: "dev@dpdk.org" CC: "declan.doherty@intel.com" , Akhil Goyal Thread-Topic: [PATCH] examples/l2fwd-crypto: support multiprocess Thread-Index: AQHU5JKc+sklEU8In0GR47jYHGaSXA== Date: Wed, 27 Mar 2019 11:45:43 +0000 Message-ID: <20190327113622.13426-1-akhil.goyal@nxp.com> Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BM1PR0101CA0039.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::25) To VI1PR04MB4893.eurprd04.prod.outlook.com (2603:10a6:803:56::26) x-mailer: git-send-email 2.17.1 authentication-results: spf=none (sender IP is ) smtp.mailfrom=akhil.goyal@nxp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [92.120.1.68] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3e46adaf-e9e6-4ace-fcff-08d6b2a9be7a x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600127)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:VI1PR04MB3072; x-ms-traffictypediagnostic: VI1PR04MB3072: x-microsoft-antispam-prvs: x-forefront-prvs: 0989A7979C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(376002)(396003)(346002)(39860400002)(136003)(189003)(199004)(478600001)(52116002)(186003)(50226002)(1076003)(99286004)(6486002)(2501003)(14454004)(6116002)(3846002)(6916009)(44832011)(6512007)(6436002)(2906002)(7736002)(36756003)(305945005)(102836004)(97736004)(8676002)(26005)(386003)(105586002)(256004)(2351001)(106356001)(71200400001)(25786009)(71190400001)(5660300002)(14444005)(6506007)(86362001)(2616005)(316002)(8936002)(476003)(486006)(5640700003)(66066001)(1730700003)(4326008)(81156014)(53936002)(81166006)(54906003)(68736007); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR04MB3072; H:VI1PR04MB4893.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: oDV68PWz0ffpot5Im2msAgN/94RFfm1gnmYOuk7tcKVDbbCdOb20RUfJRtCzmcLtPdACbWXC9BTXH2CMfj4pIJiiog1XSpziRBbYxyv6HgWcQSv+AqmjpCqAOlEfYQljJlKQofm+QCWYq6KS/PYpiZ17O4UEssq5XdcTC5CQlhbYoRiXPlt3+XurPKH95Je+i0ITbYyeAqtuPy0QH1tv2st/Qj1J2ZTKPoNdpoXGnvSV1DmRimLfgI2CNh4rqR9pdoCHZt3WrPboLb3AV8Kl8W/cli9R+2POqKzw5KjxI1JZb62Mv4jJ5zsS8WCcoXsgWo3N0y91DW5q5Q8E/8oJpQSiTW1iAKvfryiK5MgN5C7PixKx+TlE6bMZhxrx7b6I59H4YxzfGh7AfSHAS/jPjv5HPmlQmE2OEULbA2rjIHs= MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3e46adaf-e9e6-4ace-fcff-08d6b2a9be7a X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Mar 2019 11:45:43.8595 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3072 Subject: [dpdk-dev] [PATCH] examples/l2fwd-crypto: support multiprocess X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" additional parameters(mp_emask and mp_cmask) need to be added for each application instance for ethernet and crypto portmasks which are actually getting used in that instance. Primary instance of the application will init all the ports/cdevs and will use only the mp_emask and mp_cmask for I/O. net port mask and cryptodev_mask will be same for all the instances of the application only mp_emask and mp_cmask will be different. Signed-off-by: Akhil Goyal --- .../sample_app_ug/l2_forward_crypto.rst | 6 ++ examples/l2fwd-crypto/main.c | 86 +++++++++++++++---- 2 files changed, 75 insertions(+), 17 deletions(-) diff --git a/doc/guides/sample_app_ug/l2_forward_crypto.rst b/doc/guides/sample_app_ug/l2_forward_crypto.rst index e8d52dad2..c0bb63af2 100644 --- a/doc/guides/sample_app_ug/l2_forward_crypto.rst +++ b/doc/guides/sample_app_ug/l2_forward_crypto.rst @@ -152,6 +152,12 @@ where, * [no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). +* mp-emask: mask applicable for multiprocess, defining the ethernet ports + which would be used by this instance. (Default is all ports). + +* mp-cmask: mask applicable for multiprocess, defining the crypto ports + which would be used by this instance. (Default is all ports). + The application requires that crypto devices capable of performing the specified crypto operation are available on application initialization. diff --git a/examples/l2fwd-crypto/main.c b/examples/l2fwd-crypto/main.c index 9982f07e9..c3ea17ee8 100644 --- a/examples/l2fwd-crypto/main.c +++ b/examples/l2fwd-crypto/main.c @@ -168,6 +168,8 @@ struct l2fwd_crypto_options { uint64_t cryptodev_mask; unsigned int mac_updating; + uint64_t mp_emask; + uint64_t mp_cmask; }; /** l2fwd crypto lcore params */ @@ -254,6 +256,9 @@ struct l2fwd_crypto_statistics crypto_statistics[RTE_CRYPTO_MAX_DEVS]; /* default period is 10 seconds */ static int64_t timer_period = 10 * TIMER_MILLISECOND * 1000; +/* Process type*/ +static enum rte_proc_type_t proc_type = RTE_PROC_AUTO; + /* Print out statistics on packets dropped */ static void print_stats(void) @@ -994,7 +999,11 @@ l2fwd_crypto_usage(const char *prgname) " --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default)\n" " When enabled:\n" " - The source MAC address is replaced by the TX port MAC address\n" - " - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n", + " - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID\n" + " --mp-emask: mask applicable for multiprocess, defining the ethernet ports\n" + " which would be used by this instance. Default is all ports\n" + " --mp-cmask: mask applicable for multiprocess, defining the crypto ports\n" + " which would be used by this instance. Default is all ports\n", prgname); } @@ -1169,8 +1178,7 @@ parse_aead_op(enum rte_crypto_aead_operation *op, char *optarg) return -1; } static int -parse_cryptodev_mask(struct l2fwd_crypto_options *options, - const char *q_arg) +parse_mask(uint64_t *mask, const char *q_arg, const char *type) { char *end = NULL; uint64_t pm; @@ -1180,11 +1188,11 @@ parse_cryptodev_mask(struct l2fwd_crypto_options *options, if ((pm == '\0') || (end == NULL) || (*end != '\0')) pm = 0; - options->cryptodev_mask = pm; - if (options->cryptodev_mask == 0) { - printf("invalid cryptodev_mask specified\n"); + if (pm == 0) { + printf("invalid %s mask specified\n", type); return -1; } + *mask = pm; return 0; } @@ -1343,7 +1351,8 @@ l2fwd_crypto_parse_args_long_options(struct l2fwd_crypto_options *options, } else if (strcmp(lgopts[option_index].name, "cryptodev_mask") == 0) - return parse_cryptodev_mask(options, optarg); + return parse_mask(&options->cryptodev_mask, optarg, + "cryptodev_mask"); else if (strcmp(lgopts[option_index].name, "mac-updating") == 0) { options->mac_updating = 1; @@ -1355,6 +1364,12 @@ l2fwd_crypto_parse_args_long_options(struct l2fwd_crypto_options *options, return 0; } + else if (strcmp(lgopts[option_index].name, "mp-emask") == 0) + return parse_mask(&options->mp_emask, optarg, "mp-emask"); + + else if (strcmp(lgopts[option_index].name, "mp-cmask") == 0) + return parse_mask(&options->mp_cmask, optarg, "mp-cmask"); + return -1; } @@ -1488,6 +1503,8 @@ l2fwd_crypto_default_options(struct l2fwd_crypto_options *options) options->type = CDEV_TYPE_ANY; options->cryptodev_mask = UINT64_MAX; + options->mp_emask = UINT64_MAX; + options->mp_cmask = UINT64_MAX; options->mac_updating = 1; } @@ -1654,6 +1671,9 @@ l2fwd_crypto_parse_args(struct l2fwd_crypto_options *options, { "mac-updating", no_argument, 0, 0}, { "no-mac-updating", no_argument, 0, 0}, + { "mp-emask", required_argument, 0, 0}, + { "mp-cmask", required_argument, 0, 0}, + { NULL, 0, 0, 0 } }; @@ -2282,6 +2302,8 @@ initialize_cryptodevs(struct l2fwd_crypto_options *options, unsigned nb_ports, "priv_sess_mp_%u", socket_id); session_pool_socket[socket_id].priv_mp = + (proc_type == RTE_PROC_SECONDARY) ? + rte_mempool_lookup(mp_name) : rte_mempool_create(mp_name, sessions_needed, max_sess_sz, @@ -2305,6 +2327,8 @@ initialize_cryptodevs(struct l2fwd_crypto_options *options, unsigned nb_ports, "sess_mp_%u", socket_id); session_pool_socket[socket_id].sess_mp = + (proc_type == RTE_PROC_SECONDARY) ? + rte_mempool_lookup(mp_name) : rte_cryptodev_sym_session_pool_create( mp_name, sessions_needed, @@ -2455,6 +2479,9 @@ initialize_cryptodevs(struct l2fwd_crypto_options *options, unsigned nb_ports, cap->sym.auth.digest_size.min; } + if (proc_type == RTE_PROC_SECONDARY) + goto skip_cdev_init; + retval = rte_cryptodev_configure(cdev_id, &conf); if (retval < 0) { printf("Failed to configure cryptodev %u", cdev_id); @@ -2480,6 +2507,12 @@ initialize_cryptodevs(struct l2fwd_crypto_options *options, unsigned nb_ports, cdev_id, retval); return -1; } +skip_cdev_init: + if ((options->mp_cmask & (1 << cdev_id)) != 0) { + l2fwd_enabled_crypto_mask |= (((uint64_t)1) << cdev_id); + enabled_cdevs[cdev_id] = 1; + enabled_cdev_count++; + } } return enabled_cdev_count; @@ -2512,6 +2545,9 @@ initialize_ports(struct l2fwd_crypto_options *options) if ((options->portmask & (1 << portid)) == 0) continue; + if (proc_type == RTE_PROC_SECONDARY) + goto skip_port_init; + /* init port */ printf("Initializing port %u... ", portid); fflush(stdout); @@ -2570,7 +2606,7 @@ initialize_ports(struct l2fwd_crypto_options *options) } rte_eth_promiscuous_enable(portid); - +skip_port_init: rte_eth_macaddr_get(portid, &l2fwd_ports_eth_addr[portid]); printf("Port %u, MAC address: %02X:%02X:%02X:%02X:%02X:%02X\n\n", @@ -2593,8 +2629,10 @@ initialize_ports(struct l2fwd_crypto_options *options) last_portid = portid; } - l2fwd_enabled_port_mask |= (1 << portid); - enabled_portcount++; + if ((options->mp_emask & (1 << portid)) != 0) { + l2fwd_enabled_port_mask |= (1 << portid); + enabled_portcount++; + } } if (enabled_portcount == 1) { @@ -2675,17 +2713,22 @@ main(int argc, char **argv) printf("MAC updating %s\n", options.mac_updating ? "enabled" : "disabled"); + proc_type = rte_eal_process_type(); /* create the mbuf pool */ - l2fwd_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NB_MBUF, 512, - sizeof(struct rte_crypto_op), - RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id()); + l2fwd_pktmbuf_pool = (proc_type == RTE_PROC_SECONDARY) ? + rte_mempool_lookup("mbuf_pool") : + rte_pktmbuf_pool_create("mbuf_pool", NB_MBUF, 512, + sizeof(struct rte_crypto_op), + RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id()); if (l2fwd_pktmbuf_pool == NULL) rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n"); /* create crypto op pool */ - l2fwd_crypto_op_pool = rte_crypto_op_pool_create("crypto_op_pool", - RTE_CRYPTO_OP_TYPE_SYMMETRIC, NB_MBUF, 128, MAXIMUM_IV_LENGTH, - rte_socket_id()); + l2fwd_crypto_op_pool = (proc_type == RTE_PROC_SECONDARY) ? + rte_mempool_lookup("crypto_op_pool") : + rte_crypto_op_pool_create("crypto_op_pool", + RTE_CRYPTO_OP_TYPE_SYMMETRIC, NB_MBUF, 128, + MAXIMUM_IV_LENGTH, rte_socket_id()); if (l2fwd_crypto_op_pool == NULL) rte_exit(EXIT_FAILURE, "Cannot create crypto op pool\n"); @@ -2700,6 +2743,8 @@ main(int argc, char **argv) /* skip ports that are not enabled */ if ((options.portmask & (1 << portid)) == 0) continue; + if ((options.mp_emask & (1 << portid)) == 0) + continue; if (options.single_lcore && qconf == NULL) { while (rte_lcore_is_enabled(rx_lcore_id) == 0) { @@ -2731,8 +2776,13 @@ main(int argc, char **argv) } /* Enable Crypto devices */ - enabled_cdevcount = initialize_cryptodevs(&options, enabled_portcount, + if (options.mp_cmask != UINT64_MAX) + enabled_cdevcount = initialize_cryptodevs(&options, nb_ports, enabled_cdevs); + else + enabled_cdevcount = initialize_cryptodevs(&options, + enabled_portcount, enabled_cdevs); + if (enabled_cdevcount < 0) rte_exit(EXIT_FAILURE, "Failed to initialize crypto devices\n"); @@ -2750,6 +2800,8 @@ main(int argc, char **argv) /* Crypto op not supported by crypto device */ if (!enabled_cdevs[cdev_id]) continue; + if ((options.mp_cmask & (1 << cdev_id)) == 0) + continue; if (options.single_lcore && qconf == NULL) { while (rte_lcore_is_enabled(rx_lcore_id) == 0) {