From patchwork Wed Oct 9 15:10:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 60806 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 929BF1E53E; Wed, 9 Oct 2019 17:10:36 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 6D9031C214 for ; Wed, 9 Oct 2019 17:10:35 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x99Esh9W024439; Wed, 9 Oct 2019 08:10:34 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : mime-version : content-type : content-transfer-encoding; s=pfpt0818; bh=CzN5joGD+4ZMtVd3pIrmyxR3ojxa3sG88MLwdla92EI=; b=dvMryUhwbxHGtivmylvHeDzO3lqVfOFg8EyaT7doaZrPNThri4cRdSGGpKCp6X4Nfja8 Voqh3/qAaIlsnQaKxT1aNeCmSbhfeMnm4L+X2CV36rto+5CMCk1rlwmdrze1qTQTZyv8 bM8ce9bDCBH0yHJrp+vgeU/l1+QhVT5e6KHXZnquwVkDajQeGr44HCb90ak+sFxJQobZ L1MXjwQGU5tawjpyIsmIJRu1m11m1QKUAebJEjQan2WnX+qO1l+MFWecGO14jYgepne0 uUQ4VomuY2DMAGtFioSyfGXjHYfyDWNjOz3oteP/Ubi1NaPq8/lvKsUr7KtshcaqUAgo Tg== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 2vhdxbrwj2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 09 Oct 2019 08:10:34 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 9 Oct 2019 08:10:32 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Wed, 9 Oct 2019 08:10:32 -0700 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 86E3C3F7043; Wed, 9 Oct 2019 08:10:29 -0700 (PDT) From: Anoob Joseph To: Akhil Goyal , Radu Nicolau CC: Anoob Joseph , Thomas Monjalon , Jerin Jacob , Narayana Prasad , Lukasz Bartosik , Date: Wed, 9 Oct 2019 20:40:03 +0530 Message-ID: <1570633816-4706-1-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,1.0.8 definitions=2019-10-09_06:2019-10-08,2019-10-09 signatures=0 Subject: [dpdk-dev] [RFC PATCH 00/13] add eventmode to ipsec-secgw X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This series introduces event-mode additions to ipsec-secgw. This effort is based on the proposed changes for l2fwd-event and the additions in l3fwd for event support. With this series, ipsec-secgw would be able to run in eventmode. The worker thread (executing loop) would be receiving events and would be submitting it back to the eventdev after the processing. This way, multicore scaling and h/w assisted scheduling is achieved by making use of the eventdev capabilities. Since the underlying event device would be having varying capabilities, the worker thread could be drafted differently to maximize performance. This series introduces usage of multiple worker threads, among which the one to be used will be determined by the operating conditions and the underlying device capabilities. For example, if an event device - eth device pair has Tx internal port, then application can do tx_adapter_enqueue() instead of regular event_enqueue(). So a thread making an assumption that the device pair has internal port will not be the right solution for another pair. The infrastructure added with these patches aims to help application to have multiple worker threads, there by extracting maximum performance from every device without affecting existing paths/use cases. The eventmode configuration is predefined. All packets reaching one eth port will hit one event queue. All event queues will be mapped to all event ports. So all cores will be able to receive traffic from all ports. When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event device will ensure the ordering. Ordering would be lost when tried in PARALLEL. Following command line options are introduced, --transfer-mode: to choose between poll mode & event mode --schedule-type: to specify the scheduling type (RTE_SCHED_TYPE_ORDERED/ RTE_SCHED_TYPE_ATOMIC/ RTE_SCHED_TYPE_PARALLEL) --process-dir: outbound/inbound --process-mode: app mode /driver mode The two s/w config options added to ipsec-secgw can be used in benchmarking h/w performance, 1. process-dir : states whether the direction is outbound/inbound. This option aims to avoid an unnecessary check of determining whether inbound/outbound processing need to be done on the packet. For each option a different light weight worker thread would be executed. 2. process-mode: states whether the application has to run in driver mode or app mode. Driver-mode: This mode will have bare minimum changes in the application to support ipsec. There woudn't be any lookup etc done in the application. And for inline-protocol use case, the thread would resemble l2fwd as the ipsec processing would be done entirely in the h/w. This mode can be used to benchmark the raw performance of the h/w. All the application side steps (like lookup) can be redone based on the requirement of the end user. Hence the need for a mode which would report the raw performance. App-mode: This mode will have all the features currently implemented with ipsec-secgw (non librte_ipsec mode). All the lookups etc would follow the existing methods and would report numbers that can be compared against regular ipsec-secgw benchmark numbers. Example commands to execute ipsec-secgw in various modes on OCTEONTX2 platform, #Inbound driver mode ./ipsec-secgw -w 0002:02:00.0,nb_ipsec_in_sa=128 -w 0002:03:00.0,nb_ipsec_in_sa=128 -w 0002:04:00.0,nb_ipsec_in_sa=128 -w 0002:07:00.0,nb_ipsec_in_sa=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x7 – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f dpdk_internal/100g_4.3.cfg --transfer-mode 1 --schedule-type 2 --process-mode 1 --process-dir 1 #Inbound app mode ./ipsec-secgw -w 0002:02:00.0,nb_ipsec_in_sa=128 -w 0002:03:00.0,nb_ipsec_in_sa=128 -w 0002:04:00.0,nb_ipsec_in_sa=128 -w 0002:07:00.0,nb_ipsec_in_sa=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x3f – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f dpdk_internal/100g_4.3.cfg --transfer-mode 1 --schedule-type 2 --process-mode 0 --process-dir 1 #Outbound driver mode ./ipsec-secgw -w 0002:02:00.0 -w 0002:03:00.0 -w 0002:04:00.0 -w 0002:07:00.0 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x1f – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f a-aes-gcm-new.cfg --transfer-mode 1 --schedule-type 2 --process-mode 1 --process-dir 0 #Outbound app mode ./ipsec-secgw -w 0002:02:00.0 -w 0002:03:00.0 -w 0002:04:00.0 -w 0002:07:00.0 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=8 -c 0x7f – -P -p 0xf --config "(0,0,0),(1,0,0),(2,0,0),(3,0,0)" -f a-aes-gcm-new.cfg --transfer-mode 1 --schedule-type 2 --process-mode 0 --process-dir 0 This series is targeted for next release (20.02). This series doesn't introduce any library change. And the decision to add eventmode additions in ipsec-secgw was approved by the Tech Board. Following are missing in the RFC. Will add it when sending patches. 1. Documentation. 2. More cleanup is needed. There are options that are added so that future expansion is not hindered. Need inputs from the community if there is use case for them. Following are planned features, 1. Add burst mode workers. 2. Add non tx internal port worker. 3. Verify support for Rx core (the support is added but lack of h/w to verify). 4. Add lookaside protocol support. Following are features that Marvell won't be attempting. 1. Inline crypto support. 2. Lookaside crypto support. For the features that Marvell won't be attempting, new workers can be introduced by the respective stake holders. Anoob Joseph (13): examples/ipsec-secgw: add framework for eventmode helper examples/ipsec-secgw: add eventdev port-lcore link examples/ipsec-secgw: add Rx adapter support examples/ipsec-secgw: add Tx adapter support examples/ipsec-secgw: add routines to display config examples/ipsec-secgw: add routines to launch workers examples/ipsec-secgw: add support for internal ports examples/ipsec-secgw: add eventmode to ipsec-secgw examples/ipsec-secgw: add app inbound worker examples/ipsec-secgw: add app processing code examples/ipsec-secgw: add driver outbound worker examples/ipsec-secgw: add app outbound worker examples/ipsec-secgw: add cmd line option for bufs examples/ipsec-secgw/Makefile | 2 + examples/ipsec-secgw/event_helper.c | 1757 +++++++++++++++++++++++++++++++++++ examples/ipsec-secgw/event_helper.h | 334 +++++++ examples/ipsec-secgw/ipsec-secgw.c | 436 +++++++-- examples/ipsec-secgw/ipsec-secgw.h | 81 ++ examples/ipsec-secgw/ipsec.c | 4 + examples/ipsec-secgw/ipsec.h | 30 +- examples/ipsec-secgw/ipsec_worker.c | 766 +++++++++++++++ examples/ipsec-secgw/ipsec_worker.h | 39 + examples/ipsec-secgw/meson.build | 4 +- examples/ipsec-secgw/sa.c | 11 - 11 files changed, 3360 insertions(+), 104 deletions(-) create mode 100644 examples/ipsec-secgw/event_helper.c create mode 100644 examples/ipsec-secgw/event_helper.h create mode 100644 examples/ipsec-secgw/ipsec-secgw.h create mode 100644 examples/ipsec-secgw/ipsec_worker.c create mode 100644 examples/ipsec-secgw/ipsec_worker.h