From patchwork Fri Dec 4 15:14:45 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jerin Jacob X-Patchwork-Id: 9347 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id A0A3B8E8E; Fri, 4 Dec 2015 16:16:48 +0100 (CET) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0095.outbound.protection.outlook.com [207.46.100.95]) by dpdk.org (Postfix) with ESMTP id 48B0A8E8B for ; Fri, 4 Dec 2015 16:16:47 +0100 (CET) Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@caviumnetworks.com; Received: from localhost.caveonetworks.com (111.93.218.67) by BN3PR0701MB1720.namprd07.prod.outlook.com (10.163.39.19) with Microsoft SMTP Server (TLS) id 15.1.337.19; Fri, 4 Dec 2015 15:16:43 +0000 From: Jerin Jacob To: Date: Fri, 4 Dec 2015 20:44:45 +0530 Message-ID: <1449242086-19051-3-git-send-email-jerin.jacob@caviumnetworks.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1449242086-19051-1-git-send-email-jerin.jacob@caviumnetworks.com> References: <1448904253-12929-1-git-send-email-jerin.jacob@caviumnetworks.com> <1449242086-19051-1-git-send-email-jerin.jacob@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: MAXPR01CA0068.INDPRD01.PROD.OUTLOOK.COM (25.164.146.168) To BN3PR0701MB1720.namprd07.prod.outlook.com (25.163.39.19) X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1720; 2:oXHrUQSZuFy/pHuQbGVyqzHcz3kuMv8zE2V1L5WK6eX2jdSj5krfENWDAnvUU1Vob/qDd1LQBguE4tC9Og+gffQw5M85za7aOSHO+B3S5QLbANQJHzPiO/b9L3Yf3iwgOp/BJmelnWNAdKDZlAnCqA==; 3:+98bc3hb+6VVUMu6j/PeIofrLRjSHM/+6cAUagQ+vUtLgVURcFHDAE0JsNQtVWWaAEghtHoHAjSSYx+GjBtpVJqcxn+5i0TrvxUYsxvHa1ntI0nAENX9jmrIYZcvd8Iy; 25:UWC8DtWZ73Hq+vGFyDNko6X5z7yxCMjmpQfz9TTQSfZVU3rGfIZN0XRg1CCPRGMFsmw9EieL+k789T9LRLCvqZ9Ef3W8bVix6Q51Ufv5hl1GB6B9ublZYiaUY6HQQAdnnpWNtyZfN5CPr/HbscAhKv3mXs9JX54aIsMFNoPRMepsecJnH/Em+bTsXxK3YFZupQ8xwHYL9414KmSP+VaIIolvHf2wzdwpKuudR7wXMtFJvEaI2tY+VaW1cYF6nrGdJlYzazxUfXb3Z6xkVDRK4Q== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1720; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1720; 20:MbXC+ov6UkEUjgvsGLaOCz/QXs4/d4DlOuNa7vB1sZ9jrMqUUcLzWSeHIVOJA+cce0dGDCu3CrbFtGrRKZaPIJUfutX+1j//POtbW32GMTAA9tvcjW19Yll1wtNZSMHqczkDN3IkcOO/IspZMv7WnHRYid2jaFk5UgAuEX646risAabX1NWLN/u3cSRWB62NFpVl1SDKGZ3ql92yLgU8VA6bAvDh/lLAX4HWGQT1tJJ+2LHuhyHY1NAypkUiuyZUWnnCEQzqn+lB3DBLyBOfKNO3cLpYsV91CEYlZq8YKBKZZGOva6p6YRb6Zq/92eaHisaDEe+qF8G/ERceJlrAoyYtcz24x1Vqk2dl2WxVe9brLc4kN+HxvtAP5DjEqhGijdVpKvEhgwR0BYBBr6YKaBsODgMLA0QEYCK20Dy7RB20gBnmjv5BqABLAwBuj7tDO0//EZhiPyQO1s5e+5fi78MU6FGbMEwKdExWBcgA9NJdXomRDNjWxl/ifSrjssQhPp6EBa2bBYnQ5FOaQj2UmB/71qcfwRLoXU8SjHaclrgNZquY2HVxvAKqv/5rvn9vyHnuRlLLEIL4LPLk7qj2v5/ZPqLC0e0nysjV443IFo8= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(236414709691187); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(520078)(8121501046)(3002001)(10201501046); SRVR:BN3PR0701MB1720; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1720; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1720; 4:WVC8W5e1lMij8TS224DBGxPYnmT39KWGmWwb0k7XvIjvCi3GMyhtnK67wxibgzhg/cQsiHeYoUBtzqjpKYe4GPjYxbLPnJLXgiO5B8Y1ycUv/Ti2+bTBNoXjPE1UIAD9NXnwYqfX3evsB3cKwXqWOij7uQxC7gUm+F3I/Uqeb4+SU0uPCyWZ5bwezhFB6oanBoDmH0UEGq2xM19Th8czWZ2Wq/sNoGjAf7EIdHouF6u7k0z52oFzmJ81Pg6DvOrprLafWWy/FqBZGNAD4H0LWjtEBewMa4BDqSYZuobqwJRH0MkJekifqJj825qo42DL4pDL88Up5U8H9xsvx50ZwiopQai2Ln14h5c92Bszcm6KmNmkL9VIcsSubNZuKwZWB8CSFWv3QfBHssz65jnHCwdAX45j2bsi7eH5Pf1Tv2MGinRTGCE5TUcqQeOvQMY4 X-Forefront-PRVS: 07807C55DC X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(6009001)(189002)(199003)(101416001)(42186005)(53416004)(122386002)(107886002)(110136002)(50986999)(5001960100002)(76176999)(189998001)(97736004)(92566002)(2950100001)(77096005)(81156007)(40100003)(106356001)(105586002)(5009440100003)(50226001)(47776003)(87976001)(19580405001)(229853001)(6116002)(5003940100001)(50466002)(36756003)(48376002)(33646002)(19580395003)(3846002)(4001430100002)(66066001)(5008740100001)(1096002)(2351001)(69596002)(76506005)(5004730100002)(86362001)(586003)(7099028); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1720; H:localhost.caveonetworks.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: caviumnetworks.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0701MB1720; 23:zvlSm3M0qcLIPPpQI9hWbgLvYGZg5Mm3Dvmn/eR?= =?us-ascii?Q?WpFpiFqfWqZKVcC6T2MZZz8sUHaREL+EuBAK8nTGSiageRJQ4aHHZJdw260Z?= =?us-ascii?Q?SW+5bG/9K7gpXxBO4DLeXoejKiDUr7TpyLtgg85rasc3M+FhGMKsU8DZvLXE?= =?us-ascii?Q?cixBXGeF4jws487Ex6517AG5EdFeXcxOAuUAUOTQnAlcZbaFJoYxPqwqVVit?= =?us-ascii?Q?xwD+y6L4F/s8hTlZNPlNufgmDYn6URKrVnLxZHSlg9ADDQ/Pk0aiyNgkxcic?= =?us-ascii?Q?fXaR2DnX4ql3nE1G4kgpaPNXzrq1MsSeh8LZ8IZ1+w+Jnrdus9ZHVxNwwkxt?= =?us-ascii?Q?c0S6BpRIrvGJYXjKj83LtyiCuQ0wF6YjiWapqiVLf40kyVzrB+/W4dwW5RVe?= =?us-ascii?Q?AeYLHNXI529jCQON2zkS+40VAHEP3Z+WwasLQxvWT4E+lz+w7t4SjOczQeLW?= =?us-ascii?Q?aL85CWPPoBMMCODa0cSWzv+EQYJpcnI12UrZTYiUkFcdZ1I6wYpbuSgHwWNO?= =?us-ascii?Q?9gHa6Eg7WryjpWcc4VT4qPvC4Fmi15OvHzAqXZP1H4De2aFtIlzw+Nu6jnih?= =?us-ascii?Q?EmjRFwnlbNv8EoeaPi2ueDT9r26MDfwG5+R0qCDx+nT6q6sy90mcKG5ZKfcc?= =?us-ascii?Q?nnUkY+q/9NfP37BrYQrQGXUEN9+YLecUNAvI4k7yDilnj1xRC29op9YDQ3G6?= =?us-ascii?Q?k/AngqJCrS9ycKI5Bcw3vc8615qSZSalefklvTusB933vEfjoIzhhcGxSICD?= =?us-ascii?Q?p1tPF6ZDB9/ixEGLNSM9pF3RcW8+76uy5balq1HSkB/KMQ8+hpQulr/Z1P7i?= =?us-ascii?Q?5AdAxzPd4GzYQ1DU7u8WgxQqJH1cPlMwYWMus6Jo+qH0NrhdYlequ+2Em1oH?= =?us-ascii?Q?XNSt9fHHew5IX8RTRqUe3go5H6zMCOZTHPQrXUMVg8xjr7AfI8FqR5GXScdt?= =?us-ascii?Q?vVkQDNXARwk11s0rg+HM7dx9AotnWFxXvPNBg7HYyb3XM4bnsVnKYDRvd7t8?= =?us-ascii?Q?leu+dnXTqbvDvxe9PEVAJEMQWZpgFCWoNoKFA7dg7VmnmBXgk3r7T+jLBvom?= =?us-ascii?Q?2v+tl3KS3aeY/Y0xzYifH9oDV1d2LNpd0CTLWNBXPlYR9Bc6n4HW7b1/iq/g?= =?us-ascii?Q?xvM/iBLtRC2vIrtrQxbRVAjV8n0r6Fg9T8b0IEWmZH5sb+YNlErB12XIlkYs?= =?us-ascii?Q?EP06brv00KnyPRjx5PmEdvva+WI1CeiipsNPO+9MGOyqqGgyrv/wLvaAL+Q?= =?us-ascii?Q?=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1720; 5:xhfuMIJP2VBqUXYmddmpEPkdpJgNlHMHGoJPIz2v6a0vAC+/9CP0fvCXOxMTJ+sQdEN4SC3sswyU+Knh/zJG2oJhcbMWA6Xw56dydvVTP9JBrUIfde0rNzOIrFMJPmpsiU/U5lV8g3WIDCapyTCfZA==; 24:MYfh685to/SiFRtKz9D3pHjOoHZRM0VwaaIBO7beuW7kHESHFnoYk7LHJ8LjKAKh3kCHdlGyVeNn/MQe1d3jO5xQZc+rE0fwpjl1qFCyQ/E= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Dec 2015 15:16:43.4935 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1720 Subject: [dpdk-dev] [PATCH v2 2/3] lpm: add support for NEON X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Enabled CONFIG_RTE_LIBRTE_LPM, CONFIG_RTE_LIBRTE_TABLE, CONFIG_RTE_LIBRTE_PIPELINE libraries for arm and arm64 TABLE, PIPELINE libraries were disabled due to LPM library dependency. Signed-off-by: Jerin Jacob Signed-off-by: Jianbo Liu --- app/test/test_xmmt_ops.h | 20 ++++ config/defconfig_arm-armv7a-linuxapp-gcc | 3 - config/defconfig_arm64-armv8a-linuxapp-gcc | 3 - lib/librte_lpm/Makefile | 4 + lib/librte_lpm/rte_lpm.h | 4 + lib/librte_lpm/rte_lpm_neon.h | 148 +++++++++++++++++++++++++++++ 6 files changed, 176 insertions(+), 6 deletions(-) create mode 100644 lib/librte_lpm/rte_lpm_neon.h diff --git a/app/test/test_xmmt_ops.h b/app/test/test_xmmt_ops.h index c055912..c18fc12 100644 --- a/app/test/test_xmmt_ops.h +++ b/app/test/test_xmmt_ops.h @@ -36,6 +36,24 @@ #include +#if defined(RTE_ARCH_ARM) || defined(RTE_ARCH_ARM64) + +/* vect_* abstraction implementation using NEON */ + +/* loads the xmm_t value from address p(does not need to be 16-byte aligned)*/ +#define vect_loadu_sil128(p) vld1q_s32((const int32_t *)p) + +/* sets the 4 signed 32-bit integer values and returns the xmm_t variable */ +static inline xmm_t __attribute__((always_inline)) +vect_set_epi32(int i3, int i2, int i1, int i0) +{ + int32_t data[4] = {i0, i1, i2, i3}; + + return vld1q_s32(data); +} + +#else + /* vect_* abstraction implementation using SSE */ /* loads the xmm_t value from address p(does not need to be 16-byte aligned)*/ @@ -44,4 +62,6 @@ /* sets the 4 signed 32-bit integer values and returns the xmm_t variable */ #define vect_set_epi32(i3, i2, i1, i0) _mm_set_epi32(i3, i2, i1, i0) +#endif + #endif /* _TEST_XMMT_OPS_H_ */ diff --git a/config/defconfig_arm-armv7a-linuxapp-gcc b/config/defconfig_arm-armv7a-linuxapp-gcc index 9924ff9..cdbf4ac 100644 --- a/config/defconfig_arm-armv7a-linuxapp-gcc +++ b/config/defconfig_arm-armv7a-linuxapp-gcc @@ -54,9 +54,6 @@ CONFIG_RTE_EAL_IGB_UIO=n # fails to compile on ARM CONFIG_RTE_LIBRTE_ACL=n -CONFIG_RTE_LIBRTE_LPM=n -CONFIG_RTE_LIBRTE_TABLE=n -CONFIG_RTE_LIBRTE_PIPELINE=n CONFIG_RTE_SCHED_VECTOR=n # cannot use those on ARM diff --git a/config/defconfig_arm64-armv8a-linuxapp-gcc b/config/defconfig_arm64-armv8a-linuxapp-gcc index 504f3ed..57f7941 100644 --- a/config/defconfig_arm64-armv8a-linuxapp-gcc +++ b/config/defconfig_arm64-armv8a-linuxapp-gcc @@ -51,7 +51,4 @@ CONFIG_RTE_LIBRTE_IVSHMEM=n CONFIG_RTE_LIBRTE_FM10K_PMD=n CONFIG_RTE_LIBRTE_I40E_PMD=n -CONFIG_RTE_LIBRTE_LPM=n -CONFIG_RTE_LIBRTE_TABLE=n -CONFIG_RTE_LIBRTE_PIPELINE=n CONFIG_RTE_SCHED_VECTOR=n diff --git a/lib/librte_lpm/Makefile b/lib/librte_lpm/Makefile index ce3a1d1..7f93006 100644 --- a/lib/librte_lpm/Makefile +++ b/lib/librte_lpm/Makefile @@ -47,7 +47,11 @@ SRCS-$(CONFIG_RTE_LIBRTE_LPM) := rte_lpm.c rte_lpm6.c # install this header file SYMLINK-$(CONFIG_RTE_LIBRTE_LPM)-include := rte_lpm.h rte_lpm6.h +ifneq ($(filter y,$(CONFIG_RTE_ARCH_ARM) $(CONFIG_RTE_ARCH_ARM64)),) +SYMLINK-$(CONFIG_RTE_LIBRTE_LPM)-include += rte_lpm_neon.h +else SYMLINK-$(CONFIG_RTE_LIBRTE_LPM)-include += rte_lpm_sse.h +endif # this lib needs eal DEPDIRS-$(CONFIG_RTE_LIBRTE_LPM) += lib/librte_eal diff --git a/lib/librte_lpm/rte_lpm.h b/lib/librte_lpm/rte_lpm.h index dfe1378..0c892de 100644 --- a/lib/librte_lpm/rte_lpm.h +++ b/lib/librte_lpm/rte_lpm.h @@ -384,7 +384,11 @@ static inline void rte_lpm_lookupx4(const struct rte_lpm *lpm, xmm_t ip, uint16_t hop[4], uint16_t defv); +#if defined(RTE_ARCH_ARM) || defined(RTE_ARCH_ARM64) +#include "rte_lpm_neon.h" +#else #include "rte_lpm_sse.h" +#endif #ifdef __cplusplus } diff --git a/lib/librte_lpm/rte_lpm_neon.h b/lib/librte_lpm/rte_lpm_neon.h new file mode 100644 index 0000000..fcd2a8a --- /dev/null +++ b/lib/librte_lpm/rte_lpm_neon.h @@ -0,0 +1,148 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Cavium Networks. All rights reserved. + * All rights reserved. + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Derived rte_lpm_lookupx4 implementation from lib/librte_lpm/rte_lpm_sse.h + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Cavium Networks nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_LPM_NEON_H_ +#define _RTE_LPM_NEON_H_ + +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +static inline void +rte_lpm_lookupx4(const struct rte_lpm *lpm, xmm_t ip, uint16_t hop[4], + uint16_t defv) +{ + uint32x4_t i24; + rte_xmm_t i8; + uint16_t tbl[4]; + uint64_t idx, pt; + + const uint32_t mask = UINT8_MAX; + const int32x4_t mask8 = vdupq_n_s32(mask); + + /* + * RTE_LPM_VALID_EXT_ENTRY_BITMASK for 4 LPM entries + * as one 64-bit value (0x0300030003000300). + */ + const uint64_t mask_xv = + ((uint64_t)RTE_LPM_VALID_EXT_ENTRY_BITMASK | + (uint64_t)RTE_LPM_VALID_EXT_ENTRY_BITMASK << 16 | + (uint64_t)RTE_LPM_VALID_EXT_ENTRY_BITMASK << 32 | + (uint64_t)RTE_LPM_VALID_EXT_ENTRY_BITMASK << 48); + + /* + * RTE_LPM_LOOKUP_SUCCESS for 4 LPM entries + * as one 64-bit value (0x0100010001000100). + */ + const uint64_t mask_v = + ((uint64_t)RTE_LPM_LOOKUP_SUCCESS | + (uint64_t)RTE_LPM_LOOKUP_SUCCESS << 16 | + (uint64_t)RTE_LPM_LOOKUP_SUCCESS << 32 | + (uint64_t)RTE_LPM_LOOKUP_SUCCESS << 48); + + /* get 4 indexes for tbl24[]. */ + i24 = vshrq_n_u32((uint32x4_t)ip, CHAR_BIT); + + /* extract values from tbl24[] */ + idx = vgetq_lane_u64((uint64x2_t)i24, 0); + + tbl[0] = *(const uint16_t *)&lpm->tbl24[(uint32_t)idx]; + tbl[1] = *(const uint16_t *)&lpm->tbl24[idx >> 32]; + + idx = vgetq_lane_u64((uint64x2_t)i24, 1); + + tbl[2] = *(const uint16_t *)&lpm->tbl24[(uint32_t)idx]; + tbl[3] = *(const uint16_t *)&lpm->tbl24[idx >> 32]; + + /* get 4 indexes for tbl8[]. */ + i8.x = vandq_s32(ip, mask8); + + pt = (uint64_t)tbl[0] | + (uint64_t)tbl[1] << 16 | + (uint64_t)tbl[2] << 32 | + (uint64_t)tbl[3] << 48; + + /* search successfully finished for all 4 IP addresses. */ + if (likely((pt & mask_xv) == mask_v)) { + uintptr_t ph = (uintptr_t)hop; + *(uint64_t *)ph = pt & RTE_LPM_MASKX4_RES; + return; + } + + if (unlikely((pt & RTE_LPM_VALID_EXT_ENTRY_BITMASK) == + RTE_LPM_VALID_EXT_ENTRY_BITMASK)) { + i8.u32[0] = i8.u32[0] + + (uint8_t)tbl[0] * RTE_LPM_TBL8_GROUP_NUM_ENTRIES; + tbl[0] = *(const uint16_t *)&lpm->tbl8[i8.u32[0]]; + } + if (unlikely((pt >> 16 & RTE_LPM_VALID_EXT_ENTRY_BITMASK) == + RTE_LPM_VALID_EXT_ENTRY_BITMASK)) { + i8.u32[1] = i8.u32[1] + + (uint8_t)tbl[1] * RTE_LPM_TBL8_GROUP_NUM_ENTRIES; + tbl[1] = *(const uint16_t *)&lpm->tbl8[i8.u32[1]]; + } + if (unlikely((pt >> 32 & RTE_LPM_VALID_EXT_ENTRY_BITMASK) == + RTE_LPM_VALID_EXT_ENTRY_BITMASK)) { + i8.u32[2] = i8.u32[2] + + (uint8_t)tbl[2] * RTE_LPM_TBL8_GROUP_NUM_ENTRIES; + tbl[2] = *(const uint16_t *)&lpm->tbl8[i8.u32[2]]; + } + if (unlikely((pt >> 48 & RTE_LPM_VALID_EXT_ENTRY_BITMASK) == + RTE_LPM_VALID_EXT_ENTRY_BITMASK)) { + i8.u32[3] = i8.u32[3] + + (uint8_t)tbl[3] * RTE_LPM_TBL8_GROUP_NUM_ENTRIES; + tbl[3] = *(const uint16_t *)&lpm->tbl8[i8.u32[3]]; + } + + hop[0] = (tbl[0] & RTE_LPM_LOOKUP_SUCCESS) ? (uint8_t)tbl[0] : defv; + hop[1] = (tbl[1] & RTE_LPM_LOOKUP_SUCCESS) ? (uint8_t)tbl[1] : defv; + hop[2] = (tbl[2] & RTE_LPM_LOOKUP_SUCCESS) ? (uint8_t)tbl[2] : defv; + hop[3] = (tbl[3] & RTE_LPM_LOOKUP_SUCCESS) ? (uint8_t)tbl[3] : defv; +} + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_LPM_NEON_H_ */