From patchwork Mon Nov 30 17:24:12 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jerin Jacob X-Patchwork-Id: 9186 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id E27728DAC; Mon, 30 Nov 2015 18:24:53 +0100 (CET) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0060.outbound.protection.outlook.com [157.56.110.60]) by dpdk.org (Postfix) with ESMTP id B77468DA3 for ; Mon, 30 Nov 2015 18:24:51 +0100 (CET) Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@caviumnetworks.com; Received: from localhost.localdomain.localdomain (122.167.201.210) by BLUPR0701MB1714.namprd07.prod.outlook.com (10.163.85.140) with Microsoft SMTP Server (TLS) id 15.1.331.20; Mon, 30 Nov 2015 17:24:48 +0000 From: Jerin Jacob To: Date: Mon, 30 Nov 2015 22:54:12 +0530 Message-ID: <1448904253-12929-3-git-send-email-jerin.jacob@caviumnetworks.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1448904253-12929-1-git-send-email-jerin.jacob@caviumnetworks.com> References: <1448904253-12929-1-git-send-email-jerin.jacob@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [122.167.201.210] X-ClientProxiedBy: MAXPR01CA0023.INDPRD01.PROD.OUTLOOK.COM (25.164.147.30) To BLUPR0701MB1714.namprd07.prod.outlook.com (25.163.85.140) X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1714; 2:9S+YLnuBKXbIvbQW38dTpfisYs0byaoSzedEH0RVumpUIrR4BCdyYqN+2lw6QDjbNDmESi5GC4LjvN/Eus6aWFjg2n5azB6+URZJjDCjEE5DEPcL6Ey9a9oJgKy2VGzG3L1xgSb9yuLpPAy7ugx1aw==; 3:afMqfSVe8O4h04PpcC2/Wx6GwVEmeoScpR5If4ySJPQ5ngmH7LAPt34hHkNrqg7PCMfSK6gkGStWeaZjVrWX0jeUfbcDPRM+ogGGDbSYgGsFKHZvc6TwzYEH+ePvO0Qg; 25:5e83tjxP7XulrfHPNbL5/sEWNeueAOUhULSzz4QeEIgNLGJs4CLLahwbzJzFIQ0T8w1sWWw6t2KdWcTGxZJnJx8qkRm+VLToImalaYU+N4PCdS3cfQsRDsMOQ6rhAh+WUZ0EXx5E6jFzEI8jgi30qZaUQog6jRBThE10Wmz7cE5npwwz4Pnj7MSgQ3FdSLbpD5xggvGiEa6Mw5H4ze+qH2eLu3BsVkFEB74WBb69KA1Nl53OOqSc30nJvvH1SrD7cqACajmN0wubCvcKv6tfeA== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR0701MB1714; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1714; 20:acqaZ27XoySqBHktXtzOCVciRHa/KTa2bsexiRwjYZQ+CVMrDDhAzY5ZK6Q7Pl3DEpqRJFtcEoEEqNbEwQgO5GQbgreTmDs6Qc5em+ZZots/8wvpJTeZY8sHmvJ/041O2nPx/gDOQQcUr1WlIIOqNbaSJvfuU61lY20RlfC3Kcy3D6Bjuvkhmv2Me4ndKg3Dkk2oYix/oSx+8QumVs2VD7EDRYsjFzlpikjW15lXpkj/vvw+DFv9dthwMxb8+ZY89bnxc7udfb4+8ksaUFcw1njp3ykZiXhCq+9LnYW1s+R/U99/hDKT3XkmUaXVRuKJoBFv3luDPbvLP83263CuuZAuzhYaYrG5WmTc/LwtaDS08aBLmgWafKOWak58Abx/Emdnxaa62em4OLbv4aQ9UoVmX3mlHNBxVKF5rn8AwKycMs/b/kSF3EmBpKMByczcYpRO4YvFwvDyU0jPY1ZZsQOKBypcJZf7AfJUiF7KE87kDwS6G5YKs1EMq7su7sp+zvtz1a8tneUZgULhpcA7hVoJGA4UPAVWju6lh6KWIejwa/qQfaM49cCH9s0U99vCXZVJ+2mJhzO4VOgMG/PsD0Dj2Bcajk47oXA0RW+H/Mk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(236414709691187); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(520078)(10201501046)(3002001); SRVR:BLUPR0701MB1714; BCL:0; PCL:0; RULEID:; SRVR:BLUPR0701MB1714; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1714; 4:JbOOM/HMojVzWJ2hXe5gBZDRe9GNe3eWUANXDbwEJq8uslNpLhOZ6J4VSI9amYwcsoW0rtC/5lgcV2eYwW5gYa9Zz5a97zg/Aae9Qi4mMZd7sBSU4vMCtSOpXfmHiv5T1/0QHOMXY4E/ICofGnNOPGP/6FK+imbuPpw3Zezx/tyOZDZz8JR+7nS4rfqsTrYhip5zskxNjRBXKTtguYAdwCIvfQ1f+7BEckhpZkZtaw5hHgTzohKKVUmVYS/s6cEINV3sIhdbXMVXs1hXtxYWA3Mbh3W6D23Xs3Q/fYit83CbOpygIEq1scnJHiFC0WFUSmWPJEmFvt9Q7dqK18z6bd8nAV/cRtw55YYNlPCw3MjfNjERTRS7Caj3aKBFbrDPJ1bKoG2/3BMRufqFniTmX6HUzcrVIglwZWBNBSElsIqsaa1ZwYkgQQrB+JlaxhL6 X-Forefront-PRVS: 0776C39A48 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(6069001)(189002)(199003)(2351001)(42186005)(107886002)(48376002)(110136002)(5004730100002)(33646002)(5008740100001)(229853001)(47776003)(66066001)(76176999)(50466002)(97736004)(19580405001)(36756003)(106356001)(101416001)(105586002)(92566002)(81156007)(50226001)(19580395003)(5001960100002)(87976001)(189998001)(50986999)(2950100001)(86362001)(77096005)(6116002)(4001430100002)(3846002)(5003940100001)(40100003)(586003)(1096002)(122386002)(7099028); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR0701MB1714; H:localhost.localdomain.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: caviumnetworks.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BLUPR0701MB1714; 23:fiv81XJB/0b/IAfK62oP81+cLxSWfQH+UxFUv9u?= =?us-ascii?Q?P8ry3+GEq/r9rNMX8WNJkF2bVCHQrPNWc7gkko3KY2VSTNRggy4AWse67UYb?= =?us-ascii?Q?sDKEYZnT9//iGWW40lpSBKctEBIAueLNsSRbDzib8e6C+hlvnT+qW703WuXM?= =?us-ascii?Q?gav1Iz/3iPfEuXuUh5ccaiRSUsh+UvHQsG/fhMhhwCkwBVZTvvDj1hdo+NEY?= =?us-ascii?Q?QDMVjHYMBjcDLVm08uF1yH4Qu/6GHrUqwCUfJJ3yXVeq9ChWiV0h56pwmCKj?= =?us-ascii?Q?RK5HSTQLmod6ZGLKW+TCFwKUYkEpCSXiNnr0Qxq2Z0TXxxXKHYmNKUaKZhCx?= =?us-ascii?Q?FCNUh0G6aBh2rk9jh2EYAwaysJsAZa2KbyO5qdO9liNc4q5vWERHilTi+scm?= =?us-ascii?Q?0dsPE1eBwDe6OSCASUc6RvI667LlfxouCnGlcPPp8zeDZ+QckGaacRYP27GT?= =?us-ascii?Q?syWSJKdECaJWmpWEQ7sdxcNpbhcTOYbZtH+OJeKB+g4mMqBPyQXQXRO8ZTPC?= =?us-ascii?Q?wiRJoIBCM1N31KmCK5583sCT2ubk/tc6vEkGX4JmH8gDdCqAFf0N8PSVQW95?= =?us-ascii?Q?HhYoMtbZogqm7seF+p4JvjD0ALvP1OWygWe/R3vsOkF5U2qvp31WRtfyJkR/?= =?us-ascii?Q?T+NoIbjqEkMZ3MlSWdUyZ26JPC+RrGrCQka6xd6G/efu0lodeIFlybotEVkK?= =?us-ascii?Q?ivGkyJXY0RPYWLjyyaU4U2i/khIDiqFhIX28xxdCFsJffVHUyKD+Vki0Ev/n?= =?us-ascii?Q?ZsE68ZEHSvsgmYtjWvtKCXwwsFgluIL/gL8EvB/cgGbPCGx0hcEmYoZjqYaF?= =?us-ascii?Q?Lxk2uULXFNZqCNUAW9o6Ej70InEsr7UA1w+MvSCcNeSLYpM2EKeABjy9YNgw?= =?us-ascii?Q?/lci4fmwSwoND6yheoMYQgB9kO99bbm1ybwbFXWbedl1vkDcbh/F/GmgNPCq?= =?us-ascii?Q?os/Rp6RMa3r8jwxmvlJ16Mr9wpjqhkFjnuyTl7z5YBId1NyJ8BUc9p/kZ+PX?= =?us-ascii?Q?ukztuM1+iLiMLL+AkkMXAhC6+9ZUIZrsudwyijDmFBpGYoVoTr9IyaTSMlBN?= =?us-ascii?Q?UMSjpcxQjQUtVXQuzg1mX2qIVQortpXH0Zl2TmsLSyMv36DyAp71HlbteArt?= =?us-ascii?Q?z/s9P+1TJ/JajFT+X0wq3hFlpZC+WriUU?= X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1714; 5:WpHrMJMZQf6T2WITtuDh6Fwbh8r6TOiD6cZIso9as0hAQg6qy1FRkVO58O9LQAfps9Lt7in8xqDFzxHzfHsc+I4PFf4rrRFbqkIt5yvNOaZgNVjuOO4l6hWTJdjRqDnxYjVSM+Ye4iuz6fLO93UtDg==; 24:AGAr9/FJPqWWVTPVJiNcXnFl6nJBrjyxYQoTJninH2Jx6FEQy4YFILSuPiMIyX4tSCk9UtJsowFl7hc96myHUAta+Of8tPDwO1KLwGTu7io= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2015 17:24:48.1226 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0701MB1714 Subject: [dpdk-dev] [PATCH 2/3] lpm: add support for NEON X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" enabled CONFIG_RTE_LIBRTE_LPM, CONFIG_RTE_LIBRTE_TABLE, CONFIG_RTE_LIBRTE_PIPELINE libraries for arm64. TABLE, PIPELINE libraries were disabled due to LPM library dependency. Signed-off-by: Jerin Jacob --- app/test/test_lpm.c | 10 +- config/defconfig_arm64-armv8a-linuxapp-gcc | 3 - lib/librte_lpm/Makefile | 3 + lib/librte_lpm/rte_lpm.h | 5 + lib/librte_lpm/rte_lpm_neon.h | 172 +++++++++++++++++++++++++++++ 5 files changed, 185 insertions(+), 8 deletions(-) create mode 100644 lib/librte_lpm/rte_lpm_neon.h diff --git a/app/test/test_lpm.c b/app/test/test_lpm.c index 8b4ded9..207301b 100644 --- a/app/test/test_lpm.c +++ b/app/test/test_lpm.c @@ -324,7 +324,7 @@ test7(void) status = rte_lpm_lookup(lpm, ip, &next_hop_return); TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add)); - ipx4 = _mm_set_epi32(ip, ip + 0x100, ip - 0x100, ip); + ipx4 = rte_vect_set_epi32(ip, ip + 0x100, ip - 0x100, ip); rte_lpm_lookupx4(lpm, ipx4, hop, UINT16_MAX); TEST_LPM_ASSERT(hop[0] == next_hop_add); TEST_LPM_ASSERT(hop[1] == UINT16_MAX); @@ -380,7 +380,7 @@ test8(void) TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add)); - ipx4 = _mm_set_epi32(ip2, ip1, ip2, ip1); + ipx4 = rte_vect_set_epi32(ip2, ip1, ip2, ip1); rte_lpm_lookupx4(lpm, ipx4, hop, UINT16_MAX); TEST_LPM_ASSERT(hop[0] == UINT16_MAX); TEST_LPM_ASSERT(hop[1] == next_hop_add); @@ -408,7 +408,7 @@ test8(void) status = rte_lpm_lookup(lpm, ip1, &next_hop_return); TEST_LPM_ASSERT(status == -ENOENT); - ipx4 = _mm_set_epi32(ip1, ip1, ip2, ip2); + ipx4 = rte_vect_set_epi32(ip1, ip1, ip2, ip2); rte_lpm_lookupx4(lpm, ipx4, hop, UINT16_MAX); if (depth != 1) { TEST_LPM_ASSERT(hop[0] == next_hop_add); @@ -872,7 +872,7 @@ test12(void) TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add)); - ipx4 = _mm_set_epi32(ip, ip + 1, ip, ip - 1); + ipx4 = rte_vect_set_epi32(ip, ip + 1, ip, ip - 1); rte_lpm_lookupx4(lpm, ipx4, hop, UINT16_MAX); TEST_LPM_ASSERT(hop[0] == UINT16_MAX); TEST_LPM_ASSERT(hop[1] == next_hop_add); @@ -1291,7 +1291,7 @@ perf_test(void) unsigned k; __m128i ipx4; - ipx4 = _mm_loadu_si128((__m128i *)(ip_batch + j)); + ipx4 = rte_vect_loadu_sil128((__m128i *)(ip_batch + j)); ipx4 = *(__m128i *)(ip_batch + j); rte_lpm_lookupx4(lpm, ipx4, next_hops, UINT16_MAX); for (k = 0; k < RTE_DIM(next_hops); k++) diff --git a/config/defconfig_arm64-armv8a-linuxapp-gcc b/config/defconfig_arm64-armv8a-linuxapp-gcc index 504f3ed..57f7941 100644 --- a/config/defconfig_arm64-armv8a-linuxapp-gcc +++ b/config/defconfig_arm64-armv8a-linuxapp-gcc @@ -51,7 +51,4 @@ CONFIG_RTE_LIBRTE_IVSHMEM=n CONFIG_RTE_LIBRTE_FM10K_PMD=n CONFIG_RTE_LIBRTE_I40E_PMD=n -CONFIG_RTE_LIBRTE_LPM=n -CONFIG_RTE_LIBRTE_TABLE=n -CONFIG_RTE_LIBRTE_PIPELINE=n CONFIG_RTE_SCHED_VECTOR=n diff --git a/lib/librte_lpm/Makefile b/lib/librte_lpm/Makefile index 688cfc9..2fd5305 100644 --- a/lib/librte_lpm/Makefile +++ b/lib/librte_lpm/Makefile @@ -46,6 +46,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_LPM) := rte_lpm.c rte_lpm6.c # install this header file SYMLINK-$(CONFIG_RTE_LIBRTE_LPM)-include := rte_lpm.h rte_lpm6.h +ifeq ($(CONFIG_RTE_ARCH_ARM64),y) +SYMLINK-$(CONFIG_RTE_LIBRTE_LPM)-include += rte_lpm_neon.h +endif # this lib needs eal DEPDIRS-$(CONFIG_RTE_LIBRTE_LPM) += lib/librte_eal diff --git a/lib/librte_lpm/rte_lpm.h b/lib/librte_lpm/rte_lpm.h index c299ce2..12b75ce 100644 --- a/lib/librte_lpm/rte_lpm.h +++ b/lib/librte_lpm/rte_lpm.h @@ -361,6 +361,9 @@ rte_lpm_lookup_bulk_func(const struct rte_lpm *lpm, const uint32_t * ips, /* Mask four results. */ #define RTE_LPM_MASKX4_RES UINT64_C(0x00ff00ff00ff00ff) +#if defined(RTE_ARCH_ARM64) +#include "rte_lpm_neon.h" +#else /** * Lookup four IP addresses in an LPM table. * @@ -473,6 +476,8 @@ rte_lpm_lookupx4(const struct rte_lpm *lpm, __m128i ip, uint16_t hop[4], hop[3] = (tbl[3] & RTE_LPM_LOOKUP_SUCCESS) ? (uint8_t)tbl[3] : defv; } +#endif + #ifdef __cplusplus } #endif diff --git a/lib/librte_lpm/rte_lpm_neon.h b/lib/librte_lpm/rte_lpm_neon.h new file mode 100644 index 0000000..6ec4255 --- /dev/null +++ b/lib/librte_lpm/rte_lpm_neon.h @@ -0,0 +1,172 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2015 Cavium Networks. All rights reserved. + * All rights reserved. + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Derived rte_lpm_lookupx4 implementation from lib/librte_lpm/rte_lpm.h + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Cavium Networks nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_LPM_NEON_H_ +#define _RTE_LPM_NEON_H_ + +/** + * @file + * RTE Longest Prefix Match (LPM) lookup for neon + */ + +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#ifdef __cplusplus +} +#endif + +/** + * Lookup four IP addresses in an LPM table. + * + * @param lpm + * LPM object handle + * @param ip + * Four IPs to be looked up in the LPM table + * @param hop + * Next hop of the most specific rule found for IP (valid on lookup hit only). + * This is an 4 elements array of two byte values. + * If the lookup was succesfull for the given IP, then least significant byte + * of the corresponding element is the actual next hop and the most + * significant byte is zero. + * If the lookup for the given IP failed, then corresponding element would + * contain default value, see description of then next parameter. + * @param defv + * Default value to populate into corresponding element of hop[] array, + * if lookup would fail. + */ +static inline void +rte_lpm_lookupx4(const struct rte_lpm *lpm, __m128i ip, uint16_t hop[4], + uint16_t defv) +{ + uint32x4_t i24; + rte_xmm_t i8; + uint16_t tbl[4]; + uint64_t idx, pt; + + const uint32_t mask = UINT8_MAX; + const int32x4_t mask8 = vdupq_n_s32(mask); + + /* + * RTE_LPM_VALID_EXT_ENTRY_BITMASK for 4 LPM entries + * as one 64-bit value (0x0300030003000300). + */ + const uint64_t mask_xv = + ((uint64_t)RTE_LPM_VALID_EXT_ENTRY_BITMASK | + (uint64_t)RTE_LPM_VALID_EXT_ENTRY_BITMASK << 16 | + (uint64_t)RTE_LPM_VALID_EXT_ENTRY_BITMASK << 32 | + (uint64_t)RTE_LPM_VALID_EXT_ENTRY_BITMASK << 48); + + /* + * RTE_LPM_LOOKUP_SUCCESS for 4 LPM entries + * as one 64-bit value (0x0100010001000100). + */ + const uint64_t mask_v = + ((uint64_t)RTE_LPM_LOOKUP_SUCCESS | + (uint64_t)RTE_LPM_LOOKUP_SUCCESS << 16 | + (uint64_t)RTE_LPM_LOOKUP_SUCCESS << 32 | + (uint64_t)RTE_LPM_LOOKUP_SUCCESS << 48); + + /* get 4 indexes for tbl24[]. */ + i24 = vshrq_n_u32((uint32x4_t)ip, CHAR_BIT); + + /* extract values from tbl24[] */ + idx = vgetq_lane_u64((uint64x2_t)i24, 0); + + tbl[0] = *(const uint16_t *)&lpm->tbl24[(uint32_t)idx]; + tbl[1] = *(const uint16_t *)&lpm->tbl24[idx >> 32]; + + idx = vgetq_lane_u64((uint64x2_t)i24, 1); + + tbl[2] = *(const uint16_t *)&lpm->tbl24[(uint32_t)idx]; + tbl[3] = *(const uint16_t *)&lpm->tbl24[idx >> 32]; + + /* get 4 indexes for tbl8[]. */ + i8.x = vandq_s32(ip, mask8); + + pt = (uint64_t)tbl[0] | + (uint64_t)tbl[1] << 16 | + (uint64_t)tbl[2] << 32 | + (uint64_t)tbl[3] << 48; + + /* search successfully finished for all 4 IP addresses. */ + if (likely((pt & mask_xv) == mask_v)) { + uintptr_t ph = (uintptr_t)hop; + *(uint64_t *)ph = pt & RTE_LPM_MASKX4_RES; + return; + } + + if (unlikely((pt & RTE_LPM_VALID_EXT_ENTRY_BITMASK) == + RTE_LPM_VALID_EXT_ENTRY_BITMASK)) { + i8.u32[0] = i8.u32[0] + + (uint8_t)tbl[0] * RTE_LPM_TBL8_GROUP_NUM_ENTRIES; + tbl[0] = *(const uint16_t *)&lpm->tbl8[i8.u32[0]]; + } + if (unlikely((pt >> 16 & RTE_LPM_VALID_EXT_ENTRY_BITMASK) == + RTE_LPM_VALID_EXT_ENTRY_BITMASK)) { + i8.u32[1] = i8.u32[1] + + (uint8_t)tbl[1] * RTE_LPM_TBL8_GROUP_NUM_ENTRIES; + tbl[1] = *(const uint16_t *)&lpm->tbl8[i8.u32[1]]; + } + if (unlikely((pt >> 32 & RTE_LPM_VALID_EXT_ENTRY_BITMASK) == + RTE_LPM_VALID_EXT_ENTRY_BITMASK)) { + i8.u32[2] = i8.u32[2] + + (uint8_t)tbl[2] * RTE_LPM_TBL8_GROUP_NUM_ENTRIES; + tbl[2] = *(const uint16_t *)&lpm->tbl8[i8.u32[2]]; + } + if (unlikely((pt >> 48 & RTE_LPM_VALID_EXT_ENTRY_BITMASK) == + RTE_LPM_VALID_EXT_ENTRY_BITMASK)) { + i8.u32[3] = i8.u32[3] + + (uint8_t)tbl[3] * RTE_LPM_TBL8_GROUP_NUM_ENTRIES; + tbl[3] = *(const uint16_t *)&lpm->tbl8[i8.u32[3]]; + } + + hop[0] = (tbl[0] & RTE_LPM_LOOKUP_SUCCESS) ? (uint8_t)tbl[0] : defv; + hop[1] = (tbl[1] & RTE_LPM_LOOKUP_SUCCESS) ? (uint8_t)tbl[1] : defv; + hop[2] = (tbl[2] & RTE_LPM_LOOKUP_SUCCESS) ? (uint8_t)tbl[2] : defv; + hop[3] = (tbl[3] & RTE_LPM_LOOKUP_SUCCESS) ? (uint8_t)tbl[3] : defv; +} + +#endif /* _RTE_LPM_NEON_H_ */