From patchwork Fri Oct 23 23:28:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Owen Hilyard X-Patchwork-Id: 82027 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 386DDA04DD; Sat, 24 Oct 2020 01:29:23 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B330A685C; Sat, 24 Oct 2020 01:29:21 +0200 (CEST) Received: from mail-qk1-f194.google.com (mail-qk1-f194.google.com [209.85.222.194]) by dpdk.org (Postfix) with ESMTP id 1C841B62 for ; Sat, 24 Oct 2020 01:29:21 +0200 (CEST) Received: by mail-qk1-f194.google.com with SMTP id s14so3087797qkg.11 for ; Fri, 23 Oct 2020 16:29:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jGOyub4+liwUc1MR0l+4Y/FREf3/hGgyDpj6auij8qk=; b=U9Jyz8Y41C1jk7F0gTXQEA638rR3CE9YYzDEE9Y97c9lNie7vEwPlpo3Ncv5RXWvnG 3TRqZLg1XtVoQzhnsuODwEZP1+1FzTZzuu0p4/QF/Ki7GegudpwUnuIEROqVoQBqzDDy o1ipTXSAmgLA/s4wLICD41sqgWKdqP2lThUdw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jGOyub4+liwUc1MR0l+4Y/FREf3/hGgyDpj6auij8qk=; b=c08AXZGJjAgXNbwtFa7F7j/yOVWZVp4EngPTNSFnchMzfKivwWR9o0DF0pPNCHUZeF PFW/ePc0edtRNRRlI/mtY3t1ft7c4p+6Q8x11xtCh00MZBtgu9vifwFW+UVup/5fZjYJ q2C+DY1vi3lyW4Y3gLnGbzOUu+d/5SxzbcH51pTdaThgMfkVFXjwChCqbqplBhawZnrS doZa2lnLpQqVHL8o3dp3HtPh+c54CEgXphQPOcBA9ZwWd/uLr7YMe6a0JiWqtgCUzgy4 mVdH26LsXez+ucbohiQxIp1dyG46G1YgbotAzwtrPE9SeLgQXC3gafkaKDqeQpeTer5h xRjQ== X-Gm-Message-State: AOAM532hZNMIaQ2yzSFC7RFeT5cXUrFyenFuGRJi0JSJRtYOAL95907U JBWgQQ92ylSvjRXIdoHbRCigcA== X-Google-Smtp-Source: ABdhPJxnYao5y9swhG/EE+rU/YvbsdTst+Dptfj9oHcyn8C7mA9EegpJpTaIitlOafDLwXXpSAI+vQ== X-Received: by 2002:a37:6285:: with SMTP id w127mr4767241qkb.454.1603495759452; Fri, 23 Oct 2020 16:29:19 -0700 (PDT) Received: from ohilyard-Alienware-m17.iol.unh.edu (nt-238-66.w4.unh.edu. [132.177.238.66]) by smtp.googlemail.com with ESMTPSA id j16sm1851673qkg.26.2020.10.23.16.29.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 16:29:19 -0700 (PDT) From: Owen Hilyard To: ohilyard@iol.unh.edu, dts@dpdk.org, lijuan.tu@intel.com Cc: shall@iol.unh.edu, lihongx.ma@intel.com, lylavoie@iol.unh.edu, zhaoyan.chen@intel.com, yuan.peng@intel.com Date: Fri, 23 Oct 2020 19:28:58 -0400 Message-Id: <20201023232903.37387-3-ohilyard@iol.unh.edu> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201023232903.37387-2-ohilyard@iol.unh.edu> References: <20201023232903.37387-1-ohilyard@iol.unh.edu> <20201023232903.37387-2-ohilyard@iol.unh.edu> MIME-Version: 1.0 Subject: [dts] [PATCH v2 3/7] rte flow: Add base flow type X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" This type acts as a generic base for all flows. Signed-off-by: Owen Hilyard --- framework/flow/flow.py | 186 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 186 insertions(+) create mode 100644 framework/flow/flow.py diff --git a/framework/flow/flow.py b/framework/flow/flow.py new file mode 100644 index 0000000..2cce079 --- /dev/null +++ b/framework/flow/flow.py @@ -0,0 +1,186 @@ +# BSD LICENSE +# +# Copyright(c) 2020 Intel Corporation. All rights reserved. +# Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +from __future__ import annotations + +import copy +import itertools +import operator +from functools import reduce +from typing import List, FrozenSet, Union, Iterable, Tuple + +from scapy.layers.l2 import Ether + +from flow.enums import FlowItemType, FlowActionType +from flow.exceptions import InvalidFlowItemException +from flow.flow_action_items import ActionFlowItem +from flow.flow_items import FlowItem +from flow.flow_pattern_items import PatternFlowItem, TUNNELING_PROTOCOLS + +# Get reserved mac addresses +NEVER_MATCH_PACKET = Ether(src="", dst="") / ('\x00' * 64) + + +def _iterable_deep_compare(i1, i2): + return reduce( + lambda x, y: x and y, + map(lambda x, y: x == y, i1, i2), + True + ) + + +def expand_pattern_list_with_iterable_replacing_item(patterns: List[Iterable[FlowItem]], + it: Iterable[Tuple[FlowItem, FrozenSet[str], FrozenSet[str], str]], + item): + """ + This function takes a list of patterns and splits each of them into 2 + parts, excluding the item at index. It then uses the provided + iterator to fill in that value for all patterns. + + Ex: + if patterns is [['a', 'b', 'c'], ['c','b','a']], it is [1,2], and item is 'b', + then this function will produce + + [['a', 1], ['a', 2], ['a', 1], ['a', 2], ['a', 1], ['a', 2]] + + if everything is converted into a list. It is not converted + because that requires using the memory to store all of this at + the same time, which could be fairly large. + """ + + split_patterns = list(map(lambda pattern: (pattern[:pattern.index(item)], pattern[pattern.index(item) + 1:],), + filter(lambda pattern: item in pattern, patterns))) + # Tee the iterators so I can consume all of them + + iterators = itertools.tee(it, len(patterns)) + for pattern_before, pattern_after in split_patterns: + for iterator in iterators: + for dataset in iterator: + backup_dataset = copy.deepcopy(dataset) + yield ( + [*pattern_before, backup_dataset[0], *pattern_after], + *backup_dataset[1:], + ) + # yield from map( + # lambda flow_item_test_properties: ( + # [*pattern_before, flow_item_test_properties[0], *pattern_after], + # *flow_item_test_properties[1:], + # ), iterator + # ) + + yield from filter(lambda pattern: item not in pattern, patterns) + + +class Flow(object): + action_items: List[ActionFlowItem] + pattern_items: List[PatternFlowItem] + entry_points: FrozenSet[FlowItemType] + + def __init__(self, action_items=None, pattern_items=None, ): + if action_items is None: + action_items = [] + + if pattern_items is None: + pattern_items = [] + + self.action_items = action_items + self.pattern_items = pattern_items + + def __truediv__(self, item: Union[FlowItem, Flow]): + """ + Used in a similar way to scapy's packet composition. Returns a new flow with the mutated state. + @param item: The other flow item. + @return: A Flow containing both items + """ + if isinstance(item, Flow): + return Flow(pattern_items=[*self.pattern_items, *item.pattern_items], + action_items=[*self.action_items, *item.action_items]) + elif isinstance(item, PatternFlowItem): + if len(self.pattern_items) == 0: + return Flow(pattern_items=[*self.pattern_items, item], action_items=[*self.action_items]) + elif item.type in self.pattern_items[-1].valid_next_items: + return Flow(pattern_items=[*self.pattern_items, item], action_items=[*self.action_items]) + else: + raise InvalidFlowItemException(self.pattern_items[-1], item, flow=self) + elif isinstance(item, ActionFlowItem): + if len(self.action_items) == 0: + return Flow(pattern_items=[*self.pattern_items], action_items=[*self.action_items, item]) + + for action in self.action_items: + if item.type not in action.allowed_with: + raise InvalidFlowItemException(action, item, flow=self) + return Flow(pattern_items=[*self.pattern_items], action_items=[*self.action_items, item]) + + def __str__(self): + return f"ingress pattern %s actions queue index 1 / end" % ( + " / ".join(str(item) for item in self.pattern_items) + " / end") + + def __repr__(self): + return str(self) + + def __eq__(self, other): + return isinstance(other, Flow) and \ + len(self.action_items) == len(other.action_items) and \ + len(self.pattern_items) == len(other.pattern_items) and \ + _iterable_deep_compare(self.pattern_items, other.pattern_items) and \ + _iterable_deep_compare(self.action_items, other.action_items) + + def to_scapy_packet(self): + return reduce(operator.truediv, map(lambda x: x.to_scapy_packet(), self.pattern_items)) + + def get_test_property_flows(self, pattern_item_types_to_update=None, action_item_types_to_update=None) -> \ + Iterable[Flow]: + if pattern_item_types_to_update is None and action_item_types_to_update is None: + pattern_item_types_to_update = [self.pattern_items[-1]] + elif pattern_item_types_to_update is None: + pattern_item_types_to_update = [] + elif action_item_types_to_update is None: + action_item_types_to_update = [] + + # So that if this object is mutated before the generator is finished, it won't change anything + base_pattern_items = copy.deepcopy(self.pattern_items) + base_action_items = copy.deepcopy(self.action_items) + + test_flows: Iterable[Iterable[FlowItem]] = [base_pattern_items] + + tunnelling_protocols = list(filter(lambda i: type(i) in TUNNELING_PROTOCOLS, base_pattern_items)) + if len(tunnelling_protocols) > 0: + test_flows = expand_pattern_list_with_iterable_replacing_item([*test_flows], + tunnelling_protocols[0].get_property_stream(), + tunnelling_protocols[0]) + else: + test_flows = expand_pattern_list_with_iterable_replacing_item([*test_flows], + self.pattern_items[ + -1].get_property_stream(), + self.pattern_items[-1]) + for pattern in test_flows: + yield Flow(pattern_items=pattern[0], action_items=base_action_items), *pattern[1:]