From patchwork Mon Mar 19 12:23:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kumar, Ravi1" X-Patchwork-Id: 36258 X-Patchwork-Delegate: pablo.de.lara.guarch@intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CF85A1B012; Mon, 19 Mar 2018 13:24:46 +0100 (CET) Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0087.outbound.protection.outlook.com [104.47.41.87]) by dpdk.org (Postfix) with ESMTP id A3C517D19 for ; Mon, 19 Mar 2018 13:24:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=d6fiu4vLFqD0FcgWL5jLVeBhErPyDZ0EJBrS6SnQniA=; b=RW20necaj/3cX66Kxc5n4YJXFTBQYow9Tya0TC7FlycrcZ3odRt6ekz8LR1qD0x/Oidq8LNUNUtDAHoifOMt3v4Rmu8e7HaIOZBbogIFp3KcjzgbKj6O7Y9+Gdx7jxylw8DSi36KOnntd5Z6iEJBFtI6J4qAtfNNoeD4j1vDb34= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Ravi1.Kumar@amd.com; Received: from wallaby-smavila.amd.com (202.56.249.162) by BN6PR12MB1505.namprd12.prod.outlook.com (10.172.24.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.588.14; Mon, 19 Mar 2018 12:24:39 +0000 From: Ravi Kumar To: dev@dpdk.org Cc: pablo.de.lara.guarch@intel.com, hemant.agrawal@nxp.com Date: Mon, 19 Mar 2018 08:23:47 -0400 Message-Id: <1521462233-13590-13-git-send-email-Ravi1.kumar@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1521462233-13590-1-git-send-email-Ravi1.kumar@amd.com> References: <1520584520-130522-1-git-send-email-Ravi1.kumar@amd.com> <1521462233-13590-1-git-send-email-Ravi1.kumar@amd.com> MIME-Version: 1.0 X-Originating-IP: [202.56.249.162] X-ClientProxiedBy: MAXPR0101CA0025.INDPRD01.PROD.OUTLOOK.COM (10.174.62.139) To BN6PR12MB1505.namprd12.prod.outlook.com (10.172.24.146) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 008e503f-faed-470c-0620-08d58d9463cb X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:BN6PR12MB1505; X-Microsoft-Exchange-Diagnostics: 1; BN6PR12MB1505; 3:I8mqywV0lyzMDHzHPaBUiwbRAQNjP8Z05xU9NSrmDQKpAj02aItQysPncJCcRlGuOkJN1rZ0GRf481Lgwfyi9+tSMsySdibq0ixp16MFgZikIExBDr2Dxv2vTnWtqthB4dhyesxGvJXRoIpjId+0rsAqUwVSDMJJu6IlpoGl1609pNooelNFpk93AvltLSY6na0rDx3zVxBIXUP4iuBpAiTnKQt50HdOHLvLWNzeQGxXgM1vRTwuv6bmEeWVKrez; 25:LiB4wN98Bh06FlsXCbiJsDOxsgKGrVeRHFxUZXpifH72gPnFbvilF1VE7MfZgZXHwOcfdT8qTpYdMgAxXfKHE+wLi83/RIUoEOiLuclcGsUDH3v7TdwUjCw/2QULPzeCJXXBws7BKCUU+WB2wsEMD7IG7Ktce+fGEljQpx4LHvE9muE6pBMfGw2yHDGL/b4n9P4YpZqoevEMwRN+0kGCSVuOGEK9c0HSuMRC0aOPRXFkkcqpQ+JRdgJK2R7Zq6IC6XvRh1chNQhgLiS57DeW8TkHmy2ewzntX4XATTR2Tn5GW72mwLaFG/GE50+HVSnXDu6V97UgpSyu4BoNRM12Yw==; 31:wQZnQ81r8q8GHIN0Pw0Hye5YC4UhaSXMBJz2wdsNWGGqdgc6iUQAI/IhxK4Y3YVtezjxQshjMRaNKc5cYN3x4ToN68C0Gv48V6LytbEcs1gLqE0Ciy7iaVszjAXQ90gtRhLFLgRz8MwI7CFNSmK6J3arkC4PjL6ZM7caAq7FS5/VZC5zxhdGSQSTgFkvy3fTJer+GsveEgkocNl6LmCGVKXQSt2zFymDkTN8TbJciaI= X-MS-TrafficTypeDiagnostic: BN6PR12MB1505: X-Microsoft-Exchange-Diagnostics: 1; BN6PR12MB1505; 20:nt/zVqMiYgc2kG5iVU4AzmI4oq+vdWqaO9SCajhBXqROVHQIgogUaf1+wzCWFHMmdP4I0nsEilPqweNnp5G7xsBK4qf0VUUKQ8goSA8S5D0RpjDV/QBrV08J3mYcxeqztnGwPKDbcW9h42bcYdpKD0p7nyMEUR4u1FHso8i7AW6fLax1Wyoxa+VEbk2E/t7Z/fLZ4WzTw1ul4ExmzIJOh1XagUF2Am1e5126Kk6uScxn/8jl5KuUcQGQ1/lVJIVI02yv+lcDvvglEP1VXROQ8HaUwSSPGsBdAO6rY1NhNmOHOfT2BvDHxoIBYvr9TG17stCMEyFzuon9fbv7ur6z+BSFkRQAbcCdmIpHuw6FPoahwCwojK6BV7ENStzHjQGUsPpaDXdXRE5a5sZDnaTezT2LQH9bjB+CJPIFB0U6bHLPtSuxIkHf5naPWyL/9cboUzPTpUx5XK/JvWRIIvfnr9eL4i5NDHklb0StmmeqrFUOnZ89pgXHzjCisGBZfTiR; 4:La4XXf7xMMoqynodR5pCkxVP/hJZzYnxpnPb+b0caldACO6xwPm3kXOZe7l1kEN8bDrx0+8L+HVb4RLEjwziGeNpg8WpkxCZRkAnDrsVcXaJvUlsXZl2iAst2nA8EdfQJRomhixw3sHVdROsOxaV4UqwoCjc5a6WWRJUvfSlkbLNGSaLCyREy2ORRH0k40seK42m8dpZfI+3Jajvd1YE1T/OqQReR9vpsrKuczsv5mnr37MdYsIvsZYMEgXZqBWH4ZDLjPQrg1OWVGnh2P118zxtF5LMiwE2L2zZp+xwQ4Myr/sZYmjgmDhYdpNpBaGm X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3231221)(944501300)(52105095)(3002001)(93006095)(93001095)(10201501046)(6055026)(6041310)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(6072148)(201708071742011); SRVR:BN6PR12MB1505; BCL:0; PCL:0; RULEID:; SRVR:BN6PR12MB1505; X-Forefront-PRVS: 06167FAD59 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(396003)(39380400002)(39860400002)(346002)(376002)(366004)(199004)(189003)(97736004)(105586002)(2361001)(2351001)(72206003)(48376002)(4326008)(50226002)(76176011)(59450400001)(8676002)(26005)(316002)(8936002)(53936002)(106356001)(16586007)(186003)(7696005)(51416003)(16526019)(52116002)(386003)(81166006)(81156014)(8656006)(25786009)(575784001)(66066001)(68736007)(86362001)(47776003)(6486002)(5660300001)(6666003)(7736002)(53416004)(6116002)(305945005)(3846002)(478600001)(2950100002)(36756003)(6916009)(2906002)(50466002); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR12MB1505; H:wallaby-smavila.amd.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN6PR12MB1505; 23:S+mkDwHzYKpt2/bSOT9Ct7fouH/3p9lF00wXlvH+I?= ZsXipKtncuv4tJK2K4dnK4EBrrh4jMCdVlvBNkNx2RsS8rQgxPFC6bc3rdbOXKT9FNuj4ck5okj7xzSwQcXD42xfkX35TUMzxMBgMieXN9jt8UNW50U2aXXz22REFdvkpNUuWAzVxAFXxiVCd9lwKJLLk6FqWNs8qdq9uKPNGJ9viK+1Y9ZJhrejLdvBx4d0EoRsWPRTTHD+CxTYeTZGbClDlW4h8xrHgPyXn7YDoFs4n8iKvX0jrx4ftzPsren3AJ7Ex3zJSxCS5u0yFhW8XYOCfV9EGmid4qsq94UPBXfxAK0HXaAPERaNKfRTPSYtY6IK+6NaRKAXM9d+0OrbGNGgW3mZauKxniyTzOm1s2vyuM7vyOGKhko+aiEFE50TFh8U2a231t7yYt3nMuMARTMbTJ3eISkHPk858/Fdgt9J2V/YoTpuUTwzM/GzwTd3bj8GxIoLROuopkPY5P/c1uRYQhlWvdWCFajZo5MEEWFebP4E/yjJb9M46KCNrt4j88M5lFDETYoRPAjaSbudEg0xtANkyRqScALPOrfDRG37ld+p3HHkYhI+M0u2p2+yF6isGp9AtBESNXtlhTN6ZSpgsFfMQ9QKVAfxUFx438I2KAC5GyzN7YJPglc7LCk9Bjs3QARsB3U8wh2I73SnMqAX71rVPTbbwHfAqvSCBfjpSf+1J4z7kk7RjKkFLw7M2Ynh5S/azq/P5RLeU+swV0/a/1W0fyU4F2zfaThYZ0D7ZMho9H16sB/VF0ctaNWyRSLErqOp05ALRYygE9d/vt3ZuF8APOtSTZkdJnmPc0QfT0Rr6IEywZGX+y2X89ZceofA9KMnOeWL37ytwK+dI8RlLtj9xOUD/tlxV3otR5Q7hCZQ1cj/0vShvPrNNhNjyxuUzESP9hFjpou2lnDN5xnspMGhSWaE/HZQU9eymhavH/lgRK4OMlZyGB1BgiBHvHxiGtMR3UHD51v27BhxCsh4CW0rLtKsZHirxtdvMMFg3MiPLVLc5HUkMEEnXAcWDhZdNJCF8JJvW+KYjhkknJ5SiqEfA0hb6mBWkgvjtmaYwSW4PHI2hQPrT//3nfnThVgclh646qOf+fdrqhg4/wVD5P06FZsExs6fmPJC0WB+9dWb5vhD1cYCWt3YfAX6xB19yvkk16ukFsUh8A7wR79YOpjpV9hvN2dDV+rKjImEA== X-Microsoft-Antispam-Message-Info: 3nmmLvVq6OR4r5IrTsTUAKWXNyGuMsa4H15R14jbyNUmqh7MhL2bQDQYbxfskvK45hLM7mKUCQFA8lWED2PfNNQx88OuYiQWFIsPANz5KqU/C3jt7kV99hHoNUAtBUGEJ1GIlXhvRFcUKBNHGfpyU64SHrEgKUOE4RkmqpjYAz4lmkVXvQbvdgGEwC3SsMVU X-Microsoft-Exchange-Diagnostics: 1; BN6PR12MB1505; 6:oI/m5KK/zEg1VVanLIf0hxi6YRB4iStb1aT6Dai8uiBicpEC9m5Sw0X9lUfs2u5SsSQlyr+h9OyzzU3Xvd88IrMdpo4tpb8qr6Lk/9kFcG6VgdqWwoloqugu4EF/ZLFGW27MKX9cxTUuO0gOXHvT+PpO3mLkM5NF9k5khQ3sr4VwsBOAEgAmluGrP5BLfCVUO7IQLHS7ACSSy88e1wEX512FtMhkJfvPQQeeNFKSlU2MS9ERffUW5Z95m1nKxKan6c/RjMXdlibLYAxxhoI2x/f2NqS512VVMMjBpykbHU3l9ZUzKPvOcvPmL9yvPOHsga2bdV3DQX5fdF/I8qIQ5Ez4A42hxJOE0q0i96nc5/o=; 5:eaMfFBi2OjjxmglY6w9qrQ6frddkgZ1VpsAed8YjgMCuUqJLoRz86X9l6Kc2qt3raVdkD+BivzkM2kor68X/hmVMUxaqZzJTNiow58hAzj3e6A/y/jIsRHgQAuRBpAJSx7ai2co/6Mu8Wdbn/Snqmc76nhHe/gL6Bcyzf/Sdo0w=; 24:tjMrfzrKM+V0skMHAQxFInic80qamYGwUXCZk6cVOBYTOUJAU2tYQ7mSBBnP8gFkfusMHFzrQzgjzPwpggOmuK3mQxy/isKo8KBrIPkYros=; 7:g1SIbdNuFixcbBihnmbUOtsfi43LU1lRp4YuVoahJ3IWx/xbNsr3krRCXBPS9JEu0qFgjthDniI+06TrAoI9RkyIThuk3ZVvCymABWomktoFE5o6glpwtjPQ/CqfW6ErNdyCexEQC8VGuA/TusnEIqJ+KeND3hyjn2VVRTw9217ek4j8oWf1UwF+ncZoZierS/bopxQUgPyOyHre/Xor4kyjX8vaqImsUtKMh68PbxUD1204sWUrJ4hZAPIF6RT4 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN6PR12MB1505; 20:6ZtqPOYCRQ+AxsJ1YaeCj941BUsoJh5AQfeJyiWD2d0VQ+znQfJyh4TkqGWpH4LInVfacIk9WP/khw3PpM21usQ3CWTPUz18fKdKhPbNijd36gFHKYPV56/iinoQ58PMFDJkssdMSiwjCYpJYfYEIknxXfi5POL9ScAEgTxvNMaqNGOLcZkVu4rzLd319M99oMGOnLUUBhOPh+go7+DsvVeQaquIh5Y2Ol0j/p08AiHT5rTdCRtUhu0OS+bhjlbs X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2018 12:24:39.8878 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 008e503f-faed-470c-0620-08d58d9463cb X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1505 Subject: [dpdk-dev] [PATCH v5 13/19] crypto/ccp: support aes-gcm aead algo X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Ravi Kumar --- drivers/crypto/ccp/ccp_crypto.c | 235 ++++++++++++++++++++++++++++++++++++++- drivers/crypto/ccp/ccp_pmd_ops.c | 30 +++++ 2 files changed, 261 insertions(+), 4 deletions(-) diff --git a/drivers/crypto/ccp/ccp_crypto.c b/drivers/crypto/ccp/ccp_crypto.c index 6d8a0d6..3a14b77 100644 --- a/drivers/crypto/ccp/ccp_crypto.c +++ b/drivers/crypto/ccp/ccp_crypto.c @@ -273,6 +273,7 @@ ccp_configure_session_aead(struct ccp_session *sess, const struct rte_crypto_sym_xform *xform) { const struct rte_crypto_aead_xform *aead_xform = NULL; + size_t i; aead_xform = &xform->aead; @@ -287,6 +288,7 @@ ccp_configure_session_aead(struct ccp_session *sess, sess->cipher.dir = CCP_CIPHER_DIR_DECRYPT; sess->auth.op = CCP_AUTH_OP_VERIFY; } + sess->aead_algo = aead_xform->algo; sess->auth.aad_length = aead_xform->aad_length; sess->auth.digest_length = aead_xform->digest_length; @@ -295,10 +297,37 @@ ccp_configure_session_aead(struct ccp_session *sess, sess->iv.length = aead_xform->iv.length; switch (aead_xform->algo) { + case RTE_CRYPTO_AEAD_AES_GCM: + sess->cipher.algo = CCP_CIPHER_ALGO_AES_GCM; + sess->cipher.um.aes_mode = CCP_AES_MODE_GCTR; + sess->cipher.engine = CCP_ENGINE_AES; + if (sess->cipher.key_length == 16) + sess->cipher.ut.aes_type = CCP_AES_TYPE_128; + else if (sess->cipher.key_length == 24) + sess->cipher.ut.aes_type = CCP_AES_TYPE_192; + else if (sess->cipher.key_length == 32) + sess->cipher.ut.aes_type = CCP_AES_TYPE_256; + else { + CCP_LOG_ERR("Invalid aead key length"); + return -1; + } + for (i = 0; i < sess->cipher.key_length; i++) + sess->cipher.key_ccp[sess->cipher.key_length - i - 1] = + sess->cipher.key[i]; + sess->auth.algo = CCP_AUTH_ALGO_AES_GCM; + sess->auth.engine = CCP_ENGINE_AES; + sess->auth.um.aes_mode = CCP_AES_MODE_GHASH; + sess->auth.ctx_len = CCP_SB_BYTES; + sess->auth.offset = 0; + sess->auth.block_size = AES_BLOCK_SIZE; + sess->cmd_id = CCP_CMD_COMBINED; + break; default: CCP_LOG_ERR("Unsupported aead algo"); return -ENOTSUP; } + sess->cipher.nonce_phys = rte_mem_virt2phy(sess->cipher.nonce); + sess->cipher.key_phys = rte_mem_virt2phy(sess->cipher.key_ccp); return 0; } @@ -421,10 +450,27 @@ ccp_aead_slot(struct ccp_session *session) int count = 0; switch (session->aead_algo) { + case RTE_CRYPTO_AEAD_AES_GCM: + break; default: CCP_LOG_ERR("Unsupported aead algo %d", session->aead_algo); } + switch (session->auth.algo) { + case CCP_AUTH_ALGO_AES_GCM: + count = 5; + /** + * 1. Passthru iv + * 2. Hash AAD + * 3. GCTR + * 4. Reload passthru + * 5. Hash Final tag + */ + break; + default: + CCP_LOG_ERR("Unsupported combined auth ALGO %d", + session->auth.algo); + } return count; } @@ -847,6 +893,179 @@ ccp_perform_3des(struct rte_crypto_op *op, return 0; } +static int +ccp_perform_aes_gcm(struct rte_crypto_op *op, struct ccp_queue *cmd_q) +{ + struct ccp_session *session; + union ccp_function function; + uint8_t *iv; + struct ccp_passthru pst; + struct ccp_desc *desc; + uint32_t tail; + uint64_t *temp; + phys_addr_t src_addr, dest_addr, key_addr, aad_addr; + phys_addr_t digest_dest_addr; + int length, non_align_len; + + session = (struct ccp_session *)get_session_private_data( + op->sym->session, + ccp_cryptodev_driver_id); + iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset); + key_addr = session->cipher.key_phys; + + src_addr = rte_pktmbuf_mtophys_offset(op->sym->m_src, + op->sym->aead.data.offset); + if (unlikely(op->sym->m_dst != NULL)) + dest_addr = rte_pktmbuf_mtophys_offset(op->sym->m_dst, + op->sym->aead.data.offset); + else + dest_addr = src_addr; + rte_pktmbuf_append(op->sym->m_src, session->auth.ctx_len); + digest_dest_addr = op->sym->aead.digest.phys_addr; + temp = (uint64_t *)(op->sym->aead.digest.data + AES_BLOCK_SIZE); + *temp++ = rte_bswap64(session->auth.aad_length << 3); + *temp = rte_bswap64(op->sym->aead.data.length << 3); + + non_align_len = op->sym->aead.data.length % AES_BLOCK_SIZE; + length = CCP_ALIGN(op->sym->aead.data.length, AES_BLOCK_SIZE); + + aad_addr = op->sym->aead.aad.phys_addr; + + /* CMD1 IV Passthru */ + rte_memcpy(session->cipher.nonce + AES_BLOCK_SIZE, iv, + session->iv.length); + pst.src_addr = session->cipher.nonce_phys; + pst.dest_addr = (phys_addr_t)(cmd_q->sb_iv * CCP_SB_BYTES); + pst.len = CCP_SB_BYTES; + pst.dir = 1; + pst.bit_mod = CCP_PASSTHRU_BITWISE_NOOP; + pst.byte_swap = CCP_PASSTHRU_BYTESWAP_NOOP; + ccp_perform_passthru(&pst, cmd_q); + + /* CMD2 GHASH-AAD */ + function.raw = 0; + CCP_AES_ENCRYPT(&function) = CCP_AES_MODE_GHASH_AAD; + CCP_AES_MODE(&function) = CCP_AES_MODE_GHASH; + CCP_AES_TYPE(&function) = session->cipher.ut.aes_type; + + desc = &cmd_q->qbase_desc[cmd_q->qidx]; + memset(desc, 0, Q_DESC_SIZE); + + CCP_CMD_ENGINE(desc) = CCP_ENGINE_AES; + CCP_CMD_INIT(desc) = 1; + CCP_CMD_FUNCTION(desc) = function.raw; + + CCP_CMD_LEN(desc) = session->auth.aad_length; + + CCP_CMD_SRC_LO(desc) = ((uint32_t)aad_addr); + CCP_CMD_SRC_HI(desc) = high32_value(aad_addr); + CCP_CMD_SRC_MEM(desc) = CCP_MEMTYPE_SYSTEM; + + CCP_CMD_KEY_LO(desc) = ((uint32_t)key_addr); + CCP_CMD_KEY_HI(desc) = high32_value(key_addr); + CCP_CMD_KEY_MEM(desc) = CCP_MEMTYPE_SYSTEM; + + CCP_CMD_LSB_ID(desc) = cmd_q->sb_iv; + + cmd_q->qidx = (cmd_q->qidx + 1) % COMMANDS_PER_QUEUE; + rte_wmb(); + + tail = (uint32_t)(cmd_q->qbase_phys_addr + cmd_q->qidx * Q_DESC_SIZE); + CCP_WRITE_REG(cmd_q->reg_base, CMD_Q_TAIL_LO_BASE, tail); + CCP_WRITE_REG(cmd_q->reg_base, CMD_Q_CONTROL_BASE, + cmd_q->qcontrol | CMD_Q_RUN); + + /* CMD3 : GCTR Plain text */ + function.raw = 0; + CCP_AES_ENCRYPT(&function) = session->cipher.dir; + CCP_AES_MODE(&function) = CCP_AES_MODE_GCTR; + CCP_AES_TYPE(&function) = session->cipher.ut.aes_type; + if (non_align_len == 0) + CCP_AES_SIZE(&function) = (AES_BLOCK_SIZE << 3) - 1; + else + CCP_AES_SIZE(&function) = (non_align_len << 3) - 1; + + + desc = &cmd_q->qbase_desc[cmd_q->qidx]; + memset(desc, 0, Q_DESC_SIZE); + + CCP_CMD_ENGINE(desc) = CCP_ENGINE_AES; + CCP_CMD_EOM(desc) = 1; + CCP_CMD_FUNCTION(desc) = function.raw; + + CCP_CMD_LEN(desc) = length; + + CCP_CMD_SRC_LO(desc) = ((uint32_t)src_addr); + CCP_CMD_SRC_HI(desc) = high32_value(src_addr); + CCP_CMD_SRC_MEM(desc) = CCP_MEMTYPE_SYSTEM; + + CCP_CMD_DST_LO(desc) = ((uint32_t)dest_addr); + CCP_CMD_DST_HI(desc) = high32_value(dest_addr); + CCP_CMD_SRC_MEM(desc) = CCP_MEMTYPE_SYSTEM; + + CCP_CMD_KEY_LO(desc) = ((uint32_t)key_addr); + CCP_CMD_KEY_HI(desc) = high32_value(key_addr); + CCP_CMD_KEY_MEM(desc) = CCP_MEMTYPE_SYSTEM; + + CCP_CMD_LSB_ID(desc) = cmd_q->sb_iv; + + cmd_q->qidx = (cmd_q->qidx + 1) % COMMANDS_PER_QUEUE; + rte_wmb(); + + tail = (uint32_t)(cmd_q->qbase_phys_addr + cmd_q->qidx * Q_DESC_SIZE); + CCP_WRITE_REG(cmd_q->reg_base, CMD_Q_TAIL_LO_BASE, tail); + CCP_WRITE_REG(cmd_q->reg_base, CMD_Q_CONTROL_BASE, + cmd_q->qcontrol | CMD_Q_RUN); + + /* CMD4 : PT to copy IV */ + pst.src_addr = session->cipher.nonce_phys; + pst.dest_addr = (phys_addr_t)(cmd_q->sb_iv * CCP_SB_BYTES); + pst.len = AES_BLOCK_SIZE; + pst.dir = 1; + pst.bit_mod = CCP_PASSTHRU_BITWISE_NOOP; + pst.byte_swap = CCP_PASSTHRU_BYTESWAP_NOOP; + ccp_perform_passthru(&pst, cmd_q); + + /* CMD5 : GHASH-Final */ + function.raw = 0; + CCP_AES_ENCRYPT(&function) = CCP_AES_MODE_GHASH_FINAL; + CCP_AES_MODE(&function) = CCP_AES_MODE_GHASH; + CCP_AES_TYPE(&function) = session->cipher.ut.aes_type; + + desc = &cmd_q->qbase_desc[cmd_q->qidx]; + memset(desc, 0, Q_DESC_SIZE); + + CCP_CMD_ENGINE(desc) = CCP_ENGINE_AES; + CCP_CMD_FUNCTION(desc) = function.raw; + /* Last block (AAD_len || PT_len)*/ + CCP_CMD_LEN(desc) = AES_BLOCK_SIZE; + + CCP_CMD_SRC_LO(desc) = ((uint32_t)digest_dest_addr + AES_BLOCK_SIZE); + CCP_CMD_SRC_HI(desc) = high32_value(digest_dest_addr + AES_BLOCK_SIZE); + CCP_CMD_SRC_MEM(desc) = CCP_MEMTYPE_SYSTEM; + + CCP_CMD_DST_LO(desc) = ((uint32_t)digest_dest_addr); + CCP_CMD_DST_HI(desc) = high32_value(digest_dest_addr); + CCP_CMD_SRC_MEM(desc) = CCP_MEMTYPE_SYSTEM; + + CCP_CMD_KEY_LO(desc) = ((uint32_t)key_addr); + CCP_CMD_KEY_HI(desc) = high32_value(key_addr); + CCP_CMD_KEY_MEM(desc) = CCP_MEMTYPE_SYSTEM; + + CCP_CMD_LSB_ID(desc) = cmd_q->sb_iv; + + cmd_q->qidx = (cmd_q->qidx + 1) % COMMANDS_PER_QUEUE; + rte_wmb(); + + tail = (uint32_t)(cmd_q->qbase_phys_addr + cmd_q->qidx * Q_DESC_SIZE); + CCP_WRITE_REG(cmd_q->reg_base, CMD_Q_TAIL_LO_BASE, tail); + CCP_WRITE_REG(cmd_q->reg_base, CMD_Q_CONTROL_BASE, + cmd_q->qcontrol | CMD_Q_RUN); + + op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; + return 0; +} + static inline int ccp_crypto_cipher(struct rte_crypto_op *op, struct ccp_queue *cmd_q, @@ -913,17 +1132,25 @@ ccp_crypto_auth(struct rte_crypto_op *op, static inline int ccp_crypto_aead(struct rte_crypto_op *op, - struct ccp_queue *cmd_q __rte_unused, - struct ccp_batch_info *b_info __rte_unused) + struct ccp_queue *cmd_q, + struct ccp_batch_info *b_info) { int result = 0; struct ccp_session *session; session = (struct ccp_session *)get_session_private_data( - op->sym->session, + op->sym->session, ccp_cryptodev_driver_id); - switch (session->aead_algo) { + switch (session->auth.algo) { + case CCP_AUTH_ALGO_AES_GCM: + if (session->cipher.algo != CCP_CIPHER_ALGO_AES_GCM) { + CCP_LOG_ERR("Incorrect chain order"); + return -1; + } + result = ccp_perform_aes_gcm(op, cmd_q); + b_info->desccnt += 5; + break; default: CCP_LOG_ERR("Unsupported aead algo %d", session->aead_algo); diff --git a/drivers/crypto/ccp/ccp_pmd_ops.c b/drivers/crypto/ccp/ccp_pmd_ops.c index ab526f3..1314c17 100644 --- a/drivers/crypto/ccp/ccp_pmd_ops.c +++ b/drivers/crypto/ccp/ccp_pmd_ops.c @@ -113,6 +113,36 @@ static const struct rte_cryptodev_capabilities ccp_pmd_capabilities[] = { }, } }, } }, + { /* AES GCM */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_AES_GCM, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .digest_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .aad_size = { + .min = 0, + .max = 65535, + .increment = 1 + }, + .iv_size = { + .min = 12, + .max = 16, + .increment = 4 + }, + }, } + }, } + }, RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() };