LKML Archive on
help / color / mirror / Atom feed
From: Megha Dey <>
Cc:, Megha Dey <>
Subject: [PATCH V8 0/5] crypto: AES CBC multibuffer implementation
Date: Tue,  9 Jan 2018 16:09:03 -0800	[thread overview]
Message-ID: <> (raw)

In this patch series, we introduce AES CBC encryption that is parallelized
on x86_64 cpu with XMM registers. The multi-buffer technique encrypt 8
data streams in parallel with SIMD instructions. Decryption is handled as
in the existing AESNI Intel CBC implementation which can already
parallelize decryption even for a single data stream.

Please see the multi-buffer whitepaper for details of the technique:

It is important that any driver uses this algorithm properly for scenarios
where we have many data streams that can fill up the data lanes most of the
time. It shouldn't be used when only a single data stream is expected
mostly. Otherwise, we may incur extra delays when we have frequent gaps in
data lanes, causing us to wait till data come in to fill the data lanes
before initiating encryption.  We may have to wait for flush operations to
commence when no new data come in after some wait time. However, we keep
this extra delay to a minimum by opportunistically flushing the unfinished
jobs if crypto daemon is the only active task running on a cpu.

By using this technique, we saw a throughput increase of up to 5.7x under
optimal conditions when we have fully loaded encryption jobs filling up all
the data lanes.

Change Log:
1. Remove the notify_callback construct
2. Remove remaining irq_disabled check
3. Remove related tcrypt test as it is already merged

1. Add the CRYPTO_ALG_ASYNC flag to the internal algorithm
2. Remove the irq_disabled check

1. Move away from the compat naming scheme and update the names of the inner
   and outer algorithm
2. Move wrapper code around synchronous internal algorithm from simd.c
   to mcryptd.c

1. Use an async implementation of the inner algorithm instead of sync and use
   the latest skcipher interface instead of the older blkcipher interface.
   (we have picked up this work after a while)

1. Make the decrypt path also use ablkcpher walk.

1. Use ablkcipher_walk helpers to walk the scatter gather list
and eliminated needs to modify blkcipher_walk for multibuffer cipher

1. Update cpu feature check to make sure SSE is supported
2. Fix up unloading of aes-cbc-mb module to properly free memory

Megha Dey (5):
  crypto: Multi-buffer encryption infrastructure support
  crypto: AES CBC multi-buffer data structures
  crypto: AES CBC multi-buffer scheduler
  crypto: AES CBC by8 encryption
  crypto: AES CBC multi-buffer glue code

 arch/x86/crypto/Makefile                           |   1 +
 arch/x86/crypto/aes-cbc-mb/Makefile                |  22 +
 arch/x86/crypto/aes-cbc-mb/aes_cbc_enc_x8.S        | 775 +++++++++++++++++++++
 arch/x86/crypto/aes-cbc-mb/aes_cbc_mb.c            | 698 +++++++++++++++++++
 arch/x86/crypto/aes-cbc-mb/aes_cbc_mb_ctx.h        |  97 +++
 arch/x86/crypto/aes-cbc-mb/aes_cbc_mb_mgr.h        | 132 ++++
 arch/x86/crypto/aes-cbc-mb/aes_mb_mgr_init.c       | 146 ++++
 arch/x86/crypto/aes-cbc-mb/mb_mgr_datastruct.S     | 271 +++++++
 arch/x86/crypto/aes-cbc-mb/mb_mgr_inorder_x8_asm.S | 223 ++++++
 arch/x86/crypto/aes-cbc-mb/mb_mgr_ooo_x8_asm.S     | 417 +++++++++++
 arch/x86/crypto/aes-cbc-mb/reg_sizes.S             | 126 ++++
 crypto/Kconfig                                     |  15 +
 crypto/mcryptd.c                                   | 475 +++++++++++++
 include/crypto/mcryptd.h                           |  56 ++
 14 files changed, 3454 insertions(+)
 create mode 100644 arch/x86/crypto/aes-cbc-mb/Makefile
 create mode 100644 arch/x86/crypto/aes-cbc-mb/aes_cbc_enc_x8.S
 create mode 100644 arch/x86/crypto/aes-cbc-mb/aes_cbc_mb.c
 create mode 100644 arch/x86/crypto/aes-cbc-mb/aes_cbc_mb_ctx.h
 create mode 100644 arch/x86/crypto/aes-cbc-mb/aes_cbc_mb_mgr.h
 create mode 100644 arch/x86/crypto/aes-cbc-mb/aes_mb_mgr_init.c
 create mode 100644 arch/x86/crypto/aes-cbc-mb/mb_mgr_datastruct.S
 create mode 100644 arch/x86/crypto/aes-cbc-mb/mb_mgr_inorder_x8_asm.S
 create mode 100644 arch/x86/crypto/aes-cbc-mb/mb_mgr_ooo_x8_asm.S
 create mode 100644 arch/x86/crypto/aes-cbc-mb/reg_sizes.S


             reply	other threads:[~2018-01-09 23:53 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-10  0:09 Megha Dey [this message]
2018-01-10  0:09 ` [PATCH V8 1/5] crypto: Multi-buffer encryption infrastructure support Megha Dey
2018-01-18 11:39   ` Herbert Xu
2018-01-19  0:44     ` Megha Dey
2018-03-16 14:53       ` Herbert Xu
2018-04-17 18:40         ` Dey, Megha
2018-04-18 11:01           ` Herbert Xu
2018-04-19  0:54             ` Dey, Megha
2018-04-19  3:25               ` Herbert Xu
2018-04-25  1:14                 ` Dey, Megha
2018-04-26  9:44                   ` Herbert Xu
2018-05-01 22:39                     ` Dey, Megha
2018-05-07  9:35                       ` Herbert Xu
2018-05-11  1:24                         ` Dey, Megha
2018-05-11  4:45                           ` Herbert Xu
2018-05-12  1:21                             ` Dey, Megha
2018-06-21  1:05                             ` Dey, Megha
2018-01-10  0:09 ` [PATCH V8 2/5] crypto: AES CBC multi-buffer data structures Megha Dey
2018-01-10  0:09 ` [PATCH V8 3/5] crypto: AES CBC multi-buffer scheduler Megha Dey
2018-01-10  0:09 ` [PATCH V8 4/5] crypto: AES CBC by8 encryption Megha Dey
2018-01-10  0:09 ` [PATCH V8 5/5] crypto: AES CBC multi-buffer glue code Megha Dey

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \
    --subject='Re: [PATCH V8 0/5] crypto: AES CBC multibuffer implementation' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).