summaryrefslogtreecommitdiffstats
path: root/crypto/algapi.c
Commit message (Collapse)AuthorAgeFilesLines
* crypto: engine - fix crypto_queue backlog handlingOlivier Bacon2023-05-171-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [ Upstream commit 4140aafcff167b5b9e8dae6a1709a6de7cac6f74 ] CRYPTO_TFM_REQ_MAY_BACKLOG tells the crypto driver that it should internally backlog requests until the crypto hw's queue becomes full. At that point, crypto_engine backlogs the request and returns -EBUSY. Calling driver such as dm-crypt then waits until the complete() function is called with a status of -EINPROGRESS before sending a new request. The problem lies in the call to complete() with a value of -EINPROGRESS that is made when a backlog item is present on the queue. The call is done before the successful execution of the crypto request. In the case that do_one_request() returns < 0 and the retry support is available, the request is put back in the queue. This leads upper drivers to send a new request even if the queue is still full. The problem can be reproduced by doing a large dd into a crypto dm-crypt device. This is pretty easy to see when using Freescale CAAM crypto driver and SWIOTLB dma. Since the actual amount of requests that can be hold in the queue is unlimited we get IOs error and dma allocation. The fix is to call complete with a value of -EINPROGRESS only if the request is not enqueued back in crypto_queue. This is done by calling complete() later in the code. In order to delay the decision, crypto_queue is modified to correctly set the backlog pointer when a request is enqueued back. Fixes: 6a89f492f8e5 ("crypto: engine - support for parallel requests based on retry mechanism") Co-developed-by: Sylvain Ouellet <souellet@genetec.com> Signed-off-by: Sylvain Ouellet <souellet@genetec.com> Signed-off-by: Olivier Bacon <obacon@genetec.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: Sasha Levin <sashal@kernel.org>
* crypto: api - Demote BUG_ON() in crypto_unregister_alg() to a WARN_ON()Toke Høiland-Jørgensen2023-05-111-1/+3
| | | | | | | | | | | | | | | | | | | | | commit a543ada7db729514ddd3ba4efa45f4c7b802ad85 upstream. The crypto_unregister_alg() function expects callers to ensure that any algorithm that is unregistered has a refcnt of exactly 1, and issues a BUG_ON() if this is not the case. However, there are in fact drivers that will call crypto_unregister_alg() without ensuring that the refcnt has been lowered first, most notably on system shutdown. This causes the BUG_ON() to trigger, which prevents a clean shutdown and hangs the system. To avoid such hangs on shutdown, demote the BUG_ON() in crypto_unregister_alg() to a WARN_ON() with early return. Cc stable because this problem was observed on a 6.2 kernel, cf the link below. Link: https://lore.kernel.org/r/87r0tyq8ph.fsf@toke.dk Cc: stable@vger.kernel.org Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* crypto: api - Move cryptomgr soft dependency into algapiHerbert Xu2022-02-111-0/+1
| | | | | | | | | | | | | | | | commit c6ce9c5831cae515d375a01b97ae1778689acf19 upstream. The soft dependency on cryptomgr is only needed in algapi because if algapi isn't present then no algorithms can be loaded. This also fixes the case where api is built-in but algapi is built as a module as the soft dependency would otherwise get lost. Fixes: 8ab23d547f65 ("crypto: api - Add softdep on cryptomgr") Reported-by: Jan Beulich <jbeulich@suse.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Tested-by: Jan Beulich <jbeulich@suse.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* crypto: api - remove CRYPTOA_U32 and related functionsLiu Shixin2021-06-171-18/+0
| | | | | | | | | | | | | According to the advice of Eric and Herbert, type CRYPTOA_U32 has been unused for over a decade, so remove the code related to CRYPTOA_U32. After removing CRYPTOA_U32, the type of the variable attrs can be changed from union to struct. Signed-off-by: Liu Shixin <liushixin2@huawei.com> Reviewed-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - use common mechanism for inheriting flagsEric Biggers2020-07-161-1/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a template is instantiated, the template will have CRYPTO_ALG_ASYNC set if any of the algorithms it uses has CRYPTO_ALG_ASYNC set. We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets "inherited" in the same way. This is difficult because the handling of CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by: - Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that have these inheritance semantics. - Add crypto_algt_inherited_mask(), for use by template ->create() methods. It returns any of these flags that the user asked to be unset and thus must be passed in the 'mask' to crypto_grab_*(). - Also modify crypto_check_attr_type() to handle computing the 'mask' so that most templates can just use this. - Make crypto_grab_*() propagate these flags to the template instance being created so that templates don't have to do this themselves. Make crypto/simd.c propagate these flags too, since it "wraps" another algorithm, similar to a template. Based on a patch by Mikulas Patocka <mpatocka@redhat.com> (https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - create function to add request in front of queueIuliana Prodan2020-05-081-0/+8
| | | | | | | | | | | Add crypto_enqueue_request_head function that enqueues a request in front of queue. This will be used in crypto-engine, on error path. In case a request was not executed by hardware, enqueue it back in front of queue (to keep the order of requests). Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Fix use-after-free and race in crypto_spawn_algHerbert Xu2020-04-161-6/+16
| | | | | | | | | | | | | | | | | | | There are two problems in crypto_spawn_alg. First of all it may return spawn->alg even if spawn->dead is set. This results in a double-free as detected by syzbot. Secondly the setting of the DYING flag is racy because we hold the read-lock instead of the write-lock. We should instead call crypto_shoot_alg in a safe manner by gaining a refcount, dropping the lock, and then releasing the refcount. This patch fixes both problems. Reported-by: syzbot+fc0674cde00b66844470@syzkaller.appspotmail.com Fixes: 4f87ee118d16 ("crypto: api - Do not zap spawn->alg") Fixes: 73669cc55646 ("crypto: api - Fix race condition in...") Cc: <stable@vger.kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - Avoid spurious modprobe on LOADEDEric Biggers2020-04-161-1/+1
| | | | | | | | | | | | | | | | | | | | | Currently after any algorithm is registered and tested, there's an unnecessary request_module("cryptomgr") even if it's already loaded. Also, CRYPTO_MSG_ALG_LOADED is sent twice, and thus if the algorithm is "crct10dif", lib/crc-t10dif.c replaces the tfm twice rather than once. This occurs because CRYPTO_MSG_ALG_LOADED is sent using crypto_probing_notify(), which tries to load "cryptomgr" if the notification is not handled (NOTIFY_DONE). This doesn't make sense because "cryptomgr" doesn't handle this notification. Fix this by using crypto_notify() instead of crypto_probing_notify(). Fixes: dd8b083f9a5e ("crypto: api - Introduce notifier for new crypto algorithms") Cc: <stable@vger.kernel.org> # v4.20+ Cc: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - remove crypto_template::{alloc,free}()Eric Biggers2020-01-091-5/+0
| | | | | | | | | | Now that all templates provide a ->create() method which creates an instance, installs a strongly-typed ->free() method directly to it, and registers it, the older ->alloc() and ->free() methods in 'struct crypto_template' are no longer used. Remove them. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - fold crypto_init_spawn() into crypto_grab_spawn()Eric Biggers2020-01-091-29/+14
| | | | | | | | | | | | Now that crypto_init_spawn() is only called by crypto_grab_spawn(), simplify things by moving its functionality into crypto_grab_spawn(). In the process of doing this, also be more consistent about when the spawn and instance are updated, and remove the crypto_spawn::dropref flag since now it's always set. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - remove obsoleted instance creation helpersEric Biggers2020-01-091-57/+0
| | | | | | | | | | | | | | | | | | | Remove lots of helper functions that were previously used for instantiating crypto templates, but are now unused: - crypto_get_attr_alg() and similar functions looked up an inner algorithm directly from a template parameter. These were replaced with getting the algorithm's name, then calling crypto_grab_*(). - crypto_init_spawn2() and similar functions initialized a spawn, given an algorithm. Similarly, these were replaced with crypto_grab_*(). - crypto_alloc_instance() and similar functions allocated an instance with a single spawn, given the inner algorithm. These aren't useful anymore since crypto_grab_*() need the instance allocated first. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - pass instance to crypto_grab_spawn()Eric Biggers2020-01-091-3/+3
| | | | | | | | | | | | | Currently, crypto_spawn::inst is first used temporarily to pass the instance to crypto_grab_spawn(). Then crypto_init_spawn() overwrites it with crypto_spawn::next, which shares the same union. Finally, crypto_spawn::inst is set again when the instance is registered. Make this less convoluted by just passing the instance as an argument to crypto_grab_spawn() instead. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - make crypto_grab_spawn() handle an ERR_PTR() nameEric Biggers2020-01-091-0/+4
| | | | | | | | | | | | | | To allow further simplifying template ->create() functions, make crypto_grab_spawn() handle an ERR_PTR() name by passing back the error. For most templates, this will allow the result of crypto_attr_alg_name() to be passed directly to crypto_grab_*(), rather than first having to assign it to a variable [where it can then potentially be misused, as it was in the rfc7539 template prior to commit 5e27f38f1f3f ("crypto: chacha20poly1305 - set cra_name correctly")] and check it for error. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - make crypto_drop_spawn() a no-op on uninitialized spawnsEric Biggers2020-01-091-0/+3
| | | | | | | | | | | | | | | | Make crypto_drop_spawn() do nothing when the spawn hasn't been initialized with an algorithm yet. This will allow simplifying error handling in all the template ->create() functions, since on error they will be able to just call their usual "free instance" function, rather than having to handle dropping just the spawns that have been initialized so far. This does assume the spawn starts out zero-filled, but that's always the case since instances are allocated with kzalloc(). And some other code already assumes this anyway. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Retain alg refcount in crypto_grab_spawnHerbert Xu2019-12-271-8/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch changes crypto_grab_spawn to retain the reference count on the algorithm. This is because the caller needs to access the algorithm parameters and without the reference count the algorithm can be freed at any time. The reference count will be subsequently dropped by the crypto API once the instance has been registered. The helper crypto_drop_spawn will also conditionally drop the reference count depending on whether it has been registered. Note that the code is actually added to crypto_init_spawn. However, unless the caller activates this by setting spawn->dropref beforehand then nothing happens. The only caller that sets dropref is currently crypto_grab_spawn. Once all legacy users of crypto_init_spawn disappear, then we can kill the dropref flag. Internally each instance will maintain a list of its spawns prior to registration. This memory used by this list is shared with other fields that are only used after registration. In order for this to work a new flag spawn->registered is added to indicate whether spawn->inst can be used. Fixes: d6ef2f198d4c ("crypto: api - Add crypto_grab_spawn primitive") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - make unregistration functions return voidEric Biggers2019-12-201-17/+8
| | | | | | | | | | | | | | | | | | Some of the algorithm unregistration functions return -ENOENT when asked to unregister a non-registered algorithm, while others always return 0 or always return void. But no users check the return value, except for two of the bulk unregistration functions which print a message on error but still always return 0 to their caller, and crypto_del_alg() which calls crypto_unregister_instance() which always returns 0. Since unregistering a non-registered algorithm is always a kernel bug but there isn't anything callers should do to handle this situation at runtime, let's simplify things by making all the unregistration functions return void, and moving the error message into crypto_unregister_alg() and upgrading it to a WARN(). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - fix unexpectedly getting generic implementationHerbert Xu2019-12-201-3/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y, the first lookup of an algorithm that needs to be instantiated using a template will always get the generic implementation, even when an accelerated one is available. This happens because the extra self-tests for the accelerated implementation allocate the generic implementation for comparison purposes, and then crypto_alg_tested() for the generic implementation "fulfills" the original request (i.e. sets crypto_larval::adult). This patch fixes this by only fulfilling the original request if we are currently the best outstanding larval as judged by the priority. If we're not the best then we will ask all waiters on that larval request to retry the lookup. Note that this patch introduces a behaviour change when the module providing the new algorithm is unregistered during the process. Previously we would have failed with ENOENT, after the patch we will instead redo the lookup. Fixes: 9a8a6b3f0950 ("crypto: testmgr - fuzz hashes against...") Fixes: d435e10e67be ("crypto: testmgr - fuzz skciphers against...") Fixes: 40153b10d91c ("crypto: testmgr - fuzz AEADs against...") Reported-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Add more comments to crypto_remove_spawnsHerbert Xu2019-12-111-0/+25
| | | | | | | This patch explains the logic behind crypto_remove_spawns and its underling crypto_more_spawns. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Do not zap spawn->algHerbert Xu2019-12-111-10/+12
| | | | | | | | | | | | Currently when a spawn is removed we will zap its alg field. This is racy because the spawn could belong to an unregistered instance which may dereference the spawn->alg field. This patch fixes this by keeping spawn->alg constant and instead adding a new spawn->dead field to indicate that a spawn is going away. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Fix race condition in crypto_spawn_algHerbert Xu2019-12-111-11/+5
| | | | | | | | | | | The function crypto_spawn_alg is racy because it drops the lock before shooting the dying algorithm. The algorithm could disappear altogether before we shoot it. This patch fixes it by moving the shooting into the locked section. Fixes: 6bfd48096ff8 ("[CRYPTO] api: Added spawns") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Check spawn->alg under lock in crypto_drop_spawnHerbert Xu2019-12-111-4/+2
| | | | | | | | | | | We need to check whether spawn->alg is NULL under lock as otherwise the algorithm could be removed from under us after we have checked it and found it to be non-NULL. This could cause us to remove the spawn from a non-existent list. Fixes: 7ede5a5ba55a ("crypto: api - Fix crypto_drop_spawn crash...") Cc: <stable@vger.kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: ablkcipher - remove deprecated and unused ablkcipher supportArd Biesheuvel2019-11-171-26/+0
| | | | | | | | | | Now that all users of the deprecated ablkcipher interface have been moved to the skcipher interface, ablkcipher is no longer used and can be removed. Reviewed-by: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* Merge branch 'linus' of ↵Linus Torvalds2019-07-081-31/+4
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto updates from Herbert Xu: "Here is the crypto update for 5.3: API: - Test shash interface directly in testmgr - cra_driver_name is now mandatory Algorithms: - Replace arc4 crypto_cipher with library helper - Implement 5 way interleave for ECB, CBC and CTR on arm64 - Add xxhash - Add continuous self-test on noise source to drbg - Update jitter RNG Drivers: - Add support for SHA204A random number generator - Add support for 7211 in iproc-rng200 - Fix fuzz test failures in inside-secure - Fix fuzz test failures in talitos - Fix fuzz test failures in qat" * 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (143 commits) crypto: stm32/hash - remove interruptible condition for dma crypto: stm32/hash - Fix hmac issue more than 256 bytes crypto: stm32/crc32 - rename driver file crypto: amcc - remove memset after dma_alloc_coherent crypto: ccp - Switch to SPDX license identifiers crypto: ccp - Validate the the error value used to index error messages crypto: doc - Fix formatting of new crypto engine content crypto: doc - Add parameter documentation crypto: arm64/aes-ce - implement 5 way interleave for ECB, CBC and CTR crypto: arm64/aes-ce - add 5 way interleave routines crypto: talitos - drop icv_ool crypto: talitos - fix hash on SEC1. crypto: talitos - move struct talitos_edesc into talitos.h lib/scatterlist: Fix mapping iterator when sg->offset is greater than PAGE_SIZE crypto/NX: Set receive window credits to max number of CRBs in RxFIFO crypto: asymmetric_keys - select CRYPTO_HASH where needed crypto: serpent - mark __serpent_setkey_sbox noinline crypto: testmgr - dynamically allocate crypto_shash crypto: testmgr - dynamically allocate testvec_config crypto: talitos - eliminate unneeded 'done' functions at build time ...
| * crypto: algapi - require cra_name and cra_driver_nameEric Biggers2019-06-131-18/+4
| | | | | | | | | | | | | | | | | | | | | | | | Now that all algorithms explicitly set cra_driver_name, make it required for algorithm registration and remove the code that generated a default cra_driver_name. Also add an explicit check that cra_name is set too, since that's obviously required too, yet it didn't seem to be checked anywhere. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: algapi - remove crypto_tfm_in_queue()Eric Biggers2019-05-301-13/+0
| | | | | | | | | | | | | | Remove the crypto_tfm_in_queue() function, which is unused. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152Thomas Gleixner2019-05-301-6/+1
|/ | | | | | | | | | | | | | | | | | | | | Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation either version 2 of the license or at your option any later version extracted by the scancode license scanner the SPDX license identifier GPL-2.0-or-later has been chosen to replace the boilerplate/reference in 3029 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190527070032.746973796@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* crypto: api - add a helper to (un)register a array of templatesXiongfeng Wang2019-01-251-0/+27
| | | | | | | | | This patch add a helper to (un)register a array of templates. The following patches will use this helper to simplify the code. Signed-off-by: Xiongfeng Wang <xiongfeng.wang@linaro.org> Reviewed-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - reject NULL crypto_spawn::instEric Biggers2019-01-111-0/+3
| | | | | | | | | | | | | | | | It took me a while to notice the bug where the adiantum template left crypto_spawn::inst == NULL, because this only caused problems in certain cases where algorithms are dynamically loaded/unloaded. More improvements are needed, but for now make crypto_init_spawn() reject this case and WARN(), so this type of bug will be noticed immediately in the future. Note: I checked all callers and the adiantum template was the only place that had this wrong. So this WARN shouldn't trigger anymore. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - remove crypto_alloc_instance()Eric Biggers2019-01-111-31/+2
| | | | | | | | | | | Now that all "blkcipher" templates have been converted to "skcipher", crypto_alloc_instance() is no longer used. And it's not useful any longer as it creates an old-style weakly typed instance rather than a new-style strongly typed instance. So remove it, and now that the name is freed up rename crypto_alloc_instance2() to crypto_alloc_instance(). Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: user - Add crypto_stats_initCorentin Labbe2018-12-071-3/+7
| | | | | | | | This patch add the crypto_stats_init() function. This will permit to remove some ifdef from __crypto_register_alg(). Signed-off-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: user - rename err_cnt parameterCorentin Labbe2018-12-071-19/+19
| | | | | | | | Since now all crypto stats are on their own structures, it is now useless to have the algorithm name in the err_cnt member. Signed-off-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: user - Split stats in multiple structuresCorentin Labbe2018-12-071-57/+51
| | | | | | | Like for userspace, this patch splits stats into multiple structures, one for each algorithm class. Signed-off-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: user - fix use_after_free of struct xxx_requestCorentin Labbe2018-12-071-0/+233
| | | | | | | | | | | | | All crypto_stats functions use the struct xxx_request for feeding stats, but in some case this structure could already be freed. For fixing this, the needed parameters (len and alg) will be stored before the request being executed. Fixes: cac5818c25d0 ("crypto: user - Implement a generic crypto statistics") Reported-by: syzbot <syzbot+6939a606a5305e9e9799@syzkaller.appspotmail.com> Signed-off-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: user - convert all stats from u32 to u64Corentin Labbe2018-12-071-5/+5
| | | | | | | | | All the 32-bit fields need to be 64-bit. In some cases, UINT32_MAX crypto operations can be done in seconds. Reported-by: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: user - made crypto_user_stat optionalCorentin Labbe2018-12-071-0/+2
| | | | | | | | | Even if CRYPTO_STATS is set to n, some part of CRYPTO_STATS are compiled. This patch made all part of crypto_user_stat uncompiled in that case. Signed-off-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: user - Implement a generic crypto statisticsCorentin Labbe2018-09-281-0/+8
| | | | | | | | This patch implement a generic way to get statistics about all crypto usages. Signed-off-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Introduce notifier for new crypto algorithmsMartin K. Petersen2018-09-041-0/+2
| | | | | | | | | | | | | | | | Introduce a facility that can be used to receive a notification callback when a new algorithm becomes available. This can be used by existing crypto registrations to trigger a switch from a software-only algorithm to a hardware-accelerated version. A new CRYPTO_MSG_ALG_LOADED state is introduced to the existing crypto notification chain, and the register/unregister functions are exported so they can be called by subsystems outside of crypto. Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Suggested-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Introduce generic max blocksize and alignmaskKees Cook2018-09-041-1/+6
| | | | | | | | | | | | | | | | In the quest to remove all stack VLA usage from the kernel[1], this exposes a new general upper bound on crypto blocksize and alignmask (higher than for the existing cipher limits) for VLA removal, and introduces new checks. At present, the highest cra_alignmask in the kernel is 63. The highest cra_blocksize is 144 (SHA3_224_BLOCK_SIZE, 18 8-byte words). For the new blocksize limit, I went with 160 (20 8-byte words). [1] https://lkml.kernel.org/r/CA+55aFzCG-zNmZwX4A2FQpadafLfEzK6CC=qPXydAacU1RqZWA@mail.gmail.com Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - laying defines and checks for statically allocated buffersSalvatore Mesoraca2018-04-211-0/+10
| | | | | | | | | | | | | | | In preparation for the removal of VLAs[1] from crypto code. We create 2 new compile-time constants: all ciphers implemented in Linux have a block size less than or equal to 16 bytes and the most demanding hw require 16 bytes alignment for the block buffer. We also enforce these limits in crypto_check_alg when a new cipher is registered. [1] http://lkml.kernel.org/r/CA+55aFzCG-zNmZwX4A2FQpadafLfEzK6CC=qPXydAacU1RqZWA@mail.gmail.com Signed-off-by: Salvatore Mesoraca <s.mesoraca16@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Keep failed instances aliveHerbert Xu2018-03-311-8/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch reverts commit 9c521a200bc3 ("crypto: api - remove instance when test failed") and fixes the underlying problem in a different way. To recap, prior to the reverted commit, an instance that fails a self-test is kept around. However, it would satisfy any new lookups against its name and therefore the system may accumlulate an unbounded number of failed instances for the same algorithm name. The reverted commit fixed it by unregistering the instance. Hoever, this still does not prevent the creation of the same failed instance over and over again each time the name is looked up. This patch fixes it by keeping the failed instance around, just as we would if it were a normal algorithm. However, the lookup code has been udpated so that we do not attempt to create another instance as long as this failed one is still registered. Of course, you could still force a new creation by deleting the instance from user-space. A new error (ELIBBAD) has been commandeered for this purpose and will be returned when all registered algorithm of a given name have failed the self-test. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* Merge branch 'linus' of ↵Linus Torvalds2018-01-311-9/+4
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto updates from Herbert Xu: "API: - Enforce the setting of keys for keyed aead/hash/skcipher algorithms. - Add multibuf speed tests in tcrypt. Algorithms: - Improve performance of sha3-generic. - Add native sha512 support on arm64. - Add v8.2 Crypto Extentions version of sha3/sm3 on arm64. - Avoid hmac nesting by requiring underlying algorithm to be unkeyed. - Add cryptd_max_cpu_qlen module parameter to cryptd. Drivers: - Add support for EIP97 engine in inside-secure. - Add inline IPsec support to chelsio. - Add RevB core support to crypto4xx. - Fix AEAD ICV check in crypto4xx. - Add stm32 crypto driver. - Add support for BCM63xx platforms in bcm2835 and remove bcm63xx. - Add Derived Key Protocol (DKP) support in caam. - Add Samsung Exynos True RNG driver. - Add support for Exynos5250+ SoCs in exynos PRNG driver" * 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (166 commits) crypto: picoxcell - Fix error handling in spacc_probe() crypto: arm64/sha512 - fix/improve new v8.2 Crypto Extensions code crypto: arm64/sm3 - new v8.2 Crypto Extensions implementation crypto: arm64/sha3 - new v8.2 Crypto Extensions implementation crypto: testmgr - add new testcases for sha3 crypto: sha3-generic - export init/update/final routines crypto: sha3-generic - simplify code crypto: sha3-generic - rewrite KECCAK transform to help the compiler optimize crypto: sha3-generic - fixes for alignment and big endian operation crypto: aesni - handle zero length dst buffer crypto: artpec6 - remove select on non-existing CRYPTO_SHA384 hwrng: bcm2835 - Remove redundant dev_err call in bcm2835_rng_probe() crypto: stm32 - remove redundant dev_err call in stm32_cryp_probe() crypto: axis - remove unnecessary platform_get_resource() error check crypto: testmgr - test misuse of result in ahash crypto: inside-secure - make function safexcel_try_push_requests static crypto: aes-generic - fix aes-generic regression on powerpc crypto: chelsio - Fix indentation warning crypto: arm64/sha1-ce - get rid of literal pool crypto: arm64/sha2-ce - move the round constant table to .rodata section ...
| * crypto: algapi - remove unused notificationsEric Biggers2018-01-051-5/+0
| | | | | | | | | | | | | | | | | | | | | | | | There is a message posted to the crypto notifier chain when an algorithm is unregistered, and when a template is registered or unregistered. But nothing is listening for those messages; currently there are only listeners for the algorithm request and registration messages. Get rid of these unused notifications for now. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: algapi - convert cra_refcnt to refcount_tEric Biggers2018-01-051-4/+4
| | | | | | | | | | | | | | | | | | | | | | Reference counters should use refcount_t rather than atomic_t, since the refcount_t implementation can prevent overflows, reducing the exploitability of reference leak bugs. crypto_alg.cra_refcount is a reference counter with the usual semantics, so switch it over to refcount_t. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | crypto: algapi - fix NULL dereference in crypto_remove_spawns()Eric Biggers2018-01-051-0/+12
|/ | | | | | | | | | | | | | | | | | | | | | | | | syzkaller triggered a NULL pointer dereference in crypto_remove_spawns() via a program that repeatedly and concurrently requests AEADs "authenc(cmac(des3_ede-asm),pcbc-aes-aesni)" and hashes "cmac(des3_ede)" through AF_ALG, where the hashes are requested as "untested" (CRYPTO_ALG_TESTED is set in ->salg_mask but clear in ->salg_feat; this causes the template to be instantiated for every request). Although AF_ALG users really shouldn't be able to request an "untested" algorithm, the NULL pointer dereference is actually caused by a longstanding race condition where crypto_remove_spawns() can encounter an instance which has had spawn(s) "grabbed" but hasn't yet been registered, resulting in ->cra_users still being NULL. We probably should properly initialize ->cra_users earlier, but that would require updating many templates individually. For now just fix the bug in a simple way that can easily be backported: make crypto_remove_spawns() treat a NULL ->cra_users list as empty. Reported-by: syzbot <syzkaller@googlegroups.com> Cc: stable@vger.kernel.org Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: change transient busy return code to -ENOSPCGilad Ben-Yossef2017-11-031-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | The crypto API was using the -EBUSY return value to indicate both a hard failure to submit a crypto operation into a transformation provider when the latter was busy and the backlog mechanism was not enabled as well as a notification that the operation was queued into the backlog when the backlog mechanism was enabled. Having the same return code indicate two very different conditions depending on a flag is both error prone and requires extra runtime check like the following to discern between the cases: if (err == -EINPROGRESS || (err == -EBUSY && (ahash_request_flags(req) & CRYPTO_TFM_REQ_MAY_BACKLOG))) This patch changes the return code used to indicate a crypto op failed due to the transformation provider being transiently busy to -ENOSPC. Signed-off-by: Gilad Ben-Yossef <gilad@benyossef.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - use separate dst and src operands for __crypto_xor()Ard Biesheuvel2017-08-041-10/+15
| | | | | | | | | | In preparation of introducing crypto_xor_cpy(), which will use separate operands for input and output, modify the __crypto_xor() implementation, which it will share with the existing crypto_xor(), which provides the actual functionality when not using the inline version. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - Use pr_err common logging style.Karim Eshapa2017-06-191-1/+1
| | | | | | | Use more common error logging style. Signed-off-by: Karim Eshapa <karim.eshapa@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - annotate expected branch behavior in crypto_inc()Ard Biesheuvel2017-03-091-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | To prevent unnecessary branching, mark the exit condition of the primary loop as likely(), given that a carry in a 32-bit counter occurs very rarely. On arm64, the resulting code is emitted by GCC as 9a8: cmp w1, #0x3 9ac: add x3, x0, w1, uxtw 9b0: b.ls 9e0 <crypto_inc+0x38> 9b4: ldr w2, [x3,#-4]! 9b8: rev w2, w2 9bc: add w2, w2, #0x1 9c0: rev w4, w2 9c4: str w4, [x3] 9c8: cbz w2, 9d0 <crypto_inc+0x28> 9cc: ret where the two remaining branch conditions (one for size < 4 and one for the carry) are statically predicted as non-taken, resulting in optimal execution in the vast majority of cases. Also, replace the open coded alignment test with IS_ALIGNED(). Cc: Jason A. Donenfeld <Jason@zx2c4.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: algapi - make crypto_xor() and crypto_inc() alignment agnosticArd Biesheuvel2017-02-111-18/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of unconditionally forcing 4 byte alignment for all generic chaining modes that rely on crypto_xor() or crypto_inc() (which may result in unnecessary copying of data when the underlying hardware can perform unaligned accesses efficiently), make those functions deal with unaligned input explicitly, but only if the Kconfig symbol HAVE_EFFICIENT_UNALIGNED_ACCESS is set. This will allow us to drop the alignmasks from the CBC, CMAC, CTR, CTS, PCBC and SEQIV drivers. For crypto_inc(), this simply involves making the 4-byte stride conditional on HAVE_EFFICIENT_UNALIGNED_ACCESS being set, given that it typically operates on 16 byte buffers. For crypto_xor(), an algorithm is implemented that simply runs through the input using the largest strides possible if unaligned accesses are allowed. If they are not, an optimal sequence of memory accesses is emitted that takes the relative alignment of the input buffers into account, e.g., if the relative misalignment of dst and src is 4 bytes, the entire xor operation will be completed using 4 byte loads and stores (modulo unaligned bits at the start and end). Note that all expressions involving misalign are simply eliminated by the compiler when HAVE_EFFICIENT_UNALIGNED_ACCESS is defined. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: api - Clear CRYPTO_ALG_DEAD bit before registering an algSalvatore Benedetto2017-01-231-0/+1
| | | | | | | | | | Make sure CRYPTO_ALG_DEAD bit is cleared before proceeding with the algorithm registration. This fixes qat-dh registration when driver is restarted Cc: <stable@vger.kernel.org> Signed-off-by: Salvatore Benedetto <salvatore.benedetto@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-27 22:06:36 +0000