diff options
author | Tejun Heo <tj@kernel.org> | 2011-11-28 09:46:22 -0800 |
---|---|---|
committer | Tejun Heo <tj@kernel.org> | 2011-11-28 09:46:22 -0800 |
commit | d4bbf7e7759afc172e2bfbc5c416324590049cdd (patch) | |
tree | 7eab5ee5481cd3dcf1162329fec827177640018a /lib | |
parent | a150439c4a97db379f0ed6faa46fbbb6e7bf3cb2 (diff) | |
parent | 401d0069cb344f401bc9d264c31db55876ff78c0 (diff) | |
download | linux-d4bbf7e7759afc172e2bfbc5c416324590049cdd.tar.gz linux-d4bbf7e7759afc172e2bfbc5c416324590049cdd.tar.bz2 linux-d4bbf7e7759afc172e2bfbc5c416324590049cdd.zip |
Merge branch 'master' into x86/memblock
Conflicts & resolutions:
* arch/x86/xen/setup.c
dc91c728fd "xen: allow extra memory to be in multiple regions"
24aa07882b "memblock, x86: Replace memblock_x86_reserve/free..."
conflicted on xen_add_extra_mem() updates. The resolution is
trivial as the latter just want to replace
memblock_x86_reserve_range() with memblock_reserve().
* drivers/pci/intel-iommu.c
166e9278a3f "x86/ia64: intel-iommu: move to drivers/iommu/"
5dfe8660a3d "bootmem: Replace work_with_active_regions() with..."
conflicted as the former moved the file under drivers/iommu/.
Resolved by applying the chnages from the latter on the moved
file.
* mm/Kconfig
6661672053a "memblock: add NO_BOOTMEM config symbol"
c378ddd53f9 "memblock, x86: Make ARCH_DISCARD_MEMBLOCK a config option"
conflicted trivially. Both added config options. Just
letting both add their own options resolves the conflict.
* mm/memblock.c
d1f0ece6cdc "mm/memblock.c: small function definition fixes"
ed7b56a799c "memblock: Remove memblock_memory_can_coalesce()"
confliected. The former updates function removed by the
latter. Resolution is trivial.
Signed-off-by: Tejun Heo <tj@kernel.org>
Diffstat (limited to 'lib')
45 files changed, 1541 insertions, 540 deletions
diff --git a/lib/Kconfig b/lib/Kconfig index 830181cc7a83..32f3e5ae2be5 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -79,6 +79,13 @@ config LIBCRC32C require M here. See Castagnoli93. Module will be libcrc32c. +config CRC8 + tristate "CRC8 function" + help + This option provides CRC8 function. Drivers may select this + when they need to do cyclic redundancy check according CRC8 + algorithm. Module will be called crc8. + config AUDIT_GENERIC bool depends on AUDIT && !AUDIT_ARCH @@ -262,4 +269,11 @@ config AVERAGE If unsure, say N. +config CORDIC + tristate "Cordic function" + help + The option provides arithmetic function using cordic algorithm + so its calculations are in fixed point. Modules can select this + when they require this function. Module will be called cordic. + endmenu diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index dd373c8ee943..82928f5ea049 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -117,31 +117,31 @@ config DEBUG_SECTION_MISMATCH help The section mismatch analysis checks if there are illegal references from one section to another section. - Linux will during link or during runtime drop some sections - and any use of code/data previously in these sections will + During linktime or runtime, some sections are dropped; + any use of code/data previously in these sections would most likely result in an oops. - In the code functions and variables are annotated with - __init, __devinit etc. (see full list in include/linux/init.h) + In the code, functions and variables are annotated with + __init, __devinit, etc. (see the full list in include/linux/init.h), which results in the code/data being placed in specific sections. - The section mismatch analysis is always done after a full - kernel build but enabling this option will in addition - do the following: - - Add the option -fno-inline-functions-called-once to gcc - When inlining a function annotated __init in a non-init - function we would lose the section information and thus + The section mismatch analysis is always performed after a full + kernel build, and enabling this option causes the following + additional steps to occur: + - Add the option -fno-inline-functions-called-once to gcc commands. + When inlining a function annotated with __init in a non-init + function, we would lose the section information and thus the analysis would not catch the illegal reference. - This option tells gcc to inline less but will also - result in a larger kernel. - - Run the section mismatch analysis for each module/built-in.o - When we run the section mismatch analysis on vmlinux.o we + This option tells gcc to inline less (but it does result in + a larger kernel). + - Run the section mismatch analysis for each module/built-in.o file. + When we run the section mismatch analysis on vmlinux.o, we lose valueble information about where the mismatch was introduced. Running the analysis for each module/built-in.o file - will tell where the mismatch happens much closer to the - source. The drawback is that we will report the same - mismatch at least twice. - - Enable verbose reporting from modpost to help solving - the section mismatches reported. + tells where the mismatch happens much closer to the + source. The drawback is that the same mismatch is + reported at least twice. + - Enable verbose reporting from modpost in order to help resolve + the section mismatches that are reported. config DEBUG_KERNEL bool "Kernel debugging" @@ -227,7 +227,7 @@ config BOOTPARAM_SOFTLOCKUP_PANIC_VALUE config DETECT_HUNG_TASK bool "Detect Hung Tasks" depends on DEBUG_KERNEL - default DETECT_SOFTLOCKUP + default LOCKUP_DETECTOR help Say Y here to enable the kernel to detect "hung tasks", which are bugs that cause the task to be stuck in @@ -248,8 +248,9 @@ config DEFAULT_HUNG_TASK_TIMEOUT to determine when a task has become non-responsive and should be considered hung. - It can be adjusted at runtime via the kernel.hung_task_timeout - sysctl or by writing a value to /proc/sys/kernel/hung_task_timeout. + It can be adjusted at runtime via the kernel.hung_task_timeout_secs + sysctl or by writing a value to + /proc/sys/kernel/hung_task_timeout_secs. A timeout of 0 disables the check. The default is two minutes. Keeping the default should be fine in most cases. @@ -648,12 +649,15 @@ config TRACE_IRQFLAGS Enables hooks to interrupt enabling and disabling for either tracing or lock debugging. -config DEBUG_SPINLOCK_SLEEP - bool "Spinlock debugging: sleep-inside-spinlock checking" +config DEBUG_ATOMIC_SLEEP + bool "Sleep inside atomic section checking" + select PREEMPT_COUNT depends on DEBUG_KERNEL help If you say Y here, various routines which may sleep will become very - noisy if they are called with a spinlock held. + noisy if they are called inside atomic sections: when a spinlock is + held, inside an rcu read side critical section, inside preempt disabled + sections, inside an interrupt, etc... config DEBUG_LOCKING_API_SELFTESTS bool "Locking API boot-time self-tests" @@ -832,7 +836,7 @@ config DEBUG_CREDENTIALS # # Select this config option from the architecture Kconfig, if it -# it is preferred to always offer frame pointers as a config +# is preferred to always offer frame pointers as a config # option on the architecture (regardless of KERNEL_DEBUG): # config ARCH_WANT_FRAME_POINTERS @@ -866,7 +870,7 @@ config BOOT_PRINTK_DELAY system, and then set "lpj=M" before setting "boot_delay=N". NOTE: Using this option may adversely affect SMP systems. I.e., processors other than the first one may not boot up. - BOOT_PRINTK_DELAY also may cause DETECT_SOFTLOCKUP to detect + BOOT_PRINTK_DELAY also may cause LOCKUP_DETECTOR to detect what it believes to be lockup conditions. config RCU_TORTURE_TEST @@ -1067,6 +1071,17 @@ config FAIL_IO_TIMEOUT Only works with drivers that use the generic timeout handling, for others it wont do anything. +config FAIL_MMC_REQUEST + bool "Fault-injection capability for MMC IO" + select DEBUG_FS + depends on FAULT_INJECTION && MMC + help + Provide fault-injection capability for MMC IO. + This will make the mmc core return data errors. This is + useful to test the error handling in the mmc block device + and to test how the mmc host driver handles retries from + the block device. + config FAULT_INJECTION_DEBUG_FS bool "Debugfs entries for fault-injection capabilities" depends on FAULT_INJECTION && SYSFS && DEBUG_FS @@ -1078,7 +1093,7 @@ config FAULT_INJECTION_STACKTRACE_FILTER depends on FAULT_INJECTION_DEBUG_FS && STACKTRACE_SUPPORT depends on !X86_64 select STACKTRACE - select FRAME_POINTER if !PPC && !S390 && !MICROBLAZE + select FRAME_POINTER if !PPC && !S390 && !MICROBLAZE && !ARM_UNWIND help Provide stacktrace filter for fault-injection capabilities @@ -1088,7 +1103,7 @@ config LATENCYTOP depends on DEBUG_KERNEL depends on STACKTRACE_SUPPORT depends on PROC_FS - select FRAME_POINTER if !MIPS && !PPC && !S390 && !MICROBLAZE + select FRAME_POINTER if !MIPS && !PPC && !S390 && !MICROBLAZE && !ARM_UNWIND select KALLSYMS select KALLSYMS_ALL select STACKTRACE diff --git a/lib/Makefile b/lib/Makefile index 6b597fdb1898..a4da283f5dc0 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -10,9 +10,9 @@ endif lib-y := ctype.o string.o vsprintf.o cmdline.o \ rbtree.o radix-tree.o dump_stack.o timerqueue.o\ idr.o int_sqrt.o extable.o prio_tree.o \ - sha1.o irq_regs.o reciprocal_div.o argv_split.o \ + sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \ proportions.o prio_heap.o ratelimit.o show_mem.o \ - is_single_threaded.o plist.o decompress.o find_next_bit.o + is_single_threaded.o plist.o decompress.o lib-$(CONFIG_MMU) += ioremap.o lib-$(CONFIG_SMP) += cpumask.o @@ -22,7 +22,7 @@ lib-y += kobject.o kref.o klist.o obj-y += bcd.o div64.o sort.o parser.o halfmd4.o debug_locks.o random32.o \ bust_spinlocks.o hexdump.o kasprintf.o bitmap.o scatterlist.o \ string_helpers.o gcd.o lcm.o list_sort.o uuid.o flex_array.o \ - bsearch.o find_last_bit.o + bsearch.o find_last_bit.o find_next_bit.o llist.o obj-y += kstrtox.o obj-$(CONFIG_TEST_KSTRTOX) += test-kstrtox.o @@ -61,6 +61,7 @@ obj-$(CONFIG_CRC_ITU_T) += crc-itu-t.o obj-$(CONFIG_CRC32) += crc32.o obj-$(CONFIG_CRC7) += crc7.o obj-$(CONFIG_LIBCRC32C) += libcrc32c.o +obj-$(CONFIG_CRC8) += crc8.o obj-$(CONFIG_GENERIC_ALLOCATOR) += genalloc.o obj-$(CONFIG_ZLIB_INFLATE) += zlib_inflate/ @@ -112,6 +113,8 @@ obj-$(CONFIG_AVERAGE) += average.o obj-$(CONFIG_CPU_RMAP) += cpu_rmap.o +obj-$(CONFIG_CORDIC) += cordic.o + hostprogs-y := gen_crc32table clean-files := crc32table.h diff --git a/lib/atomic64.c b/lib/atomic64.c index a21c12bc727c..3975470caf4f 100644 --- a/lib/atomic64.c +++ b/lib/atomic64.c @@ -14,7 +14,7 @@ #include <linux/spinlock.h> #include <linux/init.h> #include <linux/module.h> -#include <asm/atomic.h> +#include <linux/atomic.h> /* * We use a hashed array of spinlocks to provide exclusive access @@ -29,11 +29,11 @@ * Ensure each lock is in a separate cacheline. */ static union { - spinlock_t lock; + raw_spinlock_t lock; char pad[L1_CACHE_BYTES]; } atomic64_lock[NR_LOCKS] __cacheline_aligned_in_smp; -static inline spinlock_t *lock_addr(const atomic64_t *v) +static inline raw_spinlock_t *lock_addr(const atomic64_t *v) { unsigned long addr = (unsigned long) v; @@ -45,12 +45,12 @@ static inline spinlock_t *lock_addr(const atomic64_t *v) long long atomic64_read(const atomic64_t *v) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); long long val; - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); val = v->counter; - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); return val; } EXPORT_SYMBOL(atomic64_read); @@ -58,34 +58,34 @@ EXPORT_SYMBOL(atomic64_read); void atomic64_set(atomic64_t *v, long long i) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); v->counter = i; - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); } EXPORT_SYMBOL(atomic64_set); void atomic64_add(long long a, atomic64_t *v) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); v->counter += a; - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); } EXPORT_SYMBOL(atomic64_add); long long atomic64_add_return(long long a, atomic64_t *v) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); long long val; - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); val = v->counter += a; - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); return val; } EXPORT_SYMBOL(atomic64_add_return); @@ -93,23 +93,23 @@ EXPORT_SYMBOL(atomic64_add_return); void atomic64_sub(long long a, atomic64_t *v) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); v->counter -= a; - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); } EXPORT_SYMBOL(atomic64_sub); long long atomic64_sub_return(long long a, atomic64_t *v) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); long long val; - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); val = v->counter -= a; - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); return val; } EXPORT_SYMBOL(atomic64_sub_return); @@ -117,14 +117,14 @@ EXPORT_SYMBOL(atomic64_sub_return); long long atomic64_dec_if_positive(atomic64_t *v) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); long long val; - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); val = v->counter - 1; if (val >= 0) v->counter = val; - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); return val; } EXPORT_SYMBOL(atomic64_dec_if_positive); @@ -132,14 +132,14 @@ EXPORT_SYMBOL(atomic64_dec_if_positive); long long atomic64_cmpxchg(atomic64_t *v, long long o, long long n) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); long long val; - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); val = v->counter; if (val == o) v->counter = n; - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); return val; } EXPORT_SYMBOL(atomic64_cmpxchg); @@ -147,13 +147,13 @@ EXPORT_SYMBOL(atomic64_cmpxchg); long long atomic64_xchg(atomic64_t *v, long long new) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); long long val; - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); val = v->counter; v->counter = new; - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); return val; } EXPORT_SYMBOL(atomic64_xchg); @@ -161,15 +161,15 @@ EXPORT_SYMBOL(atomic64_xchg); int atomic64_add_unless(atomic64_t *v, long long a, long long u) { unsigned long flags; - spinlock_t *lock = lock_addr(v); + raw_spinlock_t *lock = lock_addr(v); int ret = 0; - spin_lock_irqsave(lock, flags); + raw_spin_lock_irqsave(lock, flags); if (v->counter != u) { v->counter += a; ret = 1; } - spin_unlock_irqrestore(lock, flags); + raw_spin_unlock_irqrestore(lock, flags); return ret; } EXPORT_SYMBOL(atomic64_add_unless); @@ -179,7 +179,7 @@ static int init_atomic64_lock(void) int i; for (i = 0; i < NR_LOCKS; ++i) - spin_lock_init(&atomic64_lock[i].lock); + raw_spin_lock_init(&atomic64_lock[i].lock); return 0; } diff --git a/lib/atomic64_test.c b/lib/atomic64_test.c index 44524cc8c32a..0c33cde2a1e6 100644 --- a/lib/atomic64_test.c +++ b/lib/atomic64_test.c @@ -10,7 +10,7 @@ */ #include <linux/init.h> #include <linux/kernel.h> -#include <asm/atomic.h> +#include <linux/atomic.h> #define INIT(c) do { atomic64_set(&v, c); r = c; } while (0) static __init int test_atomic64(void) diff --git a/lib/bitmap.c b/lib/bitmap.c index 3f3b68199d74..0d4a127dd9b3 100644 --- a/lib/bitmap.c +++ b/lib/bitmap.c @@ -271,8 +271,6 @@ int __bitmap_weight(const unsigned long *bitmap, int bits) } EXPORT_SYMBOL(__bitmap_weight); -#define BITMAP_FIRST_WORD_MASK(start) (~0UL << ((start) % BITS_PER_LONG)) - void bitmap_set(unsigned long *map, int start, int nr) { unsigned long *p = map + BIT_WORD(start); @@ -421,7 +419,7 @@ int __bitmap_parse(const char *buf, unsigned int buflen, { int c, old_c, totaldigits, ndigits, nchunks, nbits; u32 chunk; - const char __user *ubuf = buf; + const char __user __force *ubuf = (const char __user __force *)buf; bitmap_zero(maskp, nmaskbits); @@ -506,7 +504,9 @@ int bitmap_parse_user(const char __user *ubuf, { if (!access_ok(VERIFY_READ, ubuf, ulen)) return -EFAULT; - return __bitmap_parse((const char *)ubuf, ulen, 1, maskp, nmaskbits); + return __bitmap_parse((const char __force *)ubuf, + ulen, 1, maskp, nmaskbits); + } EXPORT_SYMBOL(bitmap_parse_user); @@ -596,7 +596,7 @@ static int __bitmap_parselist(const char *buf, unsigned int buflen, { unsigned a, b; int c, old_c, totaldigits; - const char __user *ubuf = buf; + const char __user __force *ubuf = (const char __user __force *)buf; int exp_digit, in_range; totaldigits = c = 0; @@ -696,7 +696,7 @@ int bitmap_parselist_user(const char __user *ubuf, { if (!access_ok(VERIFY_READ, ubuf, ulen)) return -EFAULT; - return __bitmap_parselist((const char *)ubuf, + return __bitmap_parselist((const char __force *)ubuf, ulen, 1, maskp, nmaskbits); } EXPORT_SYMBOL(bitmap_parselist_user); @@ -756,7 +756,7 @@ static int bitmap_pos_to_ord(const unsigned long *buf, int pos, int bits) * * The bit positions 0 through @bits are valid positions in @buf. */ -static int bitmap_ord_to_pos(const unsigned long *buf, int ord, int bits) +int bitmap_ord_to_pos(const unsigned long *buf, int ord, int bits) { int pos = 0; diff --git a/lib/checksum.c b/lib/checksum.c index 097508732f34..8df2f91e6d98 100644 --- a/lib/checksum.c +++ b/lib/checksum.c @@ -49,7 +49,7 @@ static inline unsigned short from32to16(unsigned int x) static unsigned int do_csum(const unsigned char *buff, int len) { - int odd, count; + int odd; unsigned int result = 0; if (len <= 0) @@ -64,25 +64,22 @@ static unsigned int do_csum(const unsigned char *buff, int len) len--; buff++; } - count = len >> 1; /* nr of 16-bit words.. */ - if (count) { + if (len >= 2) { if (2 & (unsigned long) buff) { result += *(unsigned short *) buff; - count--; len -= 2; buff += 2; } - count >>= 1; /* nr of 32-bit words.. */ - if (count) { + if (len >= 4) { + const unsigned char *end = buff + ((unsigned)len & ~3); unsigned int carry = 0; do { unsigned int w = *(unsigned int *) buff; - count--; buff += 4; result += carry; result += w; carry = (w > result); - } while (count); + } while (buff < end); result += carry; result = (result & 0xffff) + (result >> 16); } diff --git a/lib/cordic.c b/lib/cordic.c new file mode 100644 index 000000000000..aa27a88d7e04 --- /dev/null +++ b/lib/cordic.c @@ -0,0 +1,101 @@ +/* + * Copyright (c) 2011 Broadcom Corporation + * + * Permission to use, copy, modify, and/or distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + * SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION + * OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN + * CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ +#include <linux/module.h> +#include <linux/cordic.h> + +#define CORDIC_ANGLE_GEN 39797 +#define CORDIC_PRECISION_SHIFT 16 +#define CORDIC_NUM_ITER (CORDIC_PRECISION_SHIFT + 2) + +#define FIXED(X) ((s32)((X) << CORDIC_PRECISION_SHIFT)) +#define FLOAT(X) (((X) >= 0) \ + ? ((((X) >> (CORDIC_PRECISION_SHIFT - 1)) + 1) >> 1) \ + : -((((-(X)) >> (CORDIC_PRECISION_SHIFT - 1)) + 1) >> 1)) + +static const s32 arctan_table[] = { + 2949120, + 1740967, + 919879, + 466945, + 234379, + 117304, + 58666, + 29335, + 14668, + 7334, + 3667, + 1833, + 917, + 458, + 229, + 115, + 57, + 29 +}; + +/* + * cordic_calc_iq() - calculates the i/q coordinate for given angle + * + * theta: angle in degrees for which i/q coordinate is to be calculated + * coord: function output parameter holding the i/q coordinate + */ +struct cordic_iq cordic_calc_iq(s32 theta) +{ + struct cordic_iq coord; + s32 angle, valtmp; + unsigned iter; + int signx = 1; + int signtheta; + + coord.i = CORDIC_ANGLE_GEN; + coord.q = 0; + angle = 0; + + theta = FIXED(theta); + signtheta = (theta < 0) ? -1 : 1; + theta = ((theta + FIXED(180) * signtheta) % FIXED(360)) - + FIXED(180) * signtheta; + + if (FLOAT(theta) > 90) { + theta -= FIXED(180); + signx = -1; + } else if (FLOAT(theta) < -90) { + theta += FIXED(180); + signx = -1; + } + + for (iter = 0; iter < CORDIC_NUM_ITER; iter++) { + if (theta > angle) { + valtmp = coord.i - (coord.q >> iter); + coord.q += (coord.i >> iter); + angle += arctan_table[iter]; + } else { + valtmp = coord.i + (coord.q >> iter); + coord.q -= (coord.i >> iter); + angle -= arctan_table[iter]; + } + coord.i = valtmp; + } + + coord.i *= signx; + coord.q *= signx; + return coord; +} +EXPORT_SYMBOL(cordic_calc_iq); + +MODULE_DESCRIPTION("Cordic functions"); +MODULE_AUTHOR("Broadcom Corporation"); +MODULE_LICENSE("Dual BSD/GPL"); diff --git a/lib/cpumask.c b/lib/cpumask.c index 05d6aca7fc19..af3e5817de98 100644 --- a/lib/cpumask.c +++ b/lib/cpumask.c @@ -30,7 +30,7 @@ int __any_online_cpu(const cpumask_t *mask) { int cpu; - for_each_cpu_mask(cpu, *mask) { + for_each_cpu(cpu, mask) { if (cpu_online(cpu)) break; } @@ -131,7 +131,7 @@ EXPORT_SYMBOL(zalloc_cpumask_var_node); */ bool alloc_cpumask_var(cpumask_var_t *mask, gfp_t flags) { - return alloc_cpumask_var_node(mask, flags, numa_node_id()); + return alloc_cpumask_var_node(mask, flags, NUMA_NO_NODE); } EXPORT_SYMBOL(alloc_cpumask_var); diff --git a/lib/crc32.c b/lib/crc32.c index 4855995fcde9..a6e633a48cea 100644 --- a/lib/crc32.c +++ b/lib/crc32.c @@ -26,7 +26,7 @@ #include <linux/compiler.h> #include <linux/types.h> #include <linux/init.h> -#include <asm/atomic.h> +#include <linux/atomic.h> #include "crc32defs.h" #if CRC_LE_BITS == 8 # define tole(x) __constant_cpu_to_le32(x) diff --git a/lib/crc8.c b/lib/crc8.c new file mode 100644 index 000000000000..87b59cafdb83 --- /dev/null +++ b/lib/crc8.c @@ -0,0 +1,86 @@ +/* + * Copyright (c) 2011 Broadcom Corporation + * + * Permission to use, copy, modify, and/or distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + * SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION + * OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN + * CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include <linux/module.h> +#include <linux/crc8.h> +#include <linux/printk.h> + +/* + * crc8_populate_msb - fill crc table for given polynomial in reverse bit order. + * + * table: table to be filled. + * polynomial: polynomial for which table is to be filled. + */ +void crc8_populate_msb(u8 table[CRC8_TABLE_SIZE], u8 polynomial) +{ + int i, j; + const u8 msbit = 0x80; + u8 t = msbit; + + table[0] = 0; + + for (i = 1; i < CRC8_TABLE_SIZE; i *= 2) { + t = (t << 1) ^ (t & msbit ? polynomial : 0); + for (j = 0; j < i; j++) + table[i+j] = table[j] ^ t; + } +} +EXPORT_SYMBOL(crc8_populate_msb); + +/* + * crc8_populate_lsb - fill crc table for given polynomial in regular bit order. + * + * table: table to be filled. + * polynomial: polynomial for which table is to be filled. + */ +void crc8_populate_lsb(u8 table[CRC8_TABLE_SIZE], u8 polynomial) +{ + int i, j; + u8 t = 1; + + table[0] = 0; + + for (i = (CRC8_TABLE_SIZE >> 1); i; i >>= 1) { + t = (t >> 1) ^ (t & 1 ? polynomial : 0); + for (j = 0; j < CRC8_TABLE_SIZE; j += 2*i) + table[i+j] = table[j] ^ t; + } +} +EXPORT_SYMBOL(crc8_populate_lsb); + +/* + * crc8 - calculate a crc8 over the given input data. + * + * table: crc table used for calculation. + * pdata: pointer to data buffer. + * nbytes: number of bytes in data buffer. + * crc: previous returned crc8 value. + */ +u8 crc8(const u8 table[CRC8_TABLE_SIZE], u8 *pdata, size_t nbytes, u8 crc) +{ + /* loop over the buffer data */ + while (nbytes-- > 0) + crc = table[(crc ^ *pdata++) & 0xff]; + + return crc; +} +EXPORT_SYMBOL(crc8); + +MODULE_DESCRIPTION("CRC8 (by Williams, Ross N.) function"); +MODULE_AUTHOR("Broadcom Corporation"); +MODULE_LICENSE("Dual BSD/GPL"); diff --git a/lib/dec_and_lock.c b/lib/dec_and_lock.c index e73822aa6e9a..b5257725daad 100644 --- a/lib/dec_and_lock.c +++ b/lib/dec_and_lock.c @@ -1,6 +1,6 @@ #include <linux/module.h> #include <linux/spinlock.h> -#include <asm/atomic.h> +#include <linux/atomic.h> /* * This is an implementation of the notion of "decrement a diff --git a/lib/devres.c b/lib/devres.c index 6efddf53b90c..7c0e953a7486 100644 --- a/lib/devres.c +++ b/lib/devres.c @@ -79,9 +79,9 @@ EXPORT_SYMBOL(devm_ioremap_nocache); */ void devm_iounmap(struct device *dev, void __iomem *addr) { - iounmap(addr); WARN_ON(devres_destroy(dev, devm_ioremap_release, devm_ioremap_match, (void *)addr)); + iounmap(addr); } EXPORT_SYMBOL(devm_iounmap); diff --git a/lib/dma-debug.c b/lib/dma-debug.c index db07bfd9298e..74c6c7fce749 100644 --- a/lib/dma-debug.c +++ b/lib/dma-debug.c @@ -24,6 +24,7 @@ #include <linux/spinlock.h> #include <linux/debugfs.h> #include <linux/uaccess.h> +#include <linux/export.h> #include <linux/device.h> #include <linux/types.h> #include <linux/sched.h> @@ -62,6 +63,8 @@ struct dma_debug_entry { #endif }; +typedef bool (*match_fn)(struct dma_debug_entry *, struct dma_debug_entry *); + struct hash_bucket { struct list_head list; spinlock_t lock; @@ -240,18 +243,37 @@ static void put_hash_bucket(struct hash_bucket *bucket, spin_unlock_irqrestore(&bucket->lock, __flags); } +static bool exact_match(struct dma_debug_entry *a, struct dma_debug_entry *b) +{ + return ((a->dev_addr == a->dev_addr) && + (a->dev == b->dev)) ? true : false; +} + +static bool containing_match(struct dma_debug_entry *a, + struct dma_debug_entry *b) +{ + if (a->dev != b->dev) + return false; + + if ((b->dev_addr <= a->dev_addr) && + ((b->dev_addr + b->size) >= (a->dev_addr + a->size))) + return true; + + return false; +} + /* * Search a given entry in the hash bucket list */ -static struct dma_debug_entry *hash_bucket_find(struct hash_bucket *bucket, - struct dma_debug_entry *ref) +static struct dma_debug_entry *__hash_bucket_find(struct hash_bucket *bucket, + struct dma_debug_entry *ref, + match_fn match) { struct dma_debug_entry *entry, *ret = NULL; int matches = 0, match_lvl, last_lvl = 0; list_for_each_entry(entry, &bucket->list, list) { - if ((entry->dev_addr != ref->dev_addr) || - (entry->dev != ref->dev)) + if (!match(ref, entry)) continue; /* @@ -293,6 +315,39 @@ static struct dma_debug_entry *hash_bucket_find(struct hash_bucket *bucket, return ret; } +static struct dma_debug_entry *bucket_find_exact(struct hash_bucket *bucket, + struct dma_debug_entry *ref) +{ + return __hash_bucket_find(bucket, ref, exact_match); +} + +static struct dma_debug_entry *bucket_find_contain(struct hash_bucket **bucket, + struct dma_debug_entry *ref, + unsigned long *flags) +{ + + unsigned int max_range = dma_get_max_seg_size(ref->dev); + struct dma_debug_entry *entry, index = *ref; + unsigned int range = 0; + + while (range <= max_range) { + entry = __hash_bucket_find(*bucket, &index, containing_match); + + if (entry) + return entry; + + /* + * Nothing found, go back a hash bucket + */ + put_hash_bucket(*bucket, flags); + range += (1 << HASH_FN_SHIFT); + index.dev_addr -= (1 << HASH_FN_SHIFT); + *bucket = get_hash_bucket(&index, flags); + } + + return NULL; +} + /* * Add an entry to a hash bucket */ @@ -802,7 +857,7 @@ static void check_unmap(struct dma_debug_entry *ref) } bucket = get_hash_bucket(ref, &flags); - entry = hash_bucket_find(bucket, ref); + entry = bucket_find_exact(bucket, ref); if (!entry) { err_printk(ref->dev, NULL, "DMA-API: device driver tries " @@ -902,7 +957,7 @@ static void check_sync(struct device *dev, bucket = get_hash_bucket(ref, &flags); - entry = hash_bucket_find(bucket, ref); + entry = bucket_find_contain(&bucket, ref, &flags); if (!entry) { err_printk(dev, NULL, "DMA-API: device driver tries " @@ -1060,7 +1115,7 @@ static int get_nr_mapped_entries(struct device *dev, int mapped_ents; bucket = get_hash_bucket(ref, &flags); - entry = hash_bucket_find(bucket, ref); + entry = bucket_find_exact(bucket, ref); mapped_ents = 0; if (entry) diff --git a/lib/dynamic_debug.c b/lib/dynamic_debug.c index 75ca78f3a8c9..dcdade39e47f 100644 --- a/lib/dynamic_debug.c +++ b/lib/dynamic_debug.c @@ -10,11 +10,12 @@ * Copyright (C) 2011 Bart Van Assche. All Rights Reserved. */ +#define pr_fmt(fmt) KBUILD_MODNAME ":%s: " fmt, __func__ + #include <linux/kernel.h> #include <linux/module.h> #include <linux/moduleparam.h> #include <linux/kallsyms.h> -#include <linux/version.h> #include <linux/types.h> #include <linux/mutex.h> #include <linux/proc_fs.h> @@ -30,6 +31,8 @@ #include <linux/jump_label.h> #include <linux/hardirq.h> #include <linux/sched.h> +#include <linux/device.h> +#include <linux/netdevice.h> extern struct _ddebug __start___verbose[]; extern struct _ddebug __stop___verbose[]; @@ -38,7 +41,6 @@ struct ddebug_table { struct list_head link; char *mod_name; unsigned int num_ddebugs; - unsigned int num_enabled; struct _ddebug *ddebugs; }; @@ -148,19 +150,13 @@ static void ddebug_change(const struct ddebug_query *query, newflags = (dp->flags & mask) | flags; if (newflags == dp->flags) continue; - - if (!newflags) - dt->num_enabled--; - else if (!dp->flags) - dt->num_enabled++; dp->flags = newflags; if (newflags) dp->enabled = 1; else dp->enabled = 0; if (verbose) - printk(KERN_INFO - "ddebug: changed %s:%d [%s]%s %s\n", + pr_info("changed %s:%d [%s]%s %s\n", dp->filename, dp->lineno, dt->mod_name, dp->function, ddebug_describe_flags(dp, flagbuf, @@ -170,7 +166,7 @@ static void ddebug_change(const struct ddebug_query *query, mutex_unlock(&ddebug_lock); if (!nfound && verbose) - printk(KERN_INFO "ddebug: no matches for query\n"); + pr_info("no matches for query\n"); } /* @@ -215,10 +211,10 @@ static int ddebug_tokenize(char *buf, char *words[], int maxwords) if (verbose) { int i; - printk(KERN_INFO "%s: split into words:", __func__); + pr_info("split into words:"); for (i = 0 ; i < nwords ; i++) - printk(" \"%s\"", words[i]); - printk("\n"); + pr_cont(" \"%s\"", words[i]); + pr_cont("\n"); } return nwords; @@ -330,16 +326,15 @@ static int ddebug_parse_query(char *words[], int nwords, } } else { if (verbose) - printk(KERN_ERR "%s: unknown keyword \"%s\"\n", - __func__, words[i]); + pr_err("unknown keyword \"%s\"\n", words[i]); return -EINVAL; } } if (verbose) - printk(KERN_INFO "%s: q->function=\"%s\" q->filename=\"%s\" " - "q->module=\"%s\" q->format=\"%s\" q->lineno=%u-%u\n", - __func__, query->function, query->filename, + pr_info("q->function=\"%s\" q->filename=\"%s\" " + "q->module=\"%s\" q->format=\"%s\" q->lineno=%u-%u\n", + query->function, query->filename, query->module, query->format, query->first_lineno, query->last_lineno); @@ -368,7 +363,7 @@ static int ddebug_parse_flags(const char *str, unsigned int *flagsp, return -EINVAL; } if (verbose) - printk(KERN_INFO "%s: op='%c'\n", __func__, op); + pr_info("op='%c'\n", op); for ( ; *str ; ++str) { for (i = ARRAY_SIZE(opt_array) - 1; i >= 0; i--) { @@ -383,7 +378,7 @@ static int ddebug_parse_flags(const char *str, unsigned int *flagsp, if (flags == 0) return -EINVAL; if (verbose) - printk(KERN_INFO "%s: flags=0x%x\n", __func__, flags); + pr_info("flags=0x%x\n", flags); /* calculate final *flagsp, *maskp according to mask and op */ switch (op) { @@ -401,8 +396,7 @@ static int ddebug_parse_flags(const char *str, unsigned int *flagsp, break; } if (verbose) - printk(KERN_INFO "%s: *flagsp=0x%x *maskp=0x%x\n", - __func__, *flagsp, *maskp); + pr_info("*flagsp=0x%x *maskp=0x%x\n", *flagsp, *maskp); return 0; } @@ -427,40 +421,117 @@ static int ddebug_exec_query(char *query_string) return 0; } +#define PREFIX_SIZE 64 + +static int remaining(int wrote) +{ + if (PREFIX_SIZE - wrote > 0) + return PREFIX_SIZE - wrote; + return 0; +} + +static char *dynamic_emit_prefix(const struct _ddebug *desc, char *buf) +{ + int pos_after_tid; + int pos = 0; + + pos += snprintf(buf + pos, remaining(pos), "%s", KERN_DEBUG); + if (desc->flags & _DPRINTK_FLAGS_INCL_TID) { + if (in_interrupt()) + pos += snprintf(buf + pos, remaining(pos), "%s ", + "<intr>"); + else + pos += snprintf(buf + pos, remaining(pos), "[%d] ", + task_pid_vnr(current)); + } + pos_after_tid = pos; + if (desc->flags & _DPRINTK_FLAGS_INCL_MODNAME) + pos += snprintf(buf + pos, remaining(pos), "%s:", + desc->modname); + if (desc->flags & _DPRINTK_FLAGS_INCL_FUNCNAME) + pos += snprintf(buf + pos, remaining(pos), "%s:", + desc->function); + if (desc->flags & _DPRINTK_FLAGS_INCL_LINENO) + pos += snprintf(buf + pos, remaining(pos), "%d:", desc->lineno); + if (pos - pos_after_tid) + pos += snprintf(buf + pos, remaining(pos), " "); + if (pos >= PREFIX_SIZE) + buf[PREFIX_SIZE - 1] = '\0'; + + return buf; +} + int __dynamic_pr_debug(struct _ddebug *descriptor, const char *fmt, ...) { va_list args; int res; + struct va_format vaf; + char buf[PREFIX_SIZE]; BUG_ON(!descriptor); BUG_ON(!fmt); va_start(args, fmt); - res = printk(KERN_DEBUG); - if (descriptor->flags & _DPRINTK_FLAGS_INCL_TID) { - if (in_interrupt()) - res += printk(KERN_CONT "<intr> "); - else - res += printk(KERN_CONT "[%d] ", task_pid_vnr(current)); - } - if (descriptor->flags & _DPRINTK_FLAGS_INCL_MODNAME) - res += printk(KERN_CONT "%s:", descriptor->modname); - if (descriptor->flags & _DPRINTK_FLAGS_INCL_FUNCNAME) - res += printk(KERN_CONT "%s:", descriptor->function); - if (descriptor->flags & _DPRINTK_FLAGS_INCL_LINENO) - res += printk(KERN_CONT "%d ", descriptor->lineno); - res += vprintk(fmt, args); + vaf.fmt = fmt; + vaf.va = &args; + res = printk("%s%pV", dynamic_emit_prefix(descriptor, buf), &vaf); va_end(args); return res; } EXPORT_SYMBOL(__dynamic_pr_debug); +int __dynamic_dev_dbg(struct _ddebug *descriptor, + const struct device *dev, const char *fmt, ...) +{ + struct va_format vaf; + va_list args; + int res; + char buf[PREFIX_SIZE]; + + BUG_ON(!descriptor); + BUG_ON(!fmt); + + va_start(args, fmt); + vaf.fmt = fmt; + vaf.va = &args; + res = __dev_printk(dynamic_emit_prefix(descriptor, buf), dev, &vaf); + va_end(args); + + return res; +} +EXPORT_SYMBOL(__dynamic_dev_dbg); + +#ifdef CONFIG_NET + +int __dynamic_netdev_dbg(struct _ddebug *descriptor, + const struct net_device *dev, const char *fmt, ...) +{ + struct va_format vaf; + va_list args; + int res; + char buf[PREFIX_SIZE]; + + BUG_ON(!descriptor); + BUG_ON(!fmt); + + va_start(args, fmt); + vaf.fmt = fmt; + vaf.va = &args; + res = __netdev_printk(dynamic_emit_prefix(descriptor, buf), dev, &vaf); + va_end(args); + + return res; +} +EXPORT_SYMBOL(__dynamic_netdev_dbg); + +#endif + static __initdata char ddebug_setup_string[1024]; static __init int ddebug_setup_query(char *str) { if (strlen(str) >= 1024) { - pr_warning("ddebug boot param string too large\n"); + pr_warn("ddebug boot param string too large\n"); return 0; } strcpy(ddebug_setup_string, str); @@ -488,8 +559,7 @@ static ssize_t ddebug_proc_write(struct file *file, const char __user *ubuf, return -EFAULT; tmpbuf[len] = '\0'; if (verbose) - printk(KERN_INFO "%s: read %d bytes from userspace\n", - __func__, (int)len); + pr_info("read %d bytes from userspace\n", (int)len); ret = ddebug_exec_query(tmpbuf); if (ret) @@ -552,8 +622,7 @@ static void *ddebug_proc_start(struct seq_file *m, loff_t *pos) int n = *pos; if (verbose) - printk(KERN_INFO "%s: called m=%p *pos=%lld\n", - __func__, m, (unsigned long long)*pos); + pr_info("called m=%p *pos=%lld\n", m, (unsigned long long)*pos); mutex_lock(&ddebug_lock); @@ -578,8 +647,8 @@ static void *ddebug_proc_next(struct seq_file *m, void *p, loff_t *pos) struct _ddebug *dp; if (verbose) - printk(KERN_INFO "%s: called m=%p p=%p *pos=%lld\n", - __func__, m, p, (unsigned long long)*pos); + pr_info("called m=%p p=%p *pos=%lld\n", + m, p, (unsigned long long)*pos); if (p == SEQ_START_TOKEN) dp = ddebug_iter_first(iter); @@ -602,8 +671,7 @@ static int ddebug_proc_show(struct seq_file *m, void *p) char flagsbuf[8]; if (verbose) - printk(KERN_INFO "%s: called m=%p p=%p\n", - __func__, m, p); + pr_info("called m=%p p=%p\n", m, p); if (p == SEQ_START_TOKEN) { seq_puts(m, @@ -628,8 +696,7 @@ static int ddebug_proc_show(struct seq_file *m, void *p) static void ddebug_proc_stop(struct seq_file *m, void *p) { if (verbose) - printk(KERN_INFO "%s: called m=%p p=%p\n", - __func__, m, p); + pr_info("called m=%p p=%p\n", m, p); mutex_unlock(&ddebug_lock); } @@ -652,7 +719,7 @@ static int ddebug_proc_open(struct inode *inode, struct file *file) int err; if (verbose) - printk(KERN_INFO "%s: called\n", __func__); + pr_info("called\n"); iter = kzalloc(sizeof(*iter), GFP_KERNEL); if (iter == NULL) @@ -696,7 +763,6 @@ int ddebug_add_module(struct _ddebug *tab, unsigned int n, } dt->mod_name = new_name; dt->num_ddebugs = n; - dt->num_enabled = 0; dt->ddebugs = tab; mutex_lock(&ddebug_lock); @@ -704,8 +770,7 @@ int ddebug_add_module(struct _ddebug *tab, unsigned int n, mutex_unlock(&ddebug_lock); if (verbose) - printk(KERN_INFO "%u debug prints in module %s\n", - n, dt->mod_name); + pr_info("%u debug prints in module %s\n", n, dt->mod_name); return 0; } EXPORT_SYMBOL_GPL(ddebug_add_module); @@ -727,8 +792,7 @@ int ddebug_remove_module(const char *mod_name) int ret = -ENOENT; if (verbose) - printk(KERN_INFO "%s: removing module \"%s\"\n", - __func__, mod_name); + pr_info("removing module \"%s\"\n", mod_name); mutex_lock(&ddebug_lock); list_for_each_entry_safe(dt, nextdt, &ddebug_tables, link) { @@ -804,8 +868,8 @@ static int __init dynamic_debug_init(void) if (ddebug_setup_string[0] != '\0') { ret = ddebug_exec_query(ddebug_setup_string); if (ret) - pr_warning("Invalid ddebug boot param %s", - ddebug_setup_string); + pr_warn("Invalid ddebug boot param %s", + ddebug_setup_string); else pr_info("ddebug initialized with string %s", ddebug_setup_string); diff --git a/lib/fault-inject.c b/lib/fault-inject.c index 7e65af70635e..4f7554025e30 100644 --- a/lib/fault-inject.c +++ b/lib/fault-inject.c @@ -8,14 +8,13 @@ #include <linux/module.h> #include <linux/interrupt.h> #include <linux/stacktrace.h> -#include <linux/kallsyms.h> #include <linux/fault-inject.h> /* * setup_fault_attr() is a helper function for various __setup handlers, so it * returns 0 on error, because that is what __setup handlers do. */ -int __init setup_fault_attr(struct fault_attr *attr, char *str) +int setup_fault_attr(struct fault_attr *attr, char *str) { unsigned long probability; unsigned long interval; @@ -37,6 +36,7 @@ int __init setup_fault_attr(struct fault_attr *attr, char *str) return 1; } +EXPORT_SYMBOL_GPL(setup_fault_attr); static void fail_dump(struct fault_attr *attr) { @@ -131,6 +131,7 @@ bool should_fail(struct fault_attr *attr, ssize_t size) return true; } +EXPORT_SYMBOL_GPL(should_fail); #ifdef CONFIG_FAULT_INJECTION_DEBUG_FS @@ -140,16 +141,6 @@ static int debugfs_ul_set(void *data, u64 val) return 0; } -#ifdef CONFIG_FAULT_INJECTION_STACKTRACE_FILTER -static int debugfs_ul_set_MAX_STACK_TRACE_DEPTH(void *data, u64 val) -{ - *(unsigned long *)data = - val < MAX_STACK_TRACE_DEPTH ? - val : MAX_STACK_TRACE_DEPTH; - return 0; -} -#endif /* CONFIG_FAULT_INJECTION_STACKTRACE_FILTER */ - static int debugfs_ul_get(void *data, u64 *val) { *val = *(unsigned long *)data; @@ -165,16 +156,26 @@ static struct dentry *debugfs_create_ul(const char *name, mode_t mode, } #ifdef CONFIG_FAULT_INJECTION_STACKTRACE_FILTER -DEFINE_SIMPLE_ATTRIBUTE(fops_ul_MAX_STACK_TRACE_DEPTH, debugfs_ul_get, - debugfs_ul_set_MAX_STACK_TRACE_DEPTH, "%llu\n"); -static struct dentry *debugfs_create_ul_MAX_STACK_TRACE_DEPTH( +static int debugfs_stacktrace_depth_set(void *data, u64 val) +{ + *(unsigned long *)data = + min_t(unsigned long, val, MAX_STACK_TRACE_DEPTH); + + return 0; +} + +DEFINE_SIMPLE_ATTRIBUTE(fops_stacktrace_depth, debugfs_ul_get, + debugfs_stacktrace_depth_set, "%llu\n"); + +static struct dentry *debugfs_create_stacktrace_depth( const char *name, mode_t mode, struct dentry *parent, unsigned long *value) { return debugfs_create_file(name, mode, parent, value, - &fops_ul_MAX_STACK_TRACE_DEPTH); + &fops_stacktrace_depth); } + #endif /* CONFIG_FAULT_INJECTION_STACKTRACE_FILTER */ static int debugfs_atomic_t_set(void *data, u64 val) @@ -198,118 +199,52 @@ static struct dentry *debugfs_create_atomic_t(const char *name, mode_t mode, return debugfs_create_file(name, mode, parent, value, &fops_atomic_t); } -void cleanup_fault_attr_dentries(struct fault_attr *attr) -{ - debugfs_remove(attr->dentries.probability_file); - attr->dentries.probability_file = NULL; - - debugfs_remove(attr->dentries.interval_file); - attr->dentries.interval_file = NULL; - - debugfs_remove(attr->dentries.times_file); - attr->dentries.times_file = NULL; - - debugfs_remove(attr->dentries.space_file); - attr->dentries.space_file = NULL; - - debugfs_remove(attr->dentries.verbose_file); - attr->dentries.verbose_file = NULL; - - debugfs_remove(attr->dentries.task_filter_file); - attr->dentries.task_filter_file = NULL; - -#ifdef CONFIG_FAULT_INJECTION_STACKTRACE_FILTER - - debugfs_remove(attr->dentries.stacktrace_depth_file); - attr->dentries.stacktrace_depth_file = NULL; - - debugfs_remove(attr->dentries.require_start_file); - attr->dentries.require_start_file = NULL; - - debugfs_remove(attr->dentries.require_end_file); - attr->dentries.require_end_file = NULL; - - debugfs_remove(attr->dentries.reject_start_file); - attr->dentries.reject_start_file = NULL; - - debugfs_remove(attr->dentries.reject_end_file); - attr->dentries.reject_end_file = NULL; - -#endif /* CONFIG_FAULT_INJECTION_STACKTRACE_FILTER */ - - if (attr->dentries.dir) - WARN_ON(!simple_empty(attr->dentries.dir)); - - debugfs_remove(attr->dentries.dir); - attr->dentries.dir = NULL; -} - -int init_fault_attr_dentries(struct fault_attr *attr, const char *name) +struct dentry *fault_create_debugfs_attr(const char *name, + struct dentry *parent, struct fault_attr *attr) { mode_t mode = S_IFREG | S_IRUSR | S_IWUSR; struct dentry *dir; - memset(&attr->dentries, 0, sizeof(attr->dentries)); - - dir = debugfs_create_dir(name, NULL); + dir = debugfs_create_dir(name, parent); if (!dir) - goto fail; - attr->dentries.dir = dir; - - attr->dentries.probability_file = - debugfs_create_ul("probability", mode, dir, &attr->probability); + return ERR_PTR(-ENOMEM); - attr->dentries.interval_file = - debugfs_create_ul("interval", mode, dir, &attr->interval); - - attr->dentries.times_file = - debugfs_create_atomic_t("times", mode, dir, &attr->times); - - attr->dentries.space_file = - debugfs_create_atomic_t("space", mode, dir, &attr->space); - - attr->dentries.verbose_file = - debugfs_create_ul("verbose", mode, dir, &attr->verbose); - - attr->dentries.task_filter_file = debugfs_create_bool("task-filter", - mode, dir, &attr->task_filter); - - if (!attr->dentries.probability_file || !attr->dentries.interval_file || - !attr->dentries.times_file || !attr->dentries.space_file || - !attr->dentries.verbose_file || !attr->dentries.task_filter_file) + if (!debugfs_create_ul("probability", mode, dir, &attr->probability)) + goto fail; + if (!debugfs_create_ul("interval", mode, dir, &attr->interval)) + goto fail; + if (!debugfs_create_atomic_t("times", mode, dir, &attr->times)) + goto fail; + if (!debugfs_create_atomic_t("space", mode, dir, &attr->space)) + goto fail; + if (!debugfs_create_ul("verbose", mode, dir, &attr->verbose)) + goto fail; + if (!debugfs_create_bool("task-filter", mode, dir, &attr->task_filter)) goto fail; #ifdef CONFIG_FAULT_INJECTION_STACKTRACE_FILTER - attr->dentries.stacktrace_depth_file = - debugfs_create_ul_MAX_STACK_TRACE_DEPTH( - "stacktrace-depth", mode, dir, &attr->stacktrace_depth); - - attr->dentries.require_start_file = - debugfs_create_ul("require-start", mode, dir, &attr->require_start); - - attr->dentries.require_end_file = - debugfs_create_ul("require-end", mode, dir, &attr->require_end); - - attr->dentries.reject_start_file = - debugfs_create_ul("reject-start", mode, dir, &attr->reject_start); - - attr->dentries.reject_end_file = - debugfs_create_ul("reject-end", mode, dir, &attr->reject_end); - - if (!attr->dentries.stacktrace_depth_file || - !attr->dentries.require_start_file || - !attr->dentries.require_end_file || - !attr->dentries.reject_start_file || - !attr->dentries.reject_end_file) + if (!debugfs_create_stacktrace_depth("stacktrace-depth", mode, dir, + &attr->stacktrace_depth)) + goto fail; + if (!debugfs_create_ul("require-start", mode, dir, + &attr->require_start)) + goto fail; + if (!debugfs_create_ul("require-end", mode, dir, &attr->require_end)) + goto fail; + if (!debugfs_create_ul("reject-start", mode, dir, &attr->reject_start)) + goto fail; + if (!debugfs_create_ul("reject-end", mode, dir, &attr->reject_end)) goto fail; #endif /* CONFIG_FAULT_INJECTION_STACKTRACE_FILTER */ - return 0; + return dir; fail: - cleanup_fault_attr_dentries(attr); - return -ENOMEM; + debugfs_remove_recursive(dir); + + return ERR_PTR(-ENOMEM); } +EXPORT_SYMBOL_GPL(fault_create_debugfs_attr); #endif /* CONFIG_FAULT_INJECTION_DEBUG_FS */ diff --git a/lib/genalloc.c b/lib/genalloc.c index 577ddf805975..f352cc42f4f8 100644 --- a/lib/genalloc.c +++ b/lib/genalloc.c @@ -1,8 +1,26 @@ /* - * Basic general purpose allocator for managing special purpose memory - * not managed by the regular kmalloc/kfree interface. - * Uses for this includes on-device special memory, uncached memory - * etc. + * Basic general purpose allocator for managing special purpose + * memory, for example, memory that is not managed by the regular + * kmalloc/kfree interface. Uses for this includes on-device special + * memory, uncached memory etc. + * + * It is safe to use the allocator in NMI handlers and other special + * unblockable contexts that could otherwise deadlock on locks. This + * is implemented by using atomic operations and retries on any + * conflicts. The disadvantage is that there may be livelocks in + * extreme cases. For better scalability, one allocator can be used + * for each CPU. + * + * The lockless operation only works if there is enough memory + * available. If new memory is added to the pool a lock has to be + * still taken. So any user relying on locklessness has to ensure + * that sufficient memory is preallocated. + * + * The basic atomic operation of this allocator is cmpxchg on long. + * On architectures that don't have NMI-safe cmpxchg implementation, + * the allocator can NOT be used in NMI handler. So code uses the + * allocator in NMI handler should depend on + * CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG. * * Copyright 2005 (C) Jes Sorensen <jes@trained-monkey.org> * @@ -13,8 +31,109 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/bitmap.h> +#include <linux/rculist.h> +#include <linux/interrupt.h> #include <linux/genalloc.h> +static int set_bits_ll(unsigned long *addr, unsigned long mask_to_set) +{ + unsigned long val, nval; + + nval = *addr; + do { + val = nval; + if (val & mask_to_set) + return -EBUSY; + cpu_relax(); + } while ((nval = cmpxchg(addr, val, val | mask_to_set)) != val); + + return 0; +} + +static int clear_bits_ll(unsigned long *addr, unsigned long mask_to_clear) +{ + unsigned long val, nval; + + nval = *addr; + do { + val = nval; + if ((val & mask_to_clear) != mask_to_clear) + return -EBUSY; + cpu_relax(); + } while ((nval = cmpxchg(addr, val, val & ~mask_to_clear)) != val); + + return 0; +} + +/* + * bitmap_set_ll - set the specified number of bits at the specified position + * @map: pointer to a bitmap + * @start: a bit position in @map + * @nr: number of bits to set + * + * Set @nr bits start from @start in @map lock-lessly. Several users + * can set/clear the same bitmap simultaneously without lock. If two + * users set the same bit, one user will return remain bits, otherwise + * return 0. + */ +static int bitmap_set_ll(unsigned long *map, int start, int nr) +{ + unsigned long *p = map + BIT_WORD(start); + const int size = start + nr; + int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG); + unsigned long mask_to_set = BITMAP_FIRST_WORD_MASK(start); + + while (nr - bits_to_set >= 0) { + if (set_bits_ll(p, mask_to_set)) + return nr; + nr -= bits_to_set; + bits_to_set = BITS_PER_LONG; + mask_to_set = ~0UL; + p++; + } + if (nr) { + mask_to_set &= BITMAP_LAST_WORD_MASK(size); + if (set_bits_ll(p, mask_to_set)) + return nr; + } + + return 0; +} + +/* + * bitmap_clear_ll - clear the specified number of bits at the specified position + * @map: pointer to a bitmap + * @start: a bit position in @map + * @nr: number of bits to set + * + * Clear @nr bits start from @start in @map lock-lessly. Several users + * can set/clear the same bitmap simultaneously without lock. If two + * users clear the same bit, one user will return remain bits, + * otherwise return 0. + */ +static int bitmap_clear_ll(unsigned long *map, int start, int nr) +{ + unsigned long *p = map + BIT_WORD(start); + const int size = start + nr; + int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG); + unsigned long mask_to_clear = BITMAP_FIRST_WORD_MASK(start); + + while (nr - bits_to_clear >= 0) { + if (clear_bits_ll(p, mask_to_clear)) + return nr; + nr -= bits_to_clear; + bits_to_clear = BITS_PER_LONG; + mask_to_clear = ~0UL; + p++; + } + if (nr) { + mask_to_clear &= BITMAP_LAST_WORD_MASK(size); + if (clear_bits_ll(p, mask_to_clear)) + return nr; + } + + return 0; +} /** * gen_pool_create - create a new special memory pool @@ -30,7 +149,7 @@ struct gen_pool *gen_pool_create(int min_alloc_order, int nid) pool = kmalloc_node(sizeof(struct gen_pool), GFP_KERNEL, nid); if (pool != NULL) { - rwlock_init(&pool->lock); + spin_lock_init(&pool->lock); INIT_LIST_HEAD(&pool->chunks); pool->min_alloc_order = min_alloc_order; } @@ -63,14 +182,14 @@ int gen_pool_add_virt(struct gen_pool *pool, unsigned long virt, phys_addr_t phy if (unlikely(chunk == NULL)) return -ENOMEM; - spin_lock_init(&chunk->lock); chunk->phys_addr = phys; chunk->start_addr = virt; chunk->end_addr = virt + size; + atomic_set(&chunk->avail, size); - write_lock(&pool->lock); - list_add(&chunk->next_chunk, &pool->chunks); - write_unlock(&pool->lock); + spin_lock(&pool->lock); + list_add_rcu(&chunk->next_chunk, &pool->chunks); + spin_unlock(&pool->lock); return 0; } @@ -85,19 +204,19 @@ EXPORT_SYMBOL(gen_pool_add_virt); */ phys_addr_t gen_pool_virt_to_phys(struct gen_pool *pool, unsigned long addr) { - struct list_head *_chunk; struct gen_pool_chunk *chunk; + phys_addr_t paddr = -1; - read_lock(&pool->lock); - list_for_each(_chunk, &pool->chunks) { - chunk = list_entry(_chunk, struct gen_pool_chunk, next_chunk); - - if (addr >= chunk->start_addr && addr < chunk->end_addr) - return chunk->phys_addr + addr - chunk->start_addr; + rcu_read_lock(); + list_for_each_entry_rcu(chunk, &pool->chunks, next_chunk) { + if (addr >= chunk->start_addr && addr < chunk->end_addr) { + paddr = chunk->phys_addr + (addr - chunk->start_addr); + break; + } } - read_unlock(&pool->lock); + rcu_read_unlock(); - return -1; + return paddr; } EXPORT_SYMBOL(gen_pool_virt_to_phys); @@ -115,7 +234,6 @@ void gen_pool_destroy(struct gen_pool *pool) int order = pool->min_alloc_order; int bit, end_bit; - list_for_each_safe(_chunk, _next_chunk, &pool->chunks) { chunk = list_entry(_chunk, struct gen_pool_chunk, next_chunk); list_del(&chunk->next_chunk); @@ -137,44 +255,50 @@ EXPORT_SYMBOL(gen_pool_destroy); * @size: number of bytes to allocate from the pool * * Allocate the requested number of bytes from the specified pool. - * Uses a first-fit algorithm. + * Uses a first-fit algorithm. Can not be used in NMI handler on + * architectures without NMI-safe cmpxchg implementation. */ unsigned long gen_pool_alloc(struct gen_pool *pool, size_t size) { - struct list_head *_chunk; struct gen_pool_chunk *chunk; - unsigned long addr, flags; + unsigned long addr = 0; int order = pool->min_alloc_order; - int nbits, start_bit, end_bit; + int nbits, start_bit = 0, end_bit, remain; + +#ifndef CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG + BUG_ON(in_nmi()); +#endif if (size == 0) return 0; nbits = (size + (1UL << order) - 1) >> order; - - read_lock(&pool->lock); - list_for_each(_chunk, &pool->chunks) { - chunk = list_entry(_chunk, struct gen_pool_chunk, next_chunk); + rcu_read_lock(); + list_for_each_entry_rcu(chunk, &pool->chunks, next_chunk) { + if (size > atomic_read(&chunk->avail)) + continue; end_bit = (chunk->end_addr - chunk->start_addr) >> order; - - spin_lock_irqsave(&chunk->lock, flags); - start_bit = bitmap_find_next_zero_area(chunk->bits, end_bit, 0, - nbits, 0); - if (start_bit >= end_bit) { - spin_unlock_irqrestore(&chunk->lock, flags); +retry: + start_bit = bitmap_find_next_zero_area(chunk->bits, end_bit, + start_bit, nbits, 0); + if (start_bit >= end_bit) continue; + remain = bitmap_set_ll(chunk->bits, start_bit, nbits); + if (remain) { + remain = bitmap_clear_ll(chunk->bits, start_bit, + nbits - remain); + BUG_ON(remain); + goto retry; } addr = chunk->start_addr + ((unsigned long)start_bit << order); - - bitmap_set(chunk->bits, start_bit, nbits); - spin_unlock_irqrestore(&chunk->lock, flags); - read_unlock(&pool->lock); - return addr; + size = nbits << order; + atomic_sub(size, &chunk->avail); + break; } - read_unlock(&pool->lock); - return 0; + rcu_read_unlock(); + return addr; } EXPORT_SYMBOL(gen_pool_alloc); @@ -184,33 +308,95 @@ EXPORT_SYMBOL(gen_pool_alloc); * @addr: starting address of memory to free back to pool * @size: size in bytes of memory to free * - * Free previously allocated special memory back to the specified pool. + * Free previously allocated special memory back to the specified + * pool. Can not be used in NMI handler on architectures without + * NMI-safe cmpxchg implementation. */ void gen_pool_free(struct gen_pool *pool, unsigned long addr, size_t size) { - struct list_head *_chunk; struct gen_pool_chunk *chunk; - unsigned long flags; int order = pool->min_alloc_order; - int bit, nbits; + int start_bit, nbits, remain; - nbits = (size + (1UL << order) - 1) >> order; - - read_lock(&pool->lock); - list_for_each(_chunk, &pool->chunks) { - chunk = list_entry(_chunk, struct gen_pool_chunk, next_chunk); +#ifndef CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG + BUG_ON(in_nmi()); +#endif + nbits = (size + (1UL << order) - 1) >> order; + rcu_read_lock(); + list_for_each_entry_rcu(chunk, &pool->chunks, next_chunk) { if (addr >= chunk->start_addr && addr < chunk->end_addr) { BUG_ON(addr + size > chunk->end_addr); - spin_lock_irqsave(&chunk->lock, flags); - bit = (addr - chunk->start_addr) >> order; - while (nbits--) - __clear_bit(bit++, chunk->bits); - spin_unlock_irqrestore(&chunk->lock, flags); - break; + start_bit = (addr - chunk->start_addr) >> order; + remain = bitmap_clear_ll(chunk->bits, start_bit, nbits); + BUG_ON(remain); + size = nbits << order; + atomic_add(size, &chunk->avail); + rcu_read_unlock(); + return; } } - BUG_ON(nbits > 0); - read_unlock(&pool->lock); + rcu_read_unlock(); + BUG(); } EXPORT_SYMBOL(gen_pool_free); + +/** + * gen_pool_for_each_chunk - call func for every chunk of generic memory pool + * @pool: the generic memory pool + * @func: func to call + * @data: additional data used by @func + * + * Call @func for every chunk of generic memory pool. The @func is + * called with rcu_read_lock held. + */ +void gen_pool_for_each_chunk(struct gen_pool *pool, + void (*func)(struct gen_pool *pool, struct gen_pool_chunk *chunk, void *data), + void *data) +{ + struct gen_pool_chunk *chunk; + + rcu_read_lock(); + list_for_each_entry_rcu(chunk, &(pool)->chunks, next_chunk) + func(pool, chunk, data); + rcu_read_unlock(); +} +EXPORT_SYMBOL(gen_pool_for_each_chunk); + +/** + * gen_pool_avail - get available free space of the pool + * @pool: pool to get available free space + * + * Return available free space of the specified pool. + */ +size_t gen_pool_avail(struct gen_pool *pool) +{ + struct gen_pool_chunk *chunk; + size_t avail = 0; + + rcu_read_lock(); + list_for_each_entry_rcu(chunk, &pool->chunks, next_chunk) + avail += atomic_read(&chunk->avail); + rcu_read_unlock(); + return avail; +} +EXPORT_SYMBOL_GPL(gen_pool_avail); + +/** + * gen_pool_size - get size in bytes of memory managed by the pool + * @pool: pool to get size + * + * Return size in bytes of memory managed by the pool. + */ +size_t gen_pool_size(struct gen_pool *pool) +{ + struct gen_pool_chunk *chunk; + size_t size = 0; + + rcu_read_lock(); + list_for_each_entry_rcu(chunk, &pool->chunks, next_chunk) + size += chunk->end_addr - chunk->start_addr; + rcu_read_unlock(); + return size; +} +EXPORT_SYMBOL_GPL(gen_pool_size); diff --git a/lib/hexdump.c b/lib/hexdump.c index f5fe6ba7a3ab..51d5ae210244 100644 --- a/lib/hexdump.c +++ b/lib/hexdump.c @@ -38,14 +38,21 @@ EXPORT_SYMBOL(hex_to_bin); * @dst: binary result * @src: ascii hexadecimal string * @count: result length + * + * Return 0 on success, -1 in case of bad input. */ -void hex2bin(u8 *dst, const char *src, size_t count) +int hex2bin(u8 *dst, const char *src, size_t count) { while (count--) { - *dst = hex_to_bin(*src++) << 4; - *dst += hex_to_bin(*src++); - dst++; + int hi = hex_to_bin(*src++); + int lo = hex_to_bin(*src++); + + if ((hi < 0) || (lo < 0)) + return -1; + + *dst++ = (hi << 4) | lo; } + return 0; } EXPORT_SYMBOL(hex2bin); diff --git a/lib/idr.c b/lib/idr.c index e15502e8b21e..ed055b297c81 100644 --- a/lib/idr.c +++ b/lib/idr.c @@ -34,8 +34,10 @@ #include <linux/err.h> #include <linux/string.h> #include <linux/idr.h> +#include <linux/spinlock.h> static struct kmem_cache *idr_layer_cache; +static DEFINE_SPINLOCK(simple_ida_lock); static struct idr_layer *get_from_free_list(struct idr *idp) { @@ -765,8 +767,8 @@ EXPORT_SYMBOL(ida_pre_get); * @starting_id: id to start search at * @p_id: pointer to the allocated handle * - * Allocate new ID above or equal to @ida. It should be called with - * any required locks. + * Allocate new ID above or equal to @starting_id. It should be called + * with any required locks. * * If memory is required, it will return %-EAGAIN, you should unlock * and go back to the ida_pre_get() call. If the ida is full, it will @@ -858,7 +860,7 @@ EXPORT_SYMBOL(ida_get_new_above); * and go back to the idr_pre_get() call. If the idr is full, it will * return %-ENOSPC. * - * @id returns a value in the range %0 ... %0x7fffffff. + * @p_id returns a value in the range %0 ... %0x7fffffff. */ int ida_get_new(struct ida *ida, int *p_id) { @@ -926,6 +928,74 @@ void ida_destroy(struct ida *ida) EXPORT_SYMBOL(ida_destroy); /** + * ida_simple_get - get a new id. + * @ida: the (initialized) ida. + * @start: the minimum id (inclusive, < 0x8000000) + * @end: the maximum id (exclusive, < 0x8000000 or 0) + * @gfp_mask: memory allocation flags + * + * Allocates an id in the range start <= id < end, or returns -ENOSPC. + * On memory allocation failure, returns -ENOMEM. + * + * Use ida_simple_remove() to get rid of an id. + */ +int ida_simple_get(struct ida *ida, unsigned int start, unsigned int end, + gfp_t gfp_mask) +{ + int ret, id; + unsigned int max; + unsigned long flags; + + BUG_ON((int)start < 0); + BUG_ON((int)end < 0); + + if (end == 0) + max = 0x80000000; + else { + BUG_ON(end < start); + max = end - 1; + } + +again: + if (!ida_pre_get(ida, gfp_mask)) + return -ENOMEM; + + spin_lock_irqsave(&simple_ida_lock, flags); + ret = ida_get_new_above(ida, start, &id); + if (!ret) { + if (id > max) { + ida_remove(ida, id); + ret = -ENOSPC; + } else { + ret = id; + } + } + spin_unlock_irqrestore(&simple_ida_lock, flags); + + if (unlikely(ret == -EAGAIN)) + goto again; + + return ret; +} +EXPORT_SYMBOL(ida_simple_get); + +/** + * ida_simple_remove - remove an allocated id. + * @ida: the (initialized) ida. + * @id: the id returned by ida_simple_get. + */ +void ida_simple_remove(struct ida *ida, unsigned int id) +{ + unsigned long flags; + + BUG_ON((int)id < 0); + spin_lock_irqsave(&simple_ida_lock, flags); + ida_remove(ida, id); + spin_unlock_irqrestore(&simple_ida_lock, flags); +} +EXPORT_SYMBOL(ida_simple_remove); + +/** * ida_init - initialize ida handle * @ida: ida handle * diff --git a/lib/iomap.c b/lib/iomap.c index d32229385151..5dbcb4b2d864 100644 --- a/lib/iomap.c +++ b/lib/iomap.c @@ -224,6 +224,7 @@ EXPORT_SYMBOL(iowrite8_rep); EXPORT_SYMBOL(iowrite16_rep); EXPORT_SYMBOL(iowrite32_rep); +#ifdef CONFIG_HAS_IOPORT /* Create a virtual mapping cookie for an IO port range */ void __iomem *ioport_map(unsigned long port, unsigned int nr) { @@ -238,7 +239,9 @@ void ioport_unmap(void __iomem *addr) } EXPORT_SYMBOL(ioport_map); EXPORT_SYMBOL(ioport_unmap); +#endif /* CONFIG_HAS_IOPORT */ +#ifdef CONFIG_PCI /** * pci_iomap - create a virtual mapping cookie for a PCI BAR * @dev: PCI device that owns the BAR @@ -280,3 +283,4 @@ void pci_iounmap(struct pci_dev *dev, void __iomem * addr) } EXPORT_SYMBOL(pci_iomap); EXPORT_SYMBOL(pci_iounmap); +#endif /* CONFIG_PCI */ diff --git a/lib/kobject_uevent.c b/lib/kobject_uevent.c index 70af0a7f97c0..ad72a03ce5e9 100644 --- a/lib/kobject_uevent.c +++ b/lib/kobject_uevent.c @@ -282,7 +282,7 @@ int kobject_uevent_env(struct kobject *kobj, enum kobject_action action, kobj_bcast_filter, kobj); /* ENOBUFS should be handled in userspace */ - if (retval == -ENOBUFS) + if (retval == -ENOBUFS || retval == -ESRCH) retval = 0; } else retval = -ENOMEM; diff --git a/lib/kstrtox.c b/lib/kstrtox.c index 2dbae88090ac..7a94c8f14e29 100644 --- a/lib/kstrtox.c +++ b/lib/kstrtox.c @@ -18,31 +18,40 @@ #include <linux/module.h> #include <linux/types.h> #include <asm/uaccess.h> +#include "kstrtox.h" -static inline char _tolower(const char c) +const char *_parse_integer_fixup_radix(const char *s, unsigned int *base) { - return c | 0x20; -} - -static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res) -{ - unsigned long long acc; - int ok; - - if (base == 0) { + if (*base == 0) { if (s[0] == '0') { if (_tolower(s[1]) == 'x' && isxdigit(s[2])) - base = 16; + *base = 16; else - base = 8; + *base = 8; } else - base = 10; + *base = 10; } - if (base == 16 && s[0] == '0' && _tolower(s[1]) == 'x') + if (*base == 16 && s[0] == '0' && _tolower(s[1]) == 'x') s += 2; + return s; +} - acc = 0; - ok = 0; +/* + * Convert non-negative integer string representation in explicitly given radix + * to an integer. + * Return number of characters consumed maybe or-ed with overflow bit. + * If overflow occurs, result integer (incorrect) is still returned. + * + * Don't you dare use this function. + */ +unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long *res) +{ + unsigned int rv; + int overflow; + + *res = 0; + rv = 0; + overflow = 0; while (*s) { unsigned int val; @@ -50,23 +59,40 @@ static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res) val = *s - '0'; else if ('a' <= _tolower(*s) && _tolower(*s) <= 'f') val = _tolower(*s) - 'a' + 10; - else if (*s == '\n' && *(s + 1) == '\0') - break; else - return -EINVAL; + break; if (val >= base) - return -EINVAL; - if (acc > div_u64(ULLONG_MAX - val, base)) - return -ERANGE; - acc = acc * base + val; - ok = 1; - + break; + if (*res > div_u64(ULLONG_MAX - val, base)) + overflow = 1; + *res = *res * base + val; + rv++; s++; } - if (!ok) + if (overflow) + rv |= KSTRTOX_OVERFLOW; + return rv; +} + +static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res) +{ + unsigned long long _res; + unsigned int rv; + + s = _parse_integer_fixup_radix(s, &base); + rv = _parse_integer(s, base, &_res); + if (rv & KSTRTOX_OVERFLOW) + return -ERANGE; + rv &= ~KSTRTOX_OVERFLOW; + if (rv == 0) + return -EINVAL; + s += rv; + if (*s == '\n') + s++; + if (*s) return -EINVAL; - *res = acc; + *res = _res; return 0; } diff --git a/lib/kstrtox.h b/lib/kstrtox.h new file mode 100644 index 000000000000..f13eeeaf441d --- /dev/null +++ b/lib/kstrtox.h @@ -0,0 +1,8 @@ +#ifndef _LIB_KSTRTOX_H +#define _LIB_KSTRTOX_H + +#define KSTRTOX_OVERFLOW (1U << 31) +const char *_parse_integer_fixup_radix(const char *s, unsigned int *base); +unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long *res); + +#endif diff --git a/lib/lcm.c b/lib/lcm.c index 157cd88a6ffc..10b5cfcacf6b 100644 --- a/lib/lcm.c +++ b/lib/lcm.c @@ -1,6 +1,7 @@ #include <linux/kernel.h> #include <linux/gcd.h> #include <linux/module.h> +#include <linux/lcm.h> /* Lowest common multiple */ unsigned long lcm(unsigned long a, unsigned long b) diff --git a/lib/llist.c b/lib/llist.c new file mode 100644 index 000000000000..700cff77a387 --- /dev/null +++ b/lib/llist.c @@ -0,0 +1,89 @@ +/* + * Lock-less NULL terminated single linked list + * + * The basic atomic operation of this list is cmpxchg on long. On + * architectures that don't have NMI-safe cmpxchg implementation, the + * list can NOT be used in NMI handlers. So code that uses the list in + * an NMI handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG. + * + * Copyright 2010,2011 Intel Corp. + * Author: Huang Ying <ying.huang@intel.com> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License version + * 2 as published by the Free Software Foundation; + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/interrupt.h> +#include <linux/llist.h> + +#include <asm/system.h> + +/** + * llist_add_batch - add several linked entries in batch + * @new_first: first entry in batch to be added + * @new_last: last entry in batch to be added + * @head: the head for your lock-less list + * + * Return whether list is empty before adding. + */ +bool llist_add_batch(struct llist_node *new_first, struct llist_node *new_last, + struct llist_head *head) +{ + struct llist_node *entry, *old_entry; + + entry = head->first; + for (;;) { + old_entry = entry; + new_last->next = entry; + entry = cmpxchg(&head->first, old_entry, new_first); + if (entry == old_entry) + break; + } + + return old_entry == NULL; +} +EXPORT_SYMBOL_GPL(llist_add_batch); + +/** + * llist_del_first - delete the first entry of lock-less list + * @head: the head for your lock-less list + * + * If list is empty, return NULL, otherwise, return the first entry + * deleted, this is the newest added one. + * + * Only one llist_del_first user can be used simultaneously with + * multiple llist_add users without lock. Because otherwise + * llist_del_first, llist_add, llist_add (or llist_del_all, llist_add, + * llist_add) sequence in another user may change @head->first->next, + * but keep @head->first. If multiple consumers are needed, please + * use llist_del_all or use lock between consumers. + */ +struct llist_node *llist_del_first(struct llist_head *head) +{ + struct llist_node *entry, *old_entry, *next; + + entry = head->first; + for (;;) { + if (entry == NULL) + return NULL; + old_entry = entry; + next = entry->next; + entry = cmpxchg(&head->first, old_entry, next); + if (entry == old_entry) + break; + } + + return entry; +} +EXPORT_SYMBOL_GPL(llist_del_first); diff --git a/lib/md5.c b/lib/md5.c new file mode 100644 index 000000000000..c777180e1f2f --- /dev/null +++ b/lib/md5.c @@ -0,0 +1,95 @@ +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/cryptohash.h> + +#define F1(x, y, z) (z ^ (x & (y ^ z))) +#define F2(x, y, z) F1(z, x, y) +#define F3(x, y, z) (x ^ y ^ z) +#define F4(x, y, z) (y ^ (x | ~z)) + +#define MD5STEP(f, w, x, y, z, in, s) \ + (w += f(x, y, z) + in, w = (w<<s | w>>(32-s)) + x) + +void md5_transform(__u32 *hash, __u32 const *in) +{ + u32 a, b, c, d; + + a = hash[0]; + b = hash[1]; + c = hash[2]; + d = hash[3]; + + MD5STEP(F1, a, b, c, d, in[0] + 0xd76aa478, 7); + MD5STEP(F1, d, a, b, c, in[1] + 0xe8c7b756, 12); + MD5STEP(F1, c, d, a, b, in[2] + 0x242070db, 17); + MD5STEP(F1, b, c, d, a, in[3] + 0xc1bdceee, 22); + MD5STEP(F1, a, b, c, d, in[4] + 0xf57c0faf, 7); + MD5STEP(F1, d, a, b, c, in[5] + 0x4787c62a, 12); + MD5STEP(F1, c, d, a, b, in[6] + 0xa8304613, 17); + MD5STEP(F1, b, c, d, a, in[7] + 0xfd469501, 22); + MD5STEP(F1, a, b, c, d, in[8] + 0x698098d8, 7); + MD5STEP(F1, d, a, b, c, in[9] + 0x8b44f7af, 12); + MD5STEP(F1, c, d, a, b, in[10] + 0xffff5bb1, 17); + MD5STEP(F1, b, c, d, a, in[11] + 0x895cd7be, 22); + MD5STEP(F1, a, b, c, d, in[12] + 0x6b901122, 7); + MD5STEP(F1, d, a, b, c, in[13] + 0xfd987193, 12); + MD5STEP(F1, c, d, a, b, in[14] + 0xa679438e, 17); + MD5STEP(F1, b, c, d, a, in[15] + 0x49b40821, 22); + + MD5STEP(F2, a, b, c, d, in[1] + 0xf61e2562, 5); + MD5STEP(F2, d, a, b, c, in[6] + 0xc040b340, 9); + MD5STEP(F2, c, d, a, b, in[11] + 0x265e5a51, 14); + MD5STEP(F2, b, c, d, a, in[0] + 0xe9b6c7aa, 20); + MD5STEP(F2, a, b, c, d, in[5] + 0xd62f105d, 5); + MD5STEP(F2, d, a, b, c, in[10] + 0x02441453, 9); + MD5STEP(F2, c, d, a, b, in[15] + 0xd8a1e681, 14); + MD5STEP(F2, b, c, d, a, in[4] + 0xe7d3fbc8, 20); + MD5STEP(F2, a, b, c, d, in[9] + 0x21e1cde6, 5); + MD5STEP(F2, d, a, b, c, in[14] + 0xc33707d6, 9); + MD5STEP(F2, c, d, a, b, in[3] + 0xf4d50d87, 14); + MD5STEP(F2, b, c, d, a, in[8] + 0x455a14ed, 20); + MD5STEP(F2, a, b, c, d, in[13] + 0xa9e3e905, 5); + MD5STEP(F2, d, a, b, c, in[2] + 0xfcefa3f8, 9); + MD5STEP(F2, c, d, a, b, in[7] + 0x676f02d9, 14); + MD5STEP(F2, b, c, d, a, in[12] + 0x8d2a4c8a, 20); + + MD5STEP(F3, a, b, c, d, in[5] + 0xfffa3942, 4); + MD5STEP(F3, d, a, b, c, in[8] + 0x8771f681, 11); + MD5STEP(F3, c, d, a, b, in[11] + 0x6d9d6122, 16); + MD5STEP(F3, b, c, d, a, in[14] + 0xfde5380c, 23); + MD5STEP(F3, a, b, c, d, in[1] + 0xa4beea44, 4); + MD5STEP(F3, d, a, b, c, in[4] + 0x4bdecfa9, 11); + MD5STEP(F3, c, d, a, b, in[7] + 0xf6bb4b60, 16); + MD5STEP(F3, b, c, d, a, in[10] + 0xbebfbc70, 23); + MD5STEP(F3, a, b, c, d, in[13] + 0x289b7ec6, 4); + MD5STEP(F3, d, a, b, c, in[0] + 0xeaa127fa, 11); + MD5STEP(F3, c, d, a, b, in[3] + 0xd4ef3085, 16); + MD5STEP(F3, b, c, d, a, in[6] + 0x04881d05, 23); + MD5STEP(F3, a, b, c, d, in[9] + 0xd9d4d039, 4); + MD5STEP(F3, d, a, b, c, in[12] + 0xe6db99e5, 11); + MD5STEP(F3, c, d, a, b, in[15] + 0x1fa27cf8, 16); + MD5STEP(F3, b, c, d, a, in[2] + 0xc4ac5665, 23); + + MD5STEP(F4, a, b, c, d, in[0] + 0xf4292244, 6); + MD5STEP(F4, d, a, b, c, in[7] + 0x432aff97, 10); + MD5STEP(F4, c, d, a, b, in[14] + 0xab9423a7, 15); + MD5STEP(F4, b, c, d, a, in[5] + 0xfc93a039, 21); + MD5STEP(F4, a, b, c, d, in[12] + 0x655b59c3, 6); + MD5STEP(F4, d, a, b, c, in[3] + 0x8f0ccc92, 10); + MD5STEP(F4, c, d, a, b, in[10] + 0xffeff47d, 15); + MD5STEP(F4, b, c, d, a, in[1] + 0x85845dd1, 21); + MD5STEP(F4, a, b, c, d, in[8] + 0x6fa87e4f, 6); + MD5STEP(F4, d, a, b, c, in[15] + 0xfe2ce6e0, 10); + MD5STEP(F4, c, d, a, b, in[6] + 0xa3014314, 15); + MD5STEP(F4, b, c, d, a, in[13] + 0x4e0811a1, 21); + MD5STEP(F4, a, b, c, d, in[4] + 0xf7537e82, 6); + MD5STEP(F4, d, a, b, c, in[11] + 0xbd3af235, 10); + MD5STEP(F4, c, d, a, b, in[2] + 0x2ad7d2bb, 15); + MD5STEP(F4, b, c, d, a, in[9] + 0xeb86d391, 21); + + hash[0] += a; + hash[1] += b; + hash[2] += c; + hash[3] += d; +} +EXPORT_SYMBOL(md5_transform); diff --git a/lib/nlattr.c b/lib/nlattr.c index ac09f2226dc7..a8408b6cacdf 100644 --- a/lib/nlattr.c +++ b/lib/nlattr.c @@ -20,6 +20,7 @@ static const u16 nla_attr_minlen[NLA_TYPE_MAX+1] = { [NLA_U16] = sizeof(u16), [NLA_U32] = sizeof(u32), [NLA_U64] = sizeof(u64), + [NLA_MSECS] = sizeof(u64), [NLA_NESTED] = NLA_HDRLEN, }; diff --git a/lib/percpu_counter.c b/lib/percpu_counter.c index 28f2c33c6b53..f8a3f1a829b8 100644 --- a/lib/percpu_counter.c +++ b/lib/percpu_counter.c @@ -10,8 +10,10 @@ #include <linux/module.h> #include <linux/debugobjects.h> +#ifdef CONFIG_HOTPLUG_CPU static LIST_HEAD(percpu_counters); static DEFINE_MUTEX(percpu_counters_lock); +#endif #ifdef CONFIG_DEBUG_OBJECTS_PERCPU_COUNTER @@ -59,13 +61,13 @@ void percpu_counter_set(struct percpu_counter *fbc, s64 amount) { int cpu; - spin_lock(&fbc->lock); + raw_spin_lock(&fbc->lock); for_each_possible_cpu(cpu) { s32 *pcount = per_cpu_ptr(fbc->counters, cpu); *pcount = 0; } fbc->count = amount; - spin_unlock(&fbc->lock); + raw_spin_unlock(&fbc->lock); } EXPORT_SYMBOL(percpu_counter_set); @@ -76,10 +78,10 @@ void __percpu_counter_add(struct percpu_counter *fbc, s64 amount, s32 batch) preempt_disable(); count = __this_cpu_read(*fbc->counters) + amount; if (count >= batch || count <= -batch) { - spin_lock(&fbc->lock); + raw_spin_lock(&fbc->lock); fbc->count += count; __this_cpu_write(*fbc->counters, 0); - spin_unlock(&fbc->lock); + raw_spin_unlock(&fbc->lock); } else { __this_cpu_write(*fbc->counters, count); } @@ -96,13 +98,13 @@ s64 __percpu_counter_sum(struct percpu_counter *fbc) s64 ret; int cpu; - spin_lock(&fbc->lock); + raw_spin_lock(&fbc->lock); ret = fbc->count; for_each_online_cpu(cpu) { s32 *pcount = per_cpu_ptr(fbc->counters, cpu); ret += *pcount; } - spin_unlock(&fbc->lock); + raw_spin_unlock(&fbc->lock); return ret; } EXPORT_SYMBOL(__percpu_counter_sum); @@ -110,7 +112,7 @@ EXPORT_SYMBOL(__percpu_counter_sum); int __percpu_counter_init(struct percpu_counter *fbc, s64 amount, struct lock_class_key *key) { - spin_lock_init(&fbc->lock); + raw_spin_lock_init(&fbc->lock); lockdep_set_class(&fbc->lock, key); fbc->count = amount; fbc->counters = alloc_percpu(s32); @@ -173,11 +175,11 @@ static int __cpuinit percpu_counter_hotcpu_callback(struct notifier_block *nb, s32 *pcount; unsigned long flags; - spin_lock_irqsave(&fbc->lock, flags); + raw_spin_lock_irqsave(&fbc->lock, flags); pcount = per_cpu_ptr(fbc->counters, cpu); fbc->count += *pcount; *pcount = 0; - spin_unlock_irqrestore(&fbc->lock, flags); + raw_spin_unlock_irqrestore(&fbc->lock, flags); } mutex_unlock(&percpu_counters_lock); #endif diff --git a/lib/plist.c b/lib/plist.c index 0ae7e6431726..a0a4da489c22 100644 --- a/lib/plist.c +++ b/lib/plist.c @@ -56,11 +56,6 @@ static void plist_check_list(struct list_head *top) static void plist_check_head(struct plist_head *head) { - WARN_ON(head != &test_head && !head->rawlock && !head->spinlock); - if (head->rawlock) - WARN_ON_SMP(!raw_spin_is_locked(head->rawlock)); - if (head->spinlock) - WARN_ON_SMP(!spin_is_locked(head->spinlock)); if (!plist_head_empty(head)) plist_check_list(&plist_first(head)->prio_list); plist_check_list(&head->node_list); @@ -180,7 +175,7 @@ static int __init plist_test(void) unsigned int r = local_clock(); printk(KERN_INFO "start plist test\n"); - plist_head_init(&test_head, NULL); + plist_head_init(&test_head); for (i = 0; i < ARRAY_SIZE(test_node); i++) plist_node_init(test_node + i, 0); diff --git a/lib/proportions.c b/lib/proportions.c index d50746a79de2..05df84801b56 100644 --- a/lib/proportions.c +++ b/lib/proportions.c @@ -190,7 +190,7 @@ prop_adjust_shift(int *pl_shift, unsigned long *pl_period, int new_shift) int prop_local_init_percpu(struct prop_local_percpu *pl) { - spin_lock_init(&pl->lock); + raw_spin_lock_init(&pl->lock); pl->shift = 0; pl->period = 0; return percpu_counter_init(&pl->events, 0); @@ -226,7 +226,7 @@ void prop_norm_percpu(struct prop_global *pg, struct prop_local_percpu *pl) if (pl->period == global_period) return; - spin_lock_irqsave(&pl->lock, flags); + raw_spin_lock_irqsave(&pl->lock, flags); prop_adjust_shift(&pl->shift, &pl->period, pg->shift); /* @@ -247,7 +247,7 @@ void prop_norm_percpu(struct prop_global *pg, struct prop_local_percpu *pl) percpu_counter_set(&pl->events, 0); pl->period = global_period; - spin_unlock_irqrestore(&pl->lock, flags); + raw_spin_unlock_irqrestore(&pl->lock, flags); } /* @@ -324,7 +324,7 @@ void prop_fraction_percpu(struct prop_descriptor *pd, int prop_local_init_single(struct prop_local_single *pl) { - spin_lock_init(&pl->lock); + raw_spin_lock_init(&pl->lock); pl->shift = 0; pl->period = 0; pl->events = 0; @@ -356,7 +356,7 @@ void prop_norm_single(struct prop_global *pg, struct prop_local_single *pl) if (pl->period == global_period) return; - spin_lock_irqsave(&pl->lock, flags); + raw_spin_lock_irqsave(&pl->lock, flags); prop_adjust_shift(&pl->shift, &pl->period, pg->shift); /* * For each missed period, we half the local counter. @@ -367,7 +367,7 @@ void prop_norm_single(struct prop_global *pg, struct prop_local_single *pl) else pl->events = 0; pl->period = global_period; - spin_unlock_irqrestore(&pl->lock, flags); + raw_spin_unlock_irqrestore(&pl->lock, flags); } /* diff --git a/lib/radix-tree.c b/lib/radix-tree.c index 7ea2e033d715..d9df7454519c 100644 --- a/lib/radix-tree.c +++ b/lib/radix-tree.c @@ -576,7 +576,6 @@ int radix_tree_tag_get(struct radix_tree_root *root, { unsigned int height, shift; struct radix_tree_node *node; - int saw_unset_tag = 0; /* check the root's tag bit */ if (!root_tag_get(root, tag)) @@ -603,15 +602,10 @@ int radix_tree_tag_get(struct radix_tree_root *root, return 0; offset = (index >> shift) & RADIX_TREE_MAP_MASK; - - /* - * This is just a debug check. Later, we can bale as soon as - * we see an unset tag. - */ if (!tag_get(node, tag, offset)) - saw_unset_tag = 1; + return 0; if (height == 1) - return !!tag_get(node, tag, offset); + return 1; node = rcu_dereference_raw(node->slots[offset]); shift -= RADIX_TREE_MAP_SHIFT; height--; @@ -823,8 +817,8 @@ unsigned long radix_tree_prev_hole(struct radix_tree_root *root, EXPORT_SYMBOL(radix_tree_prev_hole); static unsigned int -__lookup(struct radix_tree_node *slot, void ***results, unsigned long index, - unsigned int max_items, unsigned long *next_index) +__lookup(struct radix_tree_node *slot, void ***results, unsigned long *indices, + unsigned long index, unsigned int max_items, unsigned long *next_index) { unsigned int nr_found = 0; unsigned int shift, height; @@ -857,12 +851,16 @@ __lookup(struct radix_tree_node *slot, void ***results, unsigned long index, /* Bottom level: grab some items */ for (i = index & RADIX_TREE_MAP_MASK; i < RADIX_TREE_MAP_SIZE; i++) { - index++; if (slot->slots[i]) { - results[nr_found++] = &(slot->slots[i]); - if (nr_found == max_items) + results[nr_found] = &(slot->slots[i]); + if (indices) + indices[nr_found] = index; + if (++nr_found == max_items) { + index++; goto out; + } } + index++; } out: *next_index = index; @@ -918,8 +916,8 @@ radix_tree_gang_lookup(struct radix_tree_root *root, void **results, if (cur_index > max_index) break; - slots_found = __lookup(node, (void ***)results + ret, cur_index, - max_items - ret, &next_index); + slots_found = __lookup(node, (void ***)results + ret, NULL, + cur_index, max_items - ret, &next_index); nr_found = 0; for (i = 0; i < slots_found; i++) { struct radix_tree_node *slot; @@ -944,6 +942,7 @@ EXPORT_SYMBOL(radix_tree_gang_lookup); * radix_tree_gang_lookup_slot - perform multiple slot lookup on radix tree * @root: radix tree root * @results: where the results of the lookup are placed + * @indices: where their indices should be placed (but usually NULL) * @first_index: start the lookup from this key * @max_items: place up to this many items at *results * @@ -958,7 +957,8 @@ EXPORT_SYMBOL(radix_tree_gang_lookup); * protection, radix_tree_deref_slot may fail requiring a retry. */ unsigned int -radix_tree_gang_lookup_slot(struct radix_tree_root *root, void ***results, +radix_tree_gang_lookup_slot(struct radix_tree_root *root, + void ***results, unsigned long *indices, unsigned long first_index, unsigned int max_items) { unsigned long max_index; @@ -974,6 +974,8 @@ radix_tree_gang_lookup_slot(struct radix_tree_root *root, void ***results, if (first_index > 0) return 0; results[0] = (void **)&root->rnode; + if (indices) + indices[0] = 0; return 1; } node = indirect_to_ptr(node); @@ -987,8 +989,9 @@ radix_tree_gang_lookup_slot(struct radix_tree_root *root, void ***results, if (cur_index > max_index) break; - slots_found = __lookup(node, results + ret, cur_index, - max_items - ret, &next_index); + slots_found = __lookup(node, results + ret, + indices ? indices + ret : NULL, + cur_index, max_items - ret, &next_index); ret += slots_found; if (next_index == 0) break; @@ -1194,6 +1197,98 @@ radix_tree_gang_lookup_tag_slot(struct radix_tree_root *root, void ***results, } EXPORT_SYMBOL(radix_tree_gang_lookup_tag_slot); +#if defined(CONFIG_SHMEM) && defined(CONFIG_SWAP) +#include <linux/sched.h> /* for cond_resched() */ + +/* + * This linear search is at present only useful to shmem_unuse_inode(). + */ +static unsigned long __locate(struct radix_tree_node *slot, void *item, + unsigned long index, unsigned long *found_index) +{ + unsigned int shift, height; + unsigned long i; + + height = slot->height; + shift = (height-1) * RADIX_TREE_MAP_SHIFT; + + for ( ; height > 1; height--) { + i = (index >> shift) & RADIX_TREE_MAP_MASK; + for (;;) { + if (slot->slots[i] != NULL) + break; + index &= ~((1UL << shift) - 1); + index += 1UL << shift; + if (index == 0) + goto out; /* 32-bit wraparound */ + i++; + if (i == RADIX_TREE_MAP_SIZE) + goto out; + } + + shift -= RADIX_TREE_MAP_SHIFT; + slot = rcu_dereference_raw(slot->slots[i]); + if (slot == NULL) + goto out; + } + + /* Bottom level: check items */ + for (i = 0; i < RADIX_TREE_MAP_SIZE; i++) { + if (slot->slots[i] == item) { + *found_index = index + i; + index = 0; + goto out; + } + } + index += RADIX_TREE_MAP_SIZE; +out: + return index; +} + +/** + * radix_tree_locate_item - search through radix tree for item + * @root: radix tree root + * @item: item to be found + * + * Returns index where item was found, or -1 if not found. + * Caller must hold no lock (since this time-consuming function needs + * to be preemptible), and must check afterwards if item is still there. + */ +unsigned long radix_tree_locate_item(struct radix_tree_root *root, void *item) +{ + struct radix_tree_node *node; + unsigned long max_index; + unsigned long cur_index = 0; + unsigned long found_index = -1; + + do { + rcu_read_lock(); + node = rcu_dereference_raw(root->rnode); + if (!radix_tree_is_indirect_ptr(node)) { + rcu_read_unlock(); + if (node == item) + found_index = 0; + break; + } + + node = indirect_to_ptr(node); + max_index = radix_tree_maxindex(node->height); + if (cur_index > max_index) + break; + + cur_index = __locate(node, item, cur_index, &found_index); + rcu_read_unlock(); + cond_resched(); + } while (cur_index != 0 && cur_index <= max_index); + + return found_index; +} +#else +unsigned long radix_tree_locate_item(struct radix_tree_root *root, void *item) +{ + return -1; +} +#endif /* CONFIG_SHMEM && CONFIG_SWAP */ /** * radix_tree_shrink - shrink height of a radix tree to minimal diff --git a/lib/raid6/algos.c b/lib/raid6/algos.c index b595f560bee7..8b02f60ffc86 100644 --- a/lib/raid6/algos.c +++ b/lib/raid6/algos.c @@ -17,6 +17,7 @@ */ #include <linux/raid/pq.h> +#include <linux/module.h> #ifndef __KERNEL__ #include <sys/mman.h> #include <stdio.h> diff --git a/lib/raid6/int.uc b/lib/raid6/int.uc index d1e276a14fab..5b50f8dfc5d2 100644 --- a/lib/raid6/int.uc +++ b/lib/raid6/int.uc @@ -11,7 +11,7 @@ * ----------------------------------------------------------------------- */ /* - * raid6int$#.c + * int$#.c * * $#-way unrolled portable integer math RAID-6 instruction set * diff --git a/lib/raid6/mktables.c b/lib/raid6/mktables.c index 3b1500843bba..8a3780902cec 100644 --- a/lib/raid6/mktables.c +++ b/lib/raid6/mktables.c @@ -60,6 +60,7 @@ int main(int argc, char *argv[]) uint8_t exptbl[256], invtbl[256]; printf("#include <linux/raid/pq.h>\n"); + printf("#include <linux/export.h>\n"); /* Compute multiplication table */ printf("\nconst u8 __attribute__((aligned(256)))\n" diff --git a/lib/raid6/recov.c b/lib/raid6/recov.c index 8590d19cf522..fe275d7b6b36 100644 --- a/lib/raid6/recov.c +++ b/lib/raid6/recov.c @@ -18,6 +18,7 @@ * the syndrome.) */ +#include <linux/export.h> #include <linux/raid/pq.h> /* Recover two failed data blocks. */ diff --git a/lib/ratelimit.c b/lib/ratelimit.c index 027a03f4c56d..c96d500577de 100644 --- a/lib/ratelimit.c +++ b/lib/ratelimit.c @@ -39,7 +39,7 @@ int ___ratelimit(struct ratelimit_state *rs, const char *func) * in addition to the one that will be printed by * the entity that is holding the lock already: */ - if (!spin_trylock_irqsave(&rs->lock, flags)) + if (!raw_spin_trylock_irqsave(&rs->lock, flags)) return 0; if (!rs->begin) @@ -60,7 +60,7 @@ int ___ratelimit(struct ratelimit_state *rs, const char *func) rs->missed++; ret = 0; } - spin_unlock_irqrestore(&rs->lock, flags); + raw_spin_unlock_irqrestore(&rs->lock, flags); return ret; } diff --git a/lib/rwsem-spinlock.c b/lib/rwsem-spinlock.c index ffc9fc7f3b05..f2393c21fe85 100644 --- a/lib/rwsem-spinlock.c +++ b/lib/rwsem-spinlock.c @@ -22,9 +22,9 @@ int rwsem_is_locked(struct rw_semaphore *sem) int ret = 1; unsigned long flags; - if (spin_trylock_irqsave(&sem->wait_lock, flags)) { + if (raw_spin_trylock_irqsave(&sem->wait_lock, flags)) { ret = (sem->activity != 0); - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); } return ret; } @@ -44,7 +44,7 @@ void __init_rwsem(struct rw_semaphore *sem, const char *name, lockdep_init_map(&sem->dep_map, name, key, 0); #endif sem->activity = 0; - spin_lock_init(&sem->wait_lock); + raw_spin_lock_init(&sem->wait_lock); INIT_LIST_HEAD(&sem->wait_list); } EXPORT_SYMBOL(__init_rwsem); @@ -145,12 +145,12 @@ void __sched __down_read(struct rw_semaphore *sem) struct task_struct *tsk; unsigned long flags; - spin_lock_irqsave(&sem->wait_lock, flags); + raw_spin_lock_irqsave(&sem->wait_lock, flags); if (sem->activity >= 0 && list_empty(&sem->wait_list)) { /* granted */ sem->activity++; - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); goto out; } @@ -165,7 +165,7 @@ void __sched __down_read(struct rw_semaphore *sem) list_add_tail(&waiter.list, &sem->wait_list); /* we don't need to touch the semaphore struct anymore */ - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); /* wait to be given the lock */ for (;;) { @@ -189,7 +189,7 @@ int __down_read_trylock(struct rw_semaphore *sem) int ret = 0; - spin_lock_irqsave(&sem->wait_lock, flags); + raw_spin_lock_irqsave(&sem->wait_lock, flags); if (sem->activity >= 0 && list_empty(&sem->wait_list)) { /* granted */ @@ -197,7 +197,7 @@ int __down_read_trylock(struct rw_semaphore *sem) ret = 1; } - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); return ret; } @@ -212,12 +212,12 @@ void __sched __down_write_nested(struct rw_semaphore *sem, int subclass) struct task_struct *tsk; unsigned long flags; - spin_lock_irqsave(&sem->wait_lock, flags); + raw_spin_lock_irqsave(&sem->wait_lock, flags); if (sem->activity == 0 && list_empty(&sem->wait_list)) { /* granted */ sem->activity = -1; - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); goto out; } @@ -232,7 +232,7 @@ void __sched __down_write_nested(struct rw_semaphore *sem, int subclass) list_add_tail(&waiter.list, &sem->wait_list); /* we don't need to touch the semaphore struct anymore */ - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); /* wait to be given the lock */ for (;;) { @@ -260,7 +260,7 @@ int __down_write_trylock(struct rw_semaphore *sem) unsigned long flags; int ret = 0; - spin_lock_irqsave(&sem->wait_lock, flags); + raw_spin_lock_irqsave(&sem->wait_lock, flags); if (sem->activity == 0 && list_empty(&sem->wait_list)) { /* granted */ @@ -268,7 +268,7 @@ int __down_write_trylock(struct rw_semaphore *sem) ret = 1; } - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); return ret; } @@ -280,12 +280,12 @@ void __up_read(struct rw_semaphore *sem) { unsigned long flags; - spin_lock_irqsave(&sem->wait_lock, flags); + raw_spin_lock_irqsave(&sem->wait_lock, flags); if (--sem->activity == 0 && !list_empty(&sem->wait_list)) sem = __rwsem_wake_one_writer(sem); - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); } /* @@ -295,13 +295,13 @@ void __up_write(struct rw_semaphore *sem) { unsigned long flags; - spin_lock_irqsave(&sem->wait_lock, flags); + raw_spin_lock_irqsave(&sem->wait_lock, flags); sem->activity = 0; if (!list_empty(&sem->wait_list)) sem = __rwsem_do_wake(sem, 1); - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); } /* @@ -312,12 +312,12 @@ void __downgrade_write(struct rw_semaphore *sem) { unsigned long flags; - spin_lock_irqsave(&sem->wait_lock, flags); + raw_spin_lock_irqsave(&sem->wait_lock, flags); sem->activity = 1; if (!list_empty(&sem->wait_list)) sem = __rwsem_do_wake(sem, 0); - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); } diff --git a/lib/rwsem.c b/lib/rwsem.c index aa7c3052261f..410aa1189b13 100644 --- a/lib/rwsem.c +++ b/lib/rwsem.c @@ -22,7 +22,7 @@ void __init_rwsem(struct rw_semaphore *sem, const char *name, lockdep_init_map(&sem->dep_map, name, key, 0); #endif sem->count = RWSEM_UNLOCKED_VALUE; - spin_lock_init(&sem->wait_lock); + raw_spin_lock_init(&sem->wait_lock); INIT_LIST_HEAD(&sem->wait_list); } @@ -180,7 +180,7 @@ rwsem_down_failed_common(struct rw_semaphore *sem, set_task_state(tsk, TASK_UNINTERRUPTIBLE); /* set up my own style of waitqueue */ - spin_lock_irq(&sem->wait_lock); + raw_spin_lock_irq(&sem->wait_lock); waiter.task = tsk; waiter.flags = flags; get_task_struct(tsk); @@ -204,7 +204,7 @@ rwsem_down_failed_common(struct rw_semaphore *sem, adjustment == -RWSEM_ACTIVE_WRITE_BIAS) sem = __rwsem_do_wake(sem, RWSEM_WAKE_READ_OWNED); - spin_unlock_irq(&sem->wait_lock); + raw_spin_unlock_irq(&sem->wait_lock); /* wait to be given the lock */ for (;;) { @@ -245,13 +245,13 @@ struct rw_semaphore *rwsem_wake(struct rw_semaphore *sem) { unsigned long flags; - spin_lock_irqsave(&sem->wait_lock, flags); + raw_spin_lock_irqsave(&sem->wait_lock, flags); /* do nothing if list empty */ if (!list_empty(&sem->wait_list)) sem = __rwsem_do_wake(sem, RWSEM_WAKE_ANY); - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); return sem; } @@ -265,13 +265,13 @@ struct rw_semaphore *rwsem_downgrade_wake(struct rw_semaphore *sem) { unsigned long flags; - spin_lock_irqsave(&sem->wait_lock, flags); + raw_spin_lock_irqsave(&sem->wait_lock, flags); /* do nothing if list empty */ if (!list_empty(&sem->wait_list)) sem = __rwsem_do_wake(sem, RWSEM_WAKE_READ_OWNED); - spin_unlock_irqrestore(&sem->wait_lock, flags); + raw_spin_unlock_irqrestore(&sem->wait_lock, flags); return sem; } diff --git a/lib/sha1.c b/lib/sha1.c index 4c45fd50e913..1de509a159c8 100644 --- a/lib/sha1.c +++ b/lib/sha1.c @@ -1,31 +1,73 @@ /* - * SHA transform algorithm, originally taken from code written by - * Peter Gutmann, and placed in the public domain. + * SHA1 routine optimized to do word accesses rather than byte accesses, + * and to avoid unnecessary copies into the context array. + * + * This was based on the git SHA1 implementation. */ #include <linux/kernel.h> #include <linux/module.h> +#include <linux/bitops.h> #include <linux/cryptohash.h> +#include <asm/unaligned.h> -/* The SHA f()-functions. */ +/* + * If you have 32 registers or more, the compiler can (and should) + * try to change the array[] accesses into registers. However, on + * machines with less than ~25 registers, that won't really work, + * and at least gcc will make an unholy mess of it. + * + * So to avoid that mess which just slows things down, we force + * the stores to memory to actually happen (we might be better off + * with a 'W(t)=(val);asm("":"+m" (W(t))' there instead, as + * suggested by Artur Skawina - that will also make gcc unable to + * try to do the silly "optimize away loads" part because it won't + * see what the value will be). + * + * Ben Herrenschmidt reports that on PPC, the C version comes close + * to the optimized asm with this (ie on PPC you don't want that + * 'volatile', since there are lots of registers). + * + * On ARM we get the best code generation by forcing a full memory barrier + * between each SHA_ROUND, otherwise gcc happily get wild with spilling and + * the stack frame size simply explode and performance goes down the drain. + */ -#define f1(x,y,z) (z ^ (x & (y ^ z))) /* x ? y : z */ -#define f2(x,y,z) (x ^ y ^ z) /* XOR */ -#define f3(x,y,z) ((x & y) + (z & (x ^ y))) /* majority */ +#ifdef CONFIG_X86 + #define setW(x, val) (*(volatile __u32 *)&W(x) = (val)) +#elif defined(CONFIG_ARM) + #define setW(x, val) do { W(x) = (val); __asm__("":::"memory"); } while (0) +#else + #define setW(x, val) (W(x) = (val)) +#endif -/* The SHA Mysterious Constants */ +/* This "rolls" over the 512-bit array */ +#define W(x) (array[(x)&15]) -#define K1 0x5A827999L /* Rounds 0-19: sqrt(2) * 2^30 */ -#define K2 0x6ED9EBA1L /* Rounds 20-39: sqrt(3) * 2^30 */ -#define K3 0x8F1BBCDCL /* Rounds 40-59: sqrt(5) * 2^30 */ -#define K4 0xCA62C1D6L /* Rounds 60-79: sqrt(10) * 2^30 */ +/* + * Where do we get the source from? The first 16 iterations get it from + * the input data, the next mix it from the 512-bit array. + */ +#define SHA_SRC(t) get_unaligned_be32((__u32 *)data + t) +#define SHA_MIX(t) rol32(W(t+13) ^ W(t+8) ^ W(t+2) ^ W(t), 1) + +#define SHA_ROUND(t, input, fn, constant, A, B, C, D, E) do { \ + __u32 TEMP = input(t); setW(t, TEMP); \ + E += TEMP + rol32(A,5) + (fn) + (constant); \ + B = ror32(B, 2); } while (0) + +#define T_0_15(t, A, B, C, D, E) SHA_ROUND(t, SHA_SRC, (((C^D)&B)^D) , 0x5a827999, A, B, C, D, E ) +#define T_16_19(t, A, B, C, D, E) SHA_ROUND(t, SHA_MIX, (((C^D)&B)^D) , 0x5a827999, A, B, C, D, E ) +#define T_20_39(t, A, B, C, D, E) SHA_ROUND(t, SHA_MIX, (B^C^D) , 0x6ed9eba1, A, B, C, D, E ) +#define T_40_59(t, A, B, C, D, E) SHA_ROUND(t, SHA_MIX, ((B&C)+(D&(B^C))) , 0x8f1bbcdc, A, B, C, D, E ) +#define T_60_79(t, A, B, C, D, E) SHA_ROUND(t, SHA_MIX, (B^C^D) , 0xca62c1d6, A, B, C, D, E ) /** * sha_transform - single block SHA1 transform * * @digest: 160 bit digest to update * @data: 512 bits of data to hash - * @W: 80 words of workspace (see note) + * @array: 16 words of workspace (see note) * * This function generates a SHA1 digest for a single 512-bit block. * Be warned, it does not handle padding and message digest, do not @@ -36,47 +78,111 @@ * to clear the workspace. This is left to the caller to avoid * unnecessary clears between chained hashing operations. */ -void sha_transform(__u32 *digest, const char *in, __u32 *W) +void sha_transform(__u32 *digest, const char *data, __u32 *array) { - __u32 a, b, c, d, e, t, i; - - for (i = 0; i < 16; i++) - W[i] = be32_to_cpu(((const __be32 *)in)[i]); - - for (i = 0; i < 64; i++) - W[i+16] = rol32(W[i+13] ^ W[i+8] ^ W[i+2] ^ W[i], 1); - - a = digest[0]; - b = digest[1]; - c = digest[2]; - d = digest[3]; - e = digest[4]; - - for (i = 0; i < 20; i++) { - t = f1(b, c, d) + K1 + rol32(a, 5) + e + W[i]; - e = d; d = c; c = rol32(b, 30); b = a; a = t; - } - - for (; i < 40; i ++) { - t = f2(b, c, d) + K2 + rol32(a, 5) + e + W[i]; - e = d; d = c; c = rol32(b, 30); b = a; a = t; - } - - for (; i < 60; i ++) { - t = f3(b, c, d) + K3 + rol32(a, 5) + e + W[i]; - e = d; d = c; c = rol32(b, 30); b = a; a = t; - } - - for (; i < 80; i ++) { - t = f2(b, c, d) + K4 + rol32(a, 5) + e + W[i]; - e = d; d = c; c = rol32(b, 30); b = a; a = t; - } - - digest[0] += a; - digest[1] += b; - digest[2] += c; - digest[3] += d; - digest[4] += e; + __u32 A, B, C, D, E; + + A = digest[0]; + B = digest[1]; + C = digest[2]; + D = digest[3]; + E = digest[4]; + + /* Round 1 - iterations 0-16 take their input from 'data' */ + T_0_15( 0, A, B, C, D, E); + T_0_15( 1, E, A, B, C, D); + T_0_15( 2, D, E, A, B, C); + T_0_15( 3, C, D, E, A, B); + T_0_15( 4, B, C, D, E, A); + T_0_15( 5, A, B, C, D, E); + T_0_15( 6, E, A, B, C, D); + T_0_15( 7, D, E, A, B, C); + T_0_15( 8, C, D, E, A, B); + T_0_15( 9, B, C, D, E, A); + T_0_15(10, A, B, C, D, E); + T_0_15(11, E, A, B, C, D); + T_0_15(12, D, E, A, B, C); + T_0_15(13, C, D, E, A, B); + T_0_15(14, B, C, D, E, A); + T_0_15(15, A, B, C, D, E); + + /* Round 1 - tail. Input from 512-bit mixing array */ + T_16_19(16, E, A, B, C, D); + T_16_19(17, D, E, A, B, C); + T_16_19(18, C, D, E, A, B); + T_16_19(19, B, C, D, E, A); + + /* Round 2 */ + T_20_39(20, A, B, C, D, E); + T_20_39(21, E, A, B, C, D); + T_20_39(22, D, E, A, B, C); + T_20_39(23, C, D, E, A, B); + T_20_39(24, B, C, D, E, A); + T_20_39(25, A, B, C, D, E); + T_20_39(26, E, A, B, C, D); + T_20_39(27, D, E, A, B, C); + T_20_39(28, C, D, E, A, B); + T_20_39(29, B, C, D, E, A); + T_20_39(30, A, B, C, D, E); + T_20_39(31, E, A, B, C, D); + T_20_39(32, D, E, A, B, C); + T_20_39(33, C, D, E, A, B); + T_20_39(34, B, C, D, E, A); + T_20_39(35, A, B, C, D, E); + T_20_39(36, E, A, B, C, D); + T_20_39(37, D, E, A, B, C); + T_20_39(38, C, D, E, A, B); + T_20_39(39, B, C, D, E, A); + + /* Round 3 */ + T_40_59(40, A, B, C, D, E); + T_40_59(41, E, A, B, C, D); + T_40_59(42, D, E, A, B, C); + T_40_59(43, C, D, E, A, B); + T_40_59(44, B, C, D, E, A); + T_40_59(45, A, B, C, D, E); + T_40_59(46, E, A, B, C, D); + T_40_59(47, D, E, A, B, C); + T_40_59(48, C, D, E, A, B); + T_40_59(49, B, C, D, E, A); + T_40_59(50, A, B, C, D, E); + T_40_59(51, E, A, B, C, D); + T_40_59(52, D, E, A, B, C); + T_40_59(53, C, D, E, A, B); + T_40_59(54, B, C, D, E, A); + T_40_59(55, A, B, C, D, E); + T_40_59(56, E, A, B, C, D); + T_40_59(57, D, E, A, B, C); + T_40_59(58, C, D, E, A, B); + T_40_59(59, B, C, D, E, A); + + /* Round 4 */ + T_60_79(60, A, B, C, D, E); + T_60_79(61, E, A, B, C, D); + T_60_79(62, D, E, A, B, C); + T_60_79(63, C, D, E, A, B); + T_60_79(64, B, C, D, E, A); + T_60_79(65, A, B, C, D, E); + T_60_79(66, E, A, B, C, D); + T_60_79(67, D, E, A, B, C); + T_60_79(68, C, D, E, A, B); + T_60_79(69, B, C, D, E, A); + T_60_79(70, A, B, C, D, E); + T_60_79(71, E, A, B, C, D); + T_60_79(72, D, E, A, B, C); + T_60_79(73, C, D, E, A, B); + T_60_79(74, B, C, D, E, A); + T_60_79(75, A, B, C, D, E); + T_60_79(76, E, A, B, C, D); + T_60_79(77, D, E, A, B, C); + T_60_79(78, C, D, E, A, B); + T_60_79(79, B, C, D, E, A); + + digest[0] += A; + digest[1] += B; + digest[2] += C; + digest[3] += D; + digest[4] += E; } EXPORT_SYMBOL(sha_transform); @@ -92,4 +198,3 @@ void sha_init(__u32 *buf) buf[3] = 0x10325476; buf[4] = 0xc3d2e1f0; } - diff --git a/lib/smp_processor_id.c b/lib/smp_processor_id.c index 4689cb073da4..503f087382a4 100644 --- a/lib/smp_processor_id.c +++ b/lib/smp_processor_id.c @@ -22,7 +22,7 @@ notrace unsigned int debug_smp_processor_id(void) * Kernel threads bound to a single CPU can safely use * smp_processor_id(): */ - if (cpumask_equal(¤t->cpus_allowed, cpumask_of(this_cpu))) + if (cpumask_equal(tsk_cpus_allowed(current), cpumask_of(this_cpu))) goto out; /* diff --git a/lib/spinlock_debug.c b/lib/spinlock_debug.c index 4755b98b6dfb..5f3eacdd6178 100644 --- a/lib/spinlock_debug.c +++ b/lib/spinlock_debug.c @@ -49,13 +49,10 @@ void __rwlock_init(rwlock_t *lock, const char *name, EXPORT_SYMBOL(__rwlock_init); -static void spin_bug(raw_spinlock_t *lock, const char *msg) +static void spin_dump(raw_spinlock_t *lock, const char *msg) { struct task_struct *owner = NULL; - if (!debug_locks_off()) - return; - if (lock->owner && lock->owner != SPINLOCK_OWNER_INIT) owner = lock->owner; printk(KERN_EMERG "BUG: spinlock %s on CPU#%d, %s/%d\n", @@ -70,6 +67,14 @@ static void spin_bug(raw_spinlock_t *lock, const char *msg) dump_stack(); } +static void spin_bug(raw_spinlock_t *lock, const char *msg) +{ + if (!debug_locks_off()) + return; + + spin_dump(lock, msg); +} + #define SPIN_BUG_ON(cond, lock, msg) if (unlikely(cond)) spin_bug(lock, msg) static inline void @@ -113,11 +118,7 @@ static void __spin_lock_debug(raw_spinlock_t *lock) /* lockup suspected: */ if (print_once) { print_once = 0; - printk(KERN_EMERG "BUG: spinlock lockup on CPU#%d, " - "%s/%d, %p\n", - raw_smp_processor_id(), current->comm, - task_pid_nr(current), lock); - dump_stack(); + spin_dump(lock, "lockup"); #ifdef CONFIG_SMP trigger_all_cpu_backtrace(); #endif diff --git a/lib/string.c b/lib/string.c index 01fad9b203e1..dc4a86341f91 100644 --- a/lib/string.c +++ b/lib/string.c @@ -360,7 +360,6 @@ char *strim(char *s) size_t size; char *end; - s = skip_spaces(s); size = strlen(s); if (!size) return s; @@ -370,7 +369,7 @@ char *strim(char *s) end--; *(end + 1) = '\0'; - return s; + return skip_spaces(s); } EXPORT_SYMBOL(strim); @@ -756,3 +755,57 @@ void *memchr(const void *s, int c, size_t n) } EXPORT_SYMBOL(memchr); #endif + +static void *check_bytes8(const u8 *start, u8 value, unsigned int bytes) +{ + while (bytes) { + if (*start != value) + return (void *)start; + start++; + bytes--; + } + return NULL; +} + +/** + * memchr_inv - Find an unmatching character in an area of memory. + * @start: The memory area + * @c: Find a character other than c + * @bytes: The size of the area. + * + * returns the address of the first character other than @c, or %NULL + * if the whole buffer contains just @c. + */ +void *memchr_inv(const void *start, int c, size_t bytes) +{ + u8 value = c; + u64 value64; + unsigned int words, prefix; + + if (bytes <= 16) + return check_bytes8(start, value, bytes); + + value64 = value | value << 8 | value << 16 | value << 24; + value64 = (value64 & 0xffffffff) | value64 << 32; + prefix = 8 - ((unsigned long)start) % 8; + + if (prefix) { + u8 *r = check_bytes8(start, value, prefix); + if (r) + return r; + start += prefix; + bytes -= prefix; + } + + words = bytes / 8; + + while (words) { + if (*(u64 *)start != value64) + return check_bytes8(start, value, 8); + start += 8; + words--; + } + + return check_bytes8(start, value, bytes % 8); +} +EXPORT_SYMBOL(memchr_inv); diff --git a/lib/vsprintf.c b/lib/vsprintf.c index 4365df31a1d5..993599e66e5a 100644 --- a/lib/vsprintf.c +++ b/lib/vsprintf.c @@ -31,20 +31,7 @@ #include <asm/div64.h> #include <asm/sections.h> /* for dereference_function_descriptor() */ -/* Works only for digits and letters, but small and fast */ -#define TOLOWER(x) ((x) | 0x20) - -static unsigned int simple_guess_base(const char *cp) -{ - if (cp[0] == '0') { - if (TOLOWER(cp[1]) == 'x' && isxdigit(cp[2])) - return 16; - else - return 8; - } else { - return 10; - } -} +#include "kstrtox.h" /** * simple_strtoull - convert a string to an unsigned long long @@ -54,23 +41,14 @@ static unsigned int simple_guess_base(const char *cp) */ unsigned long long simple_strtoull(const char *cp, char **endp, unsigned int base) { - unsigned long long result = 0; - - if (!base) - base = simple_guess_base(cp); - - if (base == 16 && cp[0] == '0' && TOLOWER(cp[1]) == 'x') - cp += 2; + unsigned long long result; + unsigned int rv; - while (isxdigit(*cp)) { - unsigned int value; + cp = _parse_integer_fixup_radix(cp, &base); + rv = _parse_integer(cp, base, &result); + /* FIXME */ + cp += (rv & ~KSTRTOX_OVERFLOW); - value = isdigit(*cp) ? *cp - '0' : TOLOWER(*cp) - 'a' + 10; - if (value >= base) - break; - result = result * base + value; - cp++; - } if (endp) *endp = (char *)cp; @@ -569,7 +547,7 @@ char *mac_address_string(char *buf, char *end, u8 *addr, } for (i = 0; i < 6; i++) { - p = pack_hex_byte(p, addr[i]); + p = hex_byte_pack(p, addr[i]); if (fmt[0] == 'M' && i != 5) *p++ = separator; } @@ -689,13 +667,13 @@ char *ip6_compressed_string(char *p, const char *addr) lo = word & 0xff; if (hi) { if (hi > 0x0f) - p = pack_hex_byte(p, hi); + p = hex_byte_pack(p, hi); else *p++ = hex_asc_lo(hi); - p = pack_hex_byte(p, lo); + p = hex_byte_pack(p, lo); } else if (lo > 0x0f) - p = pack_hex_byte(p, lo); + p = hex_byte_pack(p, lo); else *p++ = hex_asc_lo(lo); needcolon = true; @@ -717,8 +695,8 @@ char *ip6_string(char *p, const char *addr, const char *fmt) int i; for (i = 0; i < 8; i++) { - p = pack_hex_byte(p, *addr++); - p = pack_hex_byte(p, *addr++); + p = hex_byte_pack(p, *addr++); + p = hex_byte_pack(p, *addr++); if (fmt[0] == 'I' && i != 7) *p++ = ':'; } @@ -776,7 +754,7 @@ char *uuid_string(char *buf, char *end, const u8 *addr, } for (i = 0; i < 16; i++) { - p = pack_hex_byte(p, addr[index[i]]); + p = hex_byte_pack(p, addr[index[i]]); switch (i) { case 3: case 5: @@ -1036,8 +1014,8 @@ precision: qualifier: /* get the conversion qualifier */ spec->qualifier = -1; - if (*fmt == 'h' || TOLOWER(*fmt) == 'l' || - TOLOWER(*fmt) == 'z' || *fmt == 't') { + if (*fmt == 'h' || _tolower(*fmt) == 'l' || + _tolower(*fmt) == 'z' || *fmt == 't') { spec->qualifier = *fmt++; if (unlikely(spec->qualifier == *fmt)) { if (spec->qualifier == 'l') { @@ -1104,7 +1082,7 @@ qualifier: spec->type = FORMAT_TYPE_LONG; else spec->type = FORMAT_TYPE_ULONG; - } else if (TOLOWER(spec->qualifier) == 'z') { + } else if (_tolower(spec->qualifier) == 'z') { spec->type = FORMAT_TYPE_SIZE_T; } else if (spec->qualifier == 't') { spec->type = FORMAT_TYPE_PTRDIFF; @@ -1149,8 +1127,7 @@ qualifier: * %pi4 print an IPv4 address with leading zeros * %pI6 print an IPv6 address with colons * %pi6 print an IPv6 address without colons - * %pI6c print an IPv6 address as specified by - * http://tools.ietf.org/html/draft-ietf-6man-text-addr-representation-00 + * %pI6c print an IPv6 address as specified by RFC 5952 * %pU[bBlL] print a UUID/GUID in big or little endian using lower or upper * case. * %n is ignored @@ -1263,7 +1240,7 @@ int vsnprintf(char *buf, size_t size, const char *fmt, va_list args) if (qualifier == 'l') { long *ip = va_arg(args, long *); *ip = (str - buf); - } else if (TOLOWER(qualifier) == 'z') { + } else if (_tolower(qualifier) == 'z') { size_t *ip = va_arg(args, size_t *); *ip = (str - buf); } else { @@ -1550,7 +1527,7 @@ do { \ void *skip_arg; if (qualifier == 'l') skip_arg = va_arg(args, long *); - else if (TOLOWER(qualifier) == 'z') + else if (_tolower(qualifier) == 'z') skip_arg = va_arg(args, size_t *); else skip_arg = va_arg(args, int *); @@ -1856,8 +1833,8 @@ int vsscanf(const char *buf, const char *fmt, va_list args) /* get conversion qualifier */ qualifier = -1; - if (*fmt == 'h' || TOLOWER(*fmt) == 'l' || - TOLOWER(*fmt) == 'z') { + if (*fmt == 'h' || _tolower(*fmt) == 'l' || + _tolower(*fmt) == 'z') { qualifier = *fmt++; if (unlikely(qualifier == *fmt)) { if (qualifier == 'h') { diff --git a/lib/xz/xz_dec_bcj.c b/lib/xz/xz_dec_bcj.c index e51e2558ca9d..a768e6d28bbb 100644 --- a/lib/xz/xz_dec_bcj.c +++ b/lib/xz/xz_dec_bcj.c @@ -441,8 +441,12 @@ XZ_EXTERN enum xz_ret xz_dec_bcj_run(struct xz_dec_bcj *s, * next filter in the chain. Apply the BCJ filter on the new data * in the output buffer. If everything cannot be filtered, copy it * to temp and rewind the output buffer position accordingly. + * + * This needs to be always run when temp.size == 0 to handle a special + * case where the output buffer is full and the next filter has no + * more output coming but hasn't returned XZ_STREAM_END yet. */ - if (s->temp.size < b->out_size - b->out_pos) { + if (s->temp.size < b->out_size - b->out_pos || s->temp.size == 0) { out_start = b->out_pos; memcpy(b->out + b->out_pos, s->temp.buf, s->temp.size); b->out_pos += s->temp.size; @@ -465,16 +469,25 @@ XZ_EXTERN enum xz_ret xz_dec_bcj_run(struct xz_dec_bcj *s, s->temp.size = b->out_pos - out_start; b->out_pos -= s->temp.size; memcpy(s->temp.buf, b->out + b->out_pos, s->temp.size); + + /* + * If there wasn't enough input to the next filter to fill + * the output buffer with unfiltered data, there's no point + * to try decoding more data to temp. + */ + if (b->out_pos + s->temp.size < b->out_size) + return XZ_OK; } /* - * If we have unfiltered data in temp, try to fill by decoding more - * data from the next filter. Apply the BCJ filter on temp. Then we - * hopefully can fill the actual output buffer by copying filtered - * data from temp. A mix of filtered and unfiltered data may be left - * in temp; it will be taken care on the next call to this function. + * We have unfiltered data in temp. If the output buffer isn't full + * yet, try to fill the temp buffer by decoding more data from the + * next filter. Apply the BCJ filter on temp. Then we hopefully can + * fill the actual output buffer by copying filtered data from temp. + * A mix of filtered and unfiltered data may be left in temp; it will + * be taken care on the next call to this function. */ - if (s->temp.size > 0) { + if (b->out_pos < b->out_size) { /* Make b->out{,_pos,_size} temporarily point to s->temp. */ s->out = b->out; s->out_pos = b->out_pos; diff --git a/lib/xz/xz_private.h b/lib/xz/xz_private.h index a65633e06962..482b90f363fe 100644 --- a/lib/xz/xz_private.h +++ b/lib/xz/xz_private.h @@ -12,7 +12,7 @@ #ifdef __KERNEL__ # include <linux/xz.h> -# include <asm/byteorder.h> +# include <linux/kernel.h> # include <asm/unaligned.h> /* XZ_PREBOOT may be defined only via decompress_unxz.c. */ # ifndef XZ_PREBOOT |