summaryrefslogtreecommitdiffstats
path: root/arch/arm/lib/bitops.h
Commit message (Collapse)AuthorAgeFilesLines
* ARM: convert all "mov.* pc, reg" to "bx reg" for ARMv6+Russell King2014-07-181-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ARMv6 and greater introduced a new instruction ("bx") which can be used to return from function calls. Recent CPUs perform better when the "bx lr" instruction is used rather than the "mov pc, lr" instruction, and this sequence is strongly recommended to be used by the ARM architecture manual (section A.4.1.1). We provide a new macro "ret" with all its variants for the condition code which will resolve to the appropriate instruction. Rather than doing this piecemeal, and miss some instances, change all the "mov pc" instances to use the new macro, with the exception of the "movs" instruction and the kprobes code. This allows us to detect the "mov pc, lr" case and fix it up - and also gives us the possibility of deploying this for other registers depending on the CPU selection. Reported-by: Will Deacon <will.deacon@arm.com> Tested-by: Stephen Warren <swarren@nvidia.com> # Tegra Jetson TK1 Tested-by: Robert Jarzmik <robert.jarzmik@free.fr> # mioa701_bootresume.S Tested-by: Andrew Lunn <andrew@lunn.ch> # Kirkwood Tested-by: Shawn Guo <shawn.guo@freescale.com> Tested-by: Tony Lindgren <tony@atomide.com> # OMAPs Tested-by: Gregory CLEMENT <gregory.clement@free-electrons.com> # Armada XP, 375, 385 Acked-by: Sekhar Nori <nsekhar@ti.com> # DaVinci Acked-by: Christoffer Dall <christoffer.dall@linaro.org> # kvm/hyp Acked-by: Haojian Zhuang <haojian.zhuang@gmail.com> # PXA3xx Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> # Xen Tested-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> # ARMv7M Tested-by: Simon Horman <horms+renesas@verge.net.au> # Shmobile Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: 7984/1: prefetch: add prefetchw invocations for barriered atomicsWill Deacon2014-02-251-0/+5
| | | | | | | | | | | | | After a bunch of benchmarking on the interaction between dmb and pldw, it turns out that issuing the pldw *after* the dmb instruction can give modest performance gains (~3% atomic_add_return improvement on a dual A15). This patch adds prefetchw invocations to our barriered atomic operations including cmpxchg, test_and_xxx and futexes. Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: 7893/1: bitops: only emit .arch_extension mp if CONFIG_SMPWill Deacon2013-11-201-1/+1
| | | | | | | | | | | | | | | | | | | | Uwe reported a build failure when targetting a NOMMU platform with my recent prefetch changes: arch/arm/lib/changebit.S: Assembler messages: arch/arm/lib/changebit.S:15: Error: architectural extension `mp' is not allowed for the current base architecture This is due to use of the .arch_extension mp directive immediately prior to an ALT_SMP(...) instruction. Whilst the ALT_SMP macro will expand to nothing if !CONFIG_SMP, gas will still choke on the directive. This patch fixes the issue by only emitting the sequence (including the directive) if CONFIG_SMP=y. Tested-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: bitops: prefetch the destination word for write prior to strexWill Deacon2013-09-301-0/+5
| | | | | | | | | | | | | | The cost of changing a cacheline from shared to exclusive state can be significant, especially when this is triggered by an exclusive store, since it may result in having to retry the transaction. This patch prefixes our atomic bitops implementation with prefetchw, to try and grab the line in exclusive state from the start. The testop macro is left alone, since the barrier semantics limit the usefulness of prefetching data. Acked-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com>
* ARM: 7171/1: unwind: add unwind directives to bitops assembly macrosWill Deacon2011-11-261-4/+22
| | | | | | | | | | | | | | | The bitops functions (e.g. _test_and_set_bit) on ARM do not have unwind annotations and therefore the kernel cannot backtrace out of them on a fatal error (for example, NULL pointer dereference). This patch annotates the bitops assembly macros with UNWIND annotations so that we can produce a meaningful backtrace on error. Callers of the macros are modified to pass their function name as a macro parameter, enforcing that the macros are used as standalone function implementations. Acked-by: Dave Martin <dave.martin@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: 6653/1: bitops: Use BX instead of MOV PC,LRDave Martin2011-02-191-2/+2
| | | | | | | | | | | | | The kernel doesn't officially need to interwork, but using BX wherever appropriate will help educate people into good assembler coding habits. BX is appropriate here because this code is predicated on __LINUX_ARM_ARCH__ >= 6 Signed-off-by: Dave Martin <dave.martin@linaro.org> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: bitops: switch set/clear/change bitops to use ldrex/strexRussell King2011-02-021-18/+20
| | | | | | | | | | | | | | | | | | Switch the set/clear/change bitops to use the word-based exclusive operations, which are only present in a wider range of ARM architectures than the byte-based exclusive operations. Tested record: - Nicolas Pitre: ext3,rw,le - Sourav Poddar: nfs,le - Will Deacon: ext3,rw,le - Tony Lindgren: ext3+nfs,le Reviewed-by: Nicolas Pitre <nicolas.pitre@linaro.org> Tested-by: Sourav Poddar <sourav.poddar@ti.com> Tested-by: Will Deacon <will.deacon@arm.com> Tested-by: Tony Lindgren <tony@atomide.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: bitops: ensure set/clear/change bitops take a word-aligned pointerRussell King2011-02-021-0/+8
| | | | | | | | | Add additional instructions to our assembly bitops functions to ensure that they only operate on word-aligned pointers. This will be necessary when we switch these operations to use the word-based exclusive operations. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* Complete irq tracing support for ARMUwe Kleine-König2009-08-131-1/+1
| | | | | | | | | | | | | | | Before this patch enabling and disabling irqs in assembler code and by the hardware wasn't tracked completly. I had to transpose two instructions in arch/arm/lib/bitops.h because restore_irqs doesn't preserve the flags with CONFIG_TRACE_IRQFLAGS=y Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Cc: Russell King <linux@arm.linux.org.uk> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Ingo Molnar <mingo@redhat.com> Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
* [ARM] barriers: improve xchg, bitops and atomic SMP barriersRussell King2009-05-281-0/+2
| | | | | | | | | | | | | | | | | Mathieu Desnoyers pointed out that the ARM barriers were lacking: - cmpxchg, xchg and atomic add return need memory barriers on architectures which can reorder the relative order in which memory read/writes can be seen between CPUs, which seems to include recent ARM architectures. Those barriers are currently missing on ARM. - test_and_xxx_bit were missing SMP barriers. So put these barriers in. Provide separate atomic_add/atomic_sub operations which do not require barriers. Reported-Reviewed-and-Acked-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] spelling fixesSimon Arlott2007-05-201-1/+1
| | | | | | | Spelling fixes in arch/arm/. Signed-off-by: Simon Arlott <simon@fire.lp0.eu> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* Remove obsolete #include <linux/config.h>Jörn Engel2006-06-301-1/+0
| | | | | Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de> Signed-off-by: Adrian Bunk <bunk@stusta.de>
* [ARM] Clean up save_and_disable_irqs macro and allow use of ARMv6 CPSIDRussell King2005-11-091-2/+2
| | | | | | | | | save_and_disable_irqs does not need to use mov + msr (which was introduced to work around a documentation bug which was propagated into binutils.) Use msr with an immediate constant, and if we're building for ARMv6 or later, use the new CPSID instruction. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM SMP] Add configuration option for ARMv6K processorsRussell King2005-11-031-1/+1
| | | | | | | The 'K' extension adds several new instructions to the ARMv6 ISA which are primerily useful for SMP. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM SMP] Only enable V6K instructions on V6 MP core CPUsRussell King2005-08-101-1/+3
| | | | Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM SMP] Fix another ARMv6 bitop problemRussell King2005-07-281-1/+1
| | | | | | | | We sometimes forgot to check whether the exclusive store succeeded. Ensure that we always check. Also ensure that we always use the out of line versions, since the inline versions are not SMP safe. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM SMP] Fix data corruption in test_* bitopsRussell King2005-07-271-3/+3
| | | | | | | If we found that the bit was already in the desired state, we would skip performing the operation, and write random data back. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [PATCH] ARM: Convert bitops to use ARMv6 ldrex/strex instructionsRussell King2005-07-161-0/+31
| | | | Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [PATCH] ARM: Add missing new file for bitops patchRussell King2005-04-181-0/+33
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-07 21:44:18 +0000