summaryrefslogtreecommitdiffstats
path: root/arch/mips/include/asm/cmpxchg.h
Commit message (Collapse)AuthorAgeFilesLines
* MIPS: cmpxchg: Rearrange __xchg() arguments to match xchg()Paul Burton2017-06-291-2/+3
| | | | | | | | | | | The __xchg() function declares its first 2 arguments in reverse order compared to the xchg() macro, which is confusing & serves no purpose. Reorder the arguments such that __xchg() & xchg() match. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16356/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg: Implement 1 byte & 2 byte cmpxchg()Paul Burton2017-06-291-0/+7
| | | | | | | | | | | | | | Implement support for 1 & 2 byte cmpxchg() using read-modify-write atop a 4 byte cmpxchg(). This allows us to support these atomic operations despite the MIPS ISA only providing 4 & 8 byte atomic operations. This is required in order to support queued rwlocks (qrwlock) in a later patch, since these make use of a 1 byte cmpxchg() in their slow path. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16355/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg: Implement 1 byte & 2 byte xchg()Paul Burton2017-06-291-2/+7
| | | | | | | | | | | | | | Implement 1 & 2 byte xchg() using read-modify-write atop a 4 byte cmpxchg(). This allows us to support these atomic operations despite the MIPS ISA only providing for 4 & 8 byte atomic operations. This is required in order to support queued spinlocks (qspinlock) in a later patch, since these make use of a 2 byte xchg() in their slow path. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16354/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg: Implement __cmpxchg() as a functionPaul Burton2017-06-291-27/+32
| | | | | | | | | | | Replace the macro definition of __cmpxchg() with an inline function, which is easier to read & modify. The cmpxchg() & cmpxchg_local() macros are adjusted to call the new __cmpxchg() function. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16353/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg: Drop __xchg_u{32,64} functionsPaul Burton2017-06-291-31/+17
| | | | | | | | | | | | | | | | | | The __xchg_u32() & __xchg_u64() functions now add very little value. This patch therefore removes them, by: - Moving memory barriers out of them & into xchg(), which also removes the duplication & readies us to support xchg_relaxed() if we wish to. - Calling __xchg_asm() directly from __xchg(). - Performing the check for CONFIG_64BIT being enabled in the size=8 case of __xchg(). Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16352/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg: Error out on unsupported xchg() callsPaul Burton2017-06-291-15/+17
| | | | | | | | | | | | | | | | | xchg() has up until now simply returned the x parameter in cases where it is called with a pointer to a value of an unsupported size. This will often cause the calling code to hit a failure path, presuming that the value of x differs from the content of the memory pointed at by ptr, but we can do better by producing a compile-time or link-time error such that unsupported calls to xchg() are detectable earlier than runtime. This patch does this in the same was as is already done for cmpxchg(), using a call to a missing function annotated with __compiletime_error(). Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16351/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg: Use __compiletime_error() for bad cmpxchg() pointersPaul Burton2017-06-291-3/+10
| | | | | | | | | | | | | | | | | | | Our cmpxchg() implementation relies upon generating a call to a function which doesn't really exist (__cmpxchg_called_with_bad_pointer) to create a link failure in cases where cmpxchg() is called with a pointer to a value of an unsupported size. The __compiletime_error macro can be used to decorate a function such that a call to it generates a compile-time, rather than a link-time, error. This patch uses __compiletime_error to cause bad cmpxchg() calls to error out at compile time rather than link time, allowing errors to occur more quickly & making it easier to spot where the problem comes from. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16350/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg: Pull xchg() asm into a macroPaul Burton2017-06-291-48/+33
| | | | | | | | | | | | Use a macro to generate the 32 & 64 bit variants of the backing code for xchg(), much as is already done for cmpxchg(). This removes the duplication that could previously be found in __xchg_u32() & __xchg_u64(). Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16349/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg: Unify R10000_LLSC_WAR & non-R10000_LLSC_WAR casesPaul Burton2017-06-291-58/+22
| | | | | | | | | | | | | | | | | | | | | | Prior to this patch the xchg & cmpxchg functions have duplicated code which is for all intents & purposes identical apart from use of a branch-likely instruction in the R10000_LLSC_WAR case & a regular branch instruction in the non-R10000_LLSC_WAR case. This patch removes the duplication, declaring a __scbeqz macro to select the branch instruction suitable for use when checking the result of an sc instruction & making use of it to unify the 2 cases. In __xchg_u{32,64}() this means writing the branch in asm, where it was previously being done in C as a do...while loop for the non-R10000_LLSC_WAR case. As this is a single instruction, and adds consistency with the R10000_LLSC_WAR cases & the cmpxchg() code, this seems worthwhile. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16348/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* arch: Remove __ARCH_HAVE_CMPXCHGThomas Gleixner2015-05-131-2/+0
| | | | | | | | We removed the only user of this define in the rtmutex code. Get rid of it. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
* MIPS: Fall back to generic implementation of cmpxchg64 on 32-bit platformsDeng-Cheng Zhu2015-04-011-5/+6
| | | | | | | | | | | | | | | | | | This is in preparation of adding HAVE_VIRT_CPU_ACCOUNTING_GEN support in the next patch. Without having cmpxchg64 to use the generic implementation, kernel linking will complain: kernel/built-in.o: In function `cputime_adjust': cputime.c:(.text+0x33748): undefined reference to `__cmpxchg_called_with_bad_pointer' cputime.c:(.text+0x33810): undefined reference to `__cmpxchg_called_with_bad_pointer' Signed-off-by: Deng-Cheng Zhu <dengcheng.zhu@imgtec.com> Cc: linux-mips@linux-mips.org Cc: macro@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9474/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: asm: cmpxchg: Update ISA constraints for MIPS R6 supportMarkos Chandras2015-02-171-5/+5
| | | | | | | | MIPS R6 changed the opcodes for LL/SC instructions so we need to set the correct ISA. Cc: Matthew Fortune <Matthew.Fortune@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
* MIPS: asm: Rename GCC_OFF12_ASM to GCC_OFF_SMALL_ASMMarkos Chandras2015-02-171-12/+12
| | | | | | | | | The GCC_OFF12_ASM macro is used for 12-bit immediate constrains but we will also use it for 9-bit constrains on MIPS R6 so we rename it to something more appropriate. Cc: Maciej W. Rozycki <macro@linux-mips.org> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
* MIPS: Fix microMIPS LL/SC immediate offsetsMaciej W. Rozycki2014-11-241-12/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the microMIPS encoding some memory access instructions have their immediate offset reduced to 12 bits only. That does not match the GCC `R' constraint we use in some places to satisfy the requirement, resulting in build failures like this: {standard input}: Assembler messages: {standard input}:720: Error: macro used $at after ".set noat" {standard input}:720: Warning: macro instruction expanded into multiple instructions Fix the problem by defining a macro, `GCC_OFF12_ASM', that expands to the right constraint depending on whether microMIPS or standard MIPS code is produced. Also apply the fix to where `m' is used as in the worst case this change does nothing, e.g. where the pointer was already in a register such as a function argument and no further offset was requested, and in the best case it avoids an extraneous sequence of up to two instructions to load the high 20 bits of the address in the LL/SC loop. This reduces the risk of lock contention that is the higher the more instructions there are in the critical section between LL and SC. Strictly speaking we could just bulk-replace `R' with `ZC' as the latter constraint adjusts automatically depending on the ISA selected. However it was only introduced with GCC 4.9 and we keep supporing older compilers for the standard MIPS configuration, hence the slightly more complicated approach I chose. The choice of a zero-argument function-like rather than an object-like macro was made so that it does not look like a function call taking the C expression used for the constraint as an argument. This is so as not to confuse the reader or formatting checkers like `checkpatch.pl' and follows previous practice. Signed-off-by: Maciej W. Rozycki <macro@codesourcery.com> Signed-off-by: Steven J. Hill <Steven.Hill@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/8482/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: Fix gigaton of warning building with microMIPS.Ralf Baechle2014-03-311-10/+10
| | | | | | | | | | | | | | | | With binutils 2.24 the attempt to switch with microMIPS mode to MIPS III mode through .set mips3 results in *lots* of warnings like {standard input}: Assembler messages: {standard input}:397: Warning: the 64-bit MIPS architecture does not support the `smartmips' extension during a kernel build. Fixed by using .set arch=r4000 instead. This breaks support for building the kernel with binutils 2.13 which was supported for 32 bit kernels only anyway and 2.14 which was a bad vintage for MIPS anyway. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: Whitespace cleanup.Ralf Baechle2013-02-011-3/+3
| | | | | | | | Having received another series of whitespace patches I decided to do this once and for all rather than dealing with this kind of patches trickling in forever. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg.h: Add missing includeAaro Koskinen2012-07-191-0/+1
| | | | | | | | | | | | | | | | Fix the following build breakage in v3.4-rc1: CC kernel/irq_work.o In file included from include/linux/irq_work.h:4:0, from kernel/irq_work.c:10: include/linux/llist.h: In function 'llist_del_all': include/linux/llist.h:178:2: error: implicit declaration of function 'BUILD_BUG_ON' [-Werror=implicit-function-declaration] Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/3568/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* Disintegrate asm/system.h for MIPSDavid Howells2012-03-281-0/+124
| | | | | | | | Disintegrate asm/system.h for MIPS. Signed-off-by: David Howells <dhowells@redhat.com> Acked-by: Ralf Baechle <ralf@linux-mips.org> cc: linux-mips@linux-mips.org
* MIPS: Get rid of branches to .subsections.Ralf Baechle2010-10-291-5/+2
| | | | | | | | | | | | | | | | | | | | | | | | It was a nice optimization - on paper at least. In practice it results in branches that may exceed the maximum legal range for a branch. We can fight that problem with -ffunction-sections but -ffunction-sections again is incompatible with -pg used by the function tracer. By rewriting the loop around all simple LL/SC blocks to C we reduce the amount of inline assembler and at the same time allow GCC to often fill the branch delay slots with something sensible or whatever else clever optimization it may have up in its sleeve. With this optimization gone we also no longer need -ffunction-sections, so drop it. This optimization was originally introduced in 2.6.21, commit 5999eca25c1fd4b9b9aca7833b04d10fe4bc877d (linux-mips.org) rsp. f65e4fa8e0c6022ad58dc88d1b11b12589ed7f9f (kernel.org). Original fix for the issues which caused me to pull this optimization by Paul Gortmaker <paul.gortmaker@windriver.com>. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: cmpxchg.h: Fix excessive indentation.Ralf Baechle2010-04-301-1/+1
| | | | Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: New macro smp_mb__before_llsc.David Daney2010-02-271-5/+5
| | | | | | | | | | | | | | | | | Replace some instances of smp_llsc_mb() with a new macro smp_mb__before_llsc(). It is used before ll/sc sequences that are documented as needing write barrier semantics. The default implementation of smp_mb__before_llsc() is just smp_llsc_mb(), so there are no changes in semantics. Also simplify definition of smp_mb(), smp_rmb(), and smp_wmb() to be just barrier() in the non-SMP case. Signed-off-by: David Daney <ddaney@caviumnetworks.com> To: linux-mips@linux-mips.org Patchwork: http://patchwork.linux-mips.org/patch/851/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: Allow kernel use of LL/SC to be separate from the presence of LL/SC.David Daney2009-09-171-2/+2
| | | | | | | | | | | | | | | | | On some CPUs, it is more efficient to disable and enable interrupts in the kernel rather than use ll/sc for atomic operations. But if we were to set cpu_has_llsc to false, we would break the userspace futex interface (in asm/futex.h). We separate the two concepts, with a new predicate kernel_uses_llsc, that lets us disable the kernel's use of ll/sc while still allowing the futex code to use it. Also there were a couple of cases in bitops.h where we were using ll/sc unconditionally even if cpu_has_llsc were false. Signed-off-by: David Daney <ddaney@caviumnetworks.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: Move headfiles to new location below arch/mips/includeRalf Baechle2008-10-111-0/+124
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-02-07 19:48:11 +0000