summaryrefslogtreecommitdiffstats
path: root/drivers/dax
Commit message (Collapse)AuthorAgeFilesLines
* device-dax: fix sysfs duplicate warningsDan Williams2017-08-063-14/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit bbb3be170ac2891526ad07b18af7db226879a8e7 upstream. Fix warnings of the form... WARNING: CPU: 10 PID: 4983 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x62/0x80 sysfs: cannot create duplicate filename '/class/dax/dax12.0' Call Trace: dump_stack+0x63/0x86 __warn+0xcb/0xf0 warn_slowpath_fmt+0x5a/0x80 ? kernfs_path_from_node+0x4f/0x60 sysfs_warn_dup+0x62/0x80 sysfs_do_create_link_sd.isra.2+0x97/0xb0 sysfs_create_link+0x25/0x40 device_add+0x266/0x630 devm_create_dax_dev+0x2cf/0x340 [dax] dax_pmem_probe+0x1f5/0x26e [dax_pmem] nvdimm_bus_probe+0x71/0x120 ...by reusing the namespace id for the device-dax instance name. Now that we have decided that there will never by more than one device-dax instance per libnvdimm-namespace parent device [1], we can directly reuse the namepace ids. There are some possible follow-on cleanups, but those are saved for a later patch to simplify the -stable backport. [1]: https://lists.01.org/pipermail/linux-nvdimm/2016-December/008266.html Fixes: 98a29c39dc68 ("libnvdimm, namespace: allow creation of multiple pmem...") Cc: Jeff Moyer <jmoyer@redhat.com> Reported-by: Dariusz Dokupil <dariusz.dokupil@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* device-dax: fix cdev leakDan Williams2017-05-201-4/+11
| | | | | | | | | | | | | | | | | | | | | commit ed01e50acdd3e4a640cf9ebd28a7e810c3ceca97 upstream. If device_add() fails, cleanup the cdev. Otherwise, we leak a kobj_map() with a stale device number. As Jason points out, there is a small possibility that userspace has opened and mapped the device in the time between cdev_add() and the device_add() failure. We need a new kill_dax_dev() helper to invalidate any established mappings. Fixes: ba09c01d2fa8 ("dax: convert to the cdev api") Reported-by: Jason Gunthorpe <jgunthorpe@obsidianresearch.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* device-dax: switch to srcu, fix rcu_read_lock() vs pte allocationDan Williams2017-04-272-6/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit 956a4cd2c957acf638ff29951aabaa9d8e92bbc2 upstream. The following warning triggers with a new unit test that stresses the device-dax interface. =============================== [ ERR: suspicious RCU usage. ] 4.11.0-rc4+ #1049 Tainted: G O ------------------------------- ./include/linux/rcupdate.h:521 Illegal context switch in RCU read-side critical section! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 0 2 locks held by fio/9070: #0: (&mm->mmap_sem){++++++}, at: [<ffffffff8d0739d7>] __do_page_fault+0x167/0x4f0 #1: (rcu_read_lock){......}, at: [<ffffffffc03fbd02>] dax_dev_huge_fault+0x32/0x620 [dax] Call Trace: dump_stack+0x86/0xc3 lockdep_rcu_suspicious+0xd7/0x110 ___might_sleep+0xac/0x250 __might_sleep+0x4a/0x80 __alloc_pages_nodemask+0x23a/0x360 alloc_pages_current+0xa1/0x1f0 pte_alloc_one+0x17/0x80 __pte_alloc+0x1e/0x120 __get_locked_pte+0x1bf/0x1d0 insert_pfn.isra.70+0x3a/0x100 ? lookup_memtype+0xa6/0xd0 vm_insert_mixed+0x64/0x90 dax_dev_huge_fault+0x520/0x620 [dax] ? dax_dev_huge_fault+0x32/0x620 [dax] dax_dev_fault+0x10/0x20 [dax] __do_fault+0x1e/0x140 __handle_mm_fault+0x9af/0x10d0 handle_mm_fault+0x16d/0x370 ? handle_mm_fault+0x47/0x370 __do_page_fault+0x28c/0x4f0 trace_do_page_fault+0x58/0x2a0 do_async_page_fault+0x1a/0xa0 async_page_fault+0x28/0x30 Inserting a page table entry may trigger an allocation while we are holding a read lock to keep the device instance alive for the duration of the fault. Use srcu for this keep-alive protection. Fixes: dee410792419 ("/dev/dax, core: file operations and dax-mmap") Signed-off-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* device-dax: fix pmd/pte fault fallback handlingDave Jiang2017-03-301-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit 0134ed4fb9e78672ee9f7b18007114404c81e63f upstream. Jeff Moyer reports: With a device dax alignment of 4KB or 2MB, I get sigbus when running the attached fio job file for the current kernel (4.11.0-rc1+). If I specify an alignment of 1GB, it works. I turned on debug output, and saw that it was failing in the huge fault code. dax dax1.0: dax_open dax dax1.0: dax_mmap dax dax1.0: dax_dev_huge_fault: fio: write (0x7f08f0a00000 - dax dax1.0: __dax_dev_pud_fault: phys_to_pgoff(0xffffffffcf60 dax dax1.0: dax_release fio config for reproduce: [global] ioengine=dev-dax direct=0 filename=/dev/dax0.0 bs=2m [write] rw=write [read] stonewall rw=read The driver fails to fallback when taking a fault that is larger than the device alignment, or handling a larger fault when a smaller mapping is already established. While we could support larger mappings for a device with a smaller alignment, that change is too large for the immediate fix. The simplest change is to force fallback until the fault size matches the alignment. Fixes: dee410792419 ("/dev/dax, core: file operations and dax-mmap") Cc: <stable@vger.kernel.org> Reported-by: Jeff Moyer <jmoyer@redhat.com> Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* device-dax: fix private mapping restriction, permit read-onlyDan Williams2016-12-061-1/+1
| | | | | | | | | | | | | | | | | | | Hugh notes in response to commit 4cb19355ea19 "device-dax: fail all private mapping attempts": "I think that is more restrictive than you intended: haven't tried, but I believe it rejects a PROT_READ, MAP_SHARED, O_RDONLY fd mmap, leaving no way to mmap /dev/dax without write permission to it." Indeed it does restrict read-only mappings, switch to checking VM_MAYSHARE, not VM_SHARED. Cc: <stable@vger.kernel.org> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Pawel Lebioda <pawel.lebioda@intel.com> Fixes: 4cb19355ea19 ("device-dax: fail all private mapping attempts") Reported-by: Hugh Dickins <hughd@google.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* device-dax: fail all private mapping attemptsDan Williams2016-11-161-2/+2
| | | | | | | | | | | | | | The device-dax implementation originally tried to be tricky and allow private read-only mappings, but in the process allowed writable MAP_PRIVATE + MAP_NORESERVE mappings. For simplicity and predictability just fail all private mapping attempts since device-dax memory is statically allocated and will never support overcommit. Cc: <stable@vger.kernel.org> Cc: Dave Hansen <dave.hansen@linux.intel.com> Fixes: dee410792419 ("/dev/dax, core: file operations and dax-mmap") Reported-by: Pawel Lebioda <pawel.lebioda@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* device-dax: check devm_nsio_enable() return valueDan Williams2016-10-281-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | If the dax_pmem driver is passed a resource that is already busy the driver probe attempt should fail with a message like the following: dax_pmem dax0.1: could not reserve region [mem 0x100000000-0x11fffffff] However, if we do not catch the error we crash for the obvious reason of accessing memory that is not mapped. BUG: unable to handle kernel paging request at ffffc90020001000 IP: [<ffffffff81496712>] __memcpy+0x12/0x20 [..] Call Trace: [<ffffffff815c4960>] ? nsio_rw_bytes+0x60/0x180 [<ffffffff815c6045>] nd_pfn_validate+0x75/0x320 [<ffffffff815c63a9>] nvdimm_setup_pfn+0xb9/0x5d0 [<ffffffff815c48ef>] ? devm_nsio_enable+0xff/0x110 [<ffffffff815cb699>] dax_pmem_probe+0x59/0x260 Cc: <stable@vger.kernel.org> Fixes: ab68f2622136 ("/dev/dax, pmem: direct access to persistent memory") Reported-by: Dave Hansen <dave.hansen@linux.intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* device-dax: fix percpu_ref_exit orderingDan Williams2016-10-271-1/+1
| | | | | | | | | | | | | | | | | | | | | We need to wait until the percpu_ref is released before exit. Otherwise, we sometimes lose the race and trigger this new warning that was added in v4.9 (commit a67823c1ed10 "percpu-refcount: init ->confirm_switch member properly"): WARNING: CPU: 0 PID: 3629 at lib/percpu-refcount.c:107 percpu_ref_exit+0x51/0x60 [..] Call Trace: [<ffffffff814bf093>] dump_stack+0x85/0xc2 [<ffffffff810b15db>] __warn+0xcb/0xf0 [<ffffffff810b170d>] warn_slowpath_null+0x1d/0x20 [<ffffffff814d70c1>] percpu_ref_exit+0x51/0x60 [<ffffffffa005706a>] dax_pmem_percpu_exit+0x1a/0x50 [dax_pmem] [<ffffffff81615f1f>] devm_action_release+0xf/0x20 Cc: <stable@vger.kernel.org> Fixes: ab68f2622136 ("/dev/dax, pmem: direct access to persistent memory") Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* nvdimm: make CONFIG_NVDIMM_DAX 'bool'Arnd Bergmann2016-10-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A bugfix just tried to address a randconfig build problem and introduced a variant of the same problem: with CONFIG_LIBNVDIMM=y and CONFIG_NVDIMM_DAX=m, the nvdimm module now fails to link: drivers/nvdimm/built-in.o: In function `to_nd_device_type': bus.c:(.text+0x1b5d): undefined reference to `is_nd_dax' drivers/nvdimm/built-in.o: In function `nd_region_notify_driver_action.constprop.2': region_devs.c:(.text+0x6b6c): undefined reference to `is_nd_dax' region_devs.c:(.text+0x6b8c): undefined reference to `to_nd_dax' drivers/nvdimm/built-in.o: In function `nd_region_probe': region.c:(.text+0x70f3): undefined reference to `nd_dax_create' drivers/nvdimm/built-in.o: In function `mode_show': namespace_devs.c:(.text+0xa196): undefined reference to `is_nd_dax' drivers/nvdimm/built-in.o: In function `nvdimm_namespace_common_probe': (.text+0xa55f): undefined reference to `is_nd_dax' drivers/nvdimm/built-in.o: In function `nvdimm_namespace_common_probe': (.text+0xa56e): undefined reference to `to_nd_dax' This reverts the earlier fix, making NVDIMM_DAX a 'bool' option again as it should be (it gets linked into the libnvdimm module). To fix the original problem, I'm adding a dependency on LIBNVDIMM to DEV_DAX_PMEM, which ensures we can't have that one built-in if the rest is a module. Fixes: 4e65e9381c7a ("/dev/dax: fix Kconfig dependency build breakage") Signed-off-by: Arnd Bergmann <arnd@arndb.de> Reviewed-by: Ross Zwisler <ross.zwisler@linux.intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* Merge branch 'for-4.9/dax' into libnvdimm-for-nextDan Williams2016-10-074-232/+362
|\
| * dax: use correct dev_t valueArnd Bergmann2016-10-071-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The dev_t variable in devm_create_dax_dev() is used before it's first set: drivers/dax/dax.c: In function 'devm_create_dax_dev': drivers/dax/dax.c:205:39: error: 'dev_t' may be used uninitialized in this function [-Werror=maybe-uninitialized] inode = iget5_locked(dax_superblock, hash_32(devt + DAXFS_MAGIC, 31), ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ drivers/dax/dax.c:688:8: note: 'dev_t' was declared here This reorders the code to how it looks correct to me. Signed-off-by: Arnd Bergmann <arnd@arndb.de> Fixes: 3bc52c45bac2 ("dax: define a unified inode/address_space for device-dax mappings") Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * dax: convert devm_create_dax_dev to PTR_ERRDan Williams2016-10-073-10/+16
| | | | | | | | | | | | | | For sub-division support we need access to the dax_dev created by devm_create_dax_dev(). Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * dax: check resource alignment at dax region/device createDan Williams2016-08-231-3/+19
| | | | | | | | | | | | | | | | All the extents of a dax-device must match the alignment of the region. Otherwise, we are unable to guarantee fault semantics of a given page size. The region must be self-consistent itself as well. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * dax: unmap/truncate on device shutdownDan Williams2016-08-231-0/+1
| | | | | | | | | | | | | | Invalidate all mappings of a device-dax instance when the device is unregistered. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * dax: define a unified inode/address_space for device-dax mappingsDan Williams2016-08-231-4/+146
| | | | | | | | | | | | | | | | In support of enabling resize / truncate of device-dax instances, define a pseudo-fs to provide a unified inode/address space for vm operations. Cc: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * dax: convert to the cdev apiDan Williams2016-08-232-41/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | A goal of the device-DAX interface is to be able to support many exclusive allocations (partitions) of performance / feature differentiated memory. This count may exceed the default minors limit of 256. As a result of switching to an embedded cdev the inode-to-dax_dev conversion is simplified, as well as reference counting which can switch to the cdev kobject lifetime. Cc: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * dax: embed a struct device in dax_devDan Williams2016-08-231-85/+45
| | | | | | | | | | | | | | | | | | The kref in dax_dev can be made redundant if the final put_device() on the device associated with the dax_dev frees the dax_dev. This can be accomplished by embedding a struct device in struct dax_dev, open coding device_create() and specifying a custom release method. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * dax: rename fops from dax_dev_ to dax_Dan Williams2016-08-231-8/+8
| | | | | | | | | | | | | | Shorten the prefix of the file operations to distinguish them from operations on the struct device associated with the dax_dev. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * dax: reorder dax_fops function definitionsDan Williams2016-08-231-169/+168
| | | | | | | | | | | | | | | | In order to convert devm_create_dax_dev() to use cdev, it will need access to dax_fops. Move dax_fops and related function definitions before devm_create_dax_dev(). Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * dax: cleanup needlessly global symbol warningsDan Williams2016-08-232-1/+2
| | | | | | | | | | | | | | | | | | drivers/dax/dax.c:75:6: warning: symbol 'dax_region_put' was not declared. drivers/dax/dax.c:95:19: warning: symbol 'alloc_dax_region' was not declared. drivers/dax/dax.c:173:5: warning: symbol 'devm_create_dax_dev' was not declared. drivers/dax/pmem.c:27:17: warning: symbol 'to_dax_pmem' was not declared. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* | dax: fix mapping size checkDan Williams2016-09-031-1/+1
| | | | | | | | | | | | | | | | pgoff_to_phys() validates that both the starting address and the length of the mapping against the resource list. We need to check for a mapping size of PMD_SIZE not PAGE_SIZE in the pmd fault path. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* | dax: fix device-dax region baseDan Williams2016-08-261-0/+3
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The data offset for a dax region needs to account for a reservation in the resource range. Otherwise, device-dax is allowing mappings directly into the memmap or device-info-block area with crash signatures like the following: BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 IP: get_zone_device_page+0x11/0x30 Call Trace: follow_devmap_pmd+0x298/0x2c0 follow_page_mask+0x275/0x530 __get_user_pages+0xe3/0x750 __gfn_to_pfn_memslot+0x1b2/0x450 [kvm] tdp_page_fault+0x130/0x280 [kvm] kvm_mmu_page_fault+0x5f/0xf0 [kvm] handle_ept_violation+0x94/0x180 [kvm_intel] vmx_handle_exit+0x1d3/0x1440 [kvm_intel] kvm_arch_vcpu_ioctl_run+0x81d/0x16a0 [kvm] kvm_vcpu_ioctl+0x33c/0x620 [kvm] do_vfs_ioctl+0xa2/0x5d0 SyS_ioctl+0x79/0x90 entry_SYSCALL_64_fastpath+0x1a/0xa4 Fixes: ab68f2622136 ("/dev/dax, pmem: direct access to persistent memory") Link: http://lkml.kernel.org/r/147205536732.1606.8994275381938837346.stgit@dwillia2-desk3.amr.corp.intel.com Signed-off-by: Dan Williams <dan.j.williams@intel.com> Reported-by: Abhilash Kumar Mulumudi <m.abhilash-kumar@hpe.com> Reported-by: Toshi Kani <toshi.kani@hpe.com> Tested-by: Toshi Kani <toshi.kani@hpe.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* dax: use devm_add_action_or_reset()Sajjan, Vikas C2016-07-062-12/+8
| | | | | | | | | | | | If devm_add_action() fails, we are explicitly calling the cleanup to free the resources allocated. Use the helper devm_add_action_or_reset() and return directly in case of error, since the cleanup function has been already called by the helper if there was any error. Reported-by: Sudip Mukherjee <sudipm.mukherjee@gmail.com> Signed-off-by: Vikas C Sajjan <vikas.cha.sajjan@hpe.com> Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* /dev/dax, core: file operations and dax-mmapDan Williams2016-05-202-0/+323
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The "Device DAX" core enables dax mappings of performance / feature differentiated memory. An open mapping or file handle keeps the backing struct device live, but new mappings are only possible while the device is enabled. Faults are handled under rcu_read_lock to synchronize with the enabled state of the device. Similar to the filesystem-dax case the backing memory may optionally have struct page entries. However, unlike fs-dax there is no support for private mappings, or mappings that are not backed by media (see use of zero-page in fs-dax). Mappings are always guaranteed to match the alignment of the dax_region. If the dax_region is configured to have a 2MB alignment, all mappings are guaranteed to be backed by a pmd entry. Contrast this determinism with the fs-dax case where pmd mappings are opportunistic. If userspace attempts to force a misaligned mapping, the driver will fail the mmap attempt. See dax_dev_check_vma() for other scenarios that are rejected, like MAP_PRIVATE mappings. Cc: Hannes Reinecke <hare@suse.de> Cc: Jeff Moyer <jmoyer@redhat.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Ross Zwisler <ross.zwisler@linux.intel.com> Acked-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* /dev/dax, pmem: direct access to persistent memoryDan Williams2016-05-205-0/+464
Device DAX is the device-centric analogue of Filesystem DAX (CONFIG_FS_DAX). It allows memory ranges to be allocated and mapped without need of an intervening file system. Device DAX is strict, precise and predictable. Specifically this interface: 1/ Guarantees fault granularity with respect to a given page size (pte, pmd, or pud) set at configuration time. 2/ Enforces deterministic behavior by being strict about what fault scenarios are supported. For example, by forcing MADV_DONTFORK semantics and omitting MAP_PRIVATE support device-dax guarantees that a mapping always behaves/performs the same once established. It is the "what you see is what you get" access mechanism to differentiated memory vs filesystem DAX which has filesystem specific implementation semantics. Persistent memory is the first target, but the mechanism is also targeted for exclusive allocations of performance differentiated memory ranges. This commit is limited to the base device driver infrastructure to associate a dax device with pmem range. Cc: Jeff Moyer <jmoyer@redhat.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Ross Zwisler <ross.zwisler@linux.intel.com> Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-20 04:18:08 +0000