summaryrefslogtreecommitdiffstats
path: root/drivers/iommu
Commit message (Collapse)AuthorAgeFilesLines
* iommu/arm-smmu-v3: Clear prior settings when updating STEsNate Watterson2017-11-151-8/+2
| | | | | | | | | | | | | [ Upstream commit 810871c57011eb3e89e6768932757f169d666cd2 ] To prevent corruption of the stage-1 context pointer field when updating STEs, rebuild the entire containing dword instead of clearing individual fields. Signed-off-by: Nate Watterson <nwatters@codeaurora.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Sasha Levin <alexander.levin@verizon.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/amd: Finish TLB flush in amd_iommu_unmap()Joerg Roedel2017-10-181-0/+1
| | | | | | | | | | | | | commit ce76353f169a6471542d999baf3d29b121dce9c0 upstream. The function only sends the flush command to the IOMMU(s), but does not wait for its completion when it returns. Fix that. Fixes: 601367d76bd1 ('x86/amd-iommu: Remove iommu_flush_domain function') Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/io-pgtable-arm: Check for leaf entry before dereferencing itOleksandr Tyshchenko2017-10-081-1/+5
| | | | | | | | | | | | | | | [ Upstream commit ed46e66cc1b3d684042f92dfa2ab15ee917b4cac ] Do a check for already installed leaf entry at the current level before dereferencing it in order to avoid walking the page table down with wrong pointer to the next level. Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com> CC: Will Deacon <will.deacon@arm.com> CC: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Sasha Levin <alexander.levin@verizon.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/exynos: Block SYSMMU while invalidating FLPD cacheMarek Szyprowski2017-10-081-1/+4
| | | | | | | | | | | | | | | [ Upstream commit 7d2aa6b814476a2e2794960f844344519246df72 ] Documentation specifies that SYSMMU should be in blocked state while performing TLB/FLPD cache invalidation, so add needed calls to sysmmu_block/unblock. Fixes: 66a7ed84b345d ("iommu/exynos: Apply workaround of caching fault page table entries") CC: stable@vger.kernel.org # v4.10+ Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Sasha Levin <alexander.levin@verizon.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/arm-smmu: Set privileged attribute to 'default' instead of 'unprivileged'Sricharan R2017-10-081-1/+1
| | | | | | | | | | | | | | | | | [ Upstream commit e19898077cfb642fe151ba22981e795c74d9e114 ] Currently the driver sets all the device transactions privileges to UNPRIVILEGED, but there are cases where the iommu masters wants to isolate privileged supervisor and unprivileged user. So don't override the privileged setting to unprivileged, instead set it to default as incoming and let it be controlled by the pagetable settings. Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Sricharan R <sricharan@codeaurora.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Sasha Levin <alexander.levin@verizon.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/amd: Enable ga_log_intr when enabling guest_modeSuravee Suthikulpanit2017-08-111-0/+1
| | | | | | | | | | | | | | | commit efe6f241602cb61466895f6816b8ea6b90f04d4e upstream. IRTE[GALogIntr] bit should set when enabling guest_mode, which enables IOMMU to generate entry in GALog when IRTE[IsRun] is not set, and send an interrupt to notify IOMMU driver. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Cc: Joerg Roedel <jroedel@suse.de> Fixes: d98de49a53e48 ('iommu/amd: Enable vAPIC interrupt remapping mode by default') Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/amd: Fix interrupt remapping when disable guest_modeSuravee Suthikulpanit2017-07-051-4/+2
| | | | | | | | | | | | | | | | | | | | commit 84a21dbdef0b96d773599c33c2afbb002198d303 upstream. Pass-through devices to VM guest can get updated IRQ affinity information via irq_set_affinity() when not running in guest mode. Currently, AMD IOMMU driver in GA mode ignores the updated information if the pass-through device is setup to use vAPIC regardless of guest_mode. This could cause invalid interrupt remapping. Also, the guest_mode bit should be set and cleared only when SVM updates posted-interrupt interrupt remapping information. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Cc: Joerg Roedel <jroedel@suse.de> Fixes: d98de49a53e48 ('iommu/amd: Enable vAPIC interrupt remapping mode by default') Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/amd: Fix incorrect error handling in amd_iommu_bind_pasid()Pan Bian2017-07-051-1/+1
| | | | | | | | | | | | | | | | commit 73dbd4a4230216b6a5540a362edceae0c9b4876b upstream. In function amd_iommu_bind_pasid(), the control flow jumps to label out_free when pasid_state->mm and mm is NULL. And mmput(mm) is called. In function mmput(mm), mm is referenced without validation. This will result in a NULL dereference bug. This patch fixes the bug. Signed-off-by: Pan Bian <bianpan2016@163.com> Fixes: f0aac63b873b ('iommu/amd: Don't hold a reference to mm_struct') Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/dma: Don't reserve PCI I/O windowsRobin Murphy2017-07-051-2/+1
| | | | | | | | | | | | | | | | commit 938f1bbe35e3a7cb07e1fa7c512e2ef8bb866bdf upstream. Even if a host controller's CPU-side MMIO windows into PCI I/O space do happen to leak into PCI memory space such that it might treat them as peer addresses, trying to reserve the corresponding I/O space addresses doesn't do anything to help solve that problem. Stop doing a silly thing. Fixes: fade1ec055dc ("iommu/dma: Avoid PCI host bridge windows") Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu: Handle default domain attach failureRobin Murphy2017-07-051-13/+24
| | | | | | | | | | | | | | | | | | | | | | | | commit 797a8b4d768c58caac58ee3e8cb36a164d1b7751 upstream. We wouldn't normally expect ops->attach_dev() to fail, but on IOMMUs with limited hardware resources, or generally misconfigured systems, it is certainly possible. We report failure correctly from the external iommu_attach_device() interface, but do not do so in iommu_group_add() when attaching to the default domain. The result of failure there is that the device, group and domain all get left in a broken, part-configured state which leads to weird errors and misbehaviour down the line when IOMMU API calls sort-of-but-don't-quite work. Check the return value of __iommu_attach_device() on the default domain, and refactor the error handling paths to cope with its failure and clean up correctly in such cases. Fixes: e39cb8a3aa98 ("iommu: Make sure a device is always attached to a domain") Reported-by: Punit Agrawal <punit.agrawal@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/vt-d: Don't over-free page table directoriesDavid Dillow2017-07-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit f7116e115acdd74bc75a4daf6492b11d43505125 upstream. dma_pte_free_level() recurses down the IOMMU page tables and frees directory pages that are entirely contained in the given PFN range. Unfortunately, it incorrectly calculates the starting address covered by the PTE under consideration, which can lead to it clearing an entry that is still in use. This occurs if we have a scatterlist with an entry that has a length greater than 1026 MB and is aligned to 2 MB for both the IOMMU and physical addresses. For example, if __domain_mapping() is asked to map a two-entry scatterlist with 2 MB and 1028 MB segments to PFN 0xffff80000, it will ask if dma_pte_free_pagetable() is asked to PFNs from 0xffff80200 to 0xffffc05ff, it will also incorrectly clear the PFNs from 0xffff80000 to 0xffff801ff because of this issue. The current code will set level_pfn to 0xffff80200, and 0xffff80200-0xffffc01ff fits inside the range being cleared. Properly setting the level_pfn for the current level under consideration catches that this PTE is outside of the range being cleared. This patch also changes the value passed into dma_pte_free_level() when it recurses. This only affects the first PTE of the range being cleared, and is handled by the existing code that ensures we start our cursor no lower than start_pfn. This was found when using dma_map_sg() to map large chunks of contiguous memory, which immediatedly led to faults on the first access of the erroneously-deleted mappings. Fixes: 3269ee0bd668 ("intel-iommu: Fix leaks in pagetable freeing") Reviewed-by: Benjamin Serebrin <serebrin@google.com> Signed-off-by: David Dillow <dillow@google.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/vt-d: Flush the IOTLB to get rid of the initial kdump mappingsKarimAllah Ahmed2017-05-251-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit f73a7eee900e95404b61408a23a1df5c5811704c upstream. Ever since commit 091d42e43d ("iommu/vt-d: Copy translation tables from old kernel") the kdump kernel copies the IOMMU context tables from the previous kernel. Each device mappings will be destroyed once the driver for the respective device takes over. This unfortunately breaks the workflow of mapping and unmapping a new context to the IOMMU. The mapping function assumes that either: 1) Unmapping did the proper IOMMU flushing and it only ever flush if the IOMMU unit supports caching invalid entries. 2) The system just booted and the initialization code took care of flushing all IOMMU caches. This assumption is not true for the kdump kernel since the context tables have been copied from the previous kernel and translations could have been cached ever since. So make sure to flush the IOTLB as well when we destroy these old copied mappings. Cc: Joerg Roedel <joro@8bytes.org> Cc: David Woodhouse <dwmw2@infradead.org> Cc: David Woodhouse <dwmw@amazon.co.uk> Cc: Anthony Liguori <aliguori@amazon.com> Signed-off-by: KarimAllah Ahmed <karahmed@amazon.de> Acked-by: David Woodhouse <dwmw@amazon.co.uk> Fixes: 091d42e43d ("iommu/vt-d: Copy translation tables from old kernel") Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/vt-d: Fix NULL pointer dereference in device_to_iommuKoos Vriezen2017-03-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit 5003ae1e735e6bfe4679d9bed6846274f322e77e upstream. The function device_to_iommu() in the Intel VT-d driver lacks a NULL-ptr check, resulting in this oops at boot on some platforms: BUG: unable to handle kernel NULL pointer dereference at 00000000000007ab IP: [<ffffffff8132234a>] device_to_iommu+0x11a/0x1a0 PGD 0 [...] Call Trace: ? find_or_alloc_domain.constprop.29+0x1a/0x300 ? dw_dma_probe+0x561/0x580 [dw_dmac_core] ? __get_valid_domain_for_dev+0x39/0x120 ? __intel_map_single+0x138/0x180 ? intel_alloc_coherent+0xb6/0x120 ? sst_hsw_dsp_init+0x173/0x420 [snd_soc_sst_haswell_pcm] ? mutex_lock+0x9/0x30 ? kernfs_add_one+0xdb/0x130 ? devres_add+0x19/0x60 ? hsw_pcm_dev_probe+0x46/0xd0 [snd_soc_sst_haswell_pcm] ? platform_drv_probe+0x30/0x90 ? driver_probe_device+0x1ed/0x2b0 ? __driver_attach+0x8f/0xa0 ? driver_probe_device+0x2b0/0x2b0 ? bus_for_each_dev+0x55/0x90 ? bus_add_driver+0x110/0x210 ? 0xffffffffa11ea000 ? driver_register+0x52/0xc0 ? 0xffffffffa11ea000 ? do_one_initcall+0x32/0x130 ? free_vmap_area_noflush+0x37/0x70 ? kmem_cache_alloc+0x88/0xd0 ? do_init_module+0x51/0x1c4 ? load_module+0x1ee9/0x2430 ? show_taint+0x20/0x20 ? kernel_read_file+0xfd/0x190 ? SyS_finit_module+0xa3/0xb0 ? do_syscall_64+0x4a/0xb0 ? entry_SYSCALL64_slow_path+0x25/0x25 Code: 78 ff ff ff 4d 85 c0 74 ee 49 8b 5a 10 0f b6 9b e0 00 00 00 41 38 98 e0 00 00 00 77 da 0f b6 eb 49 39 a8 88 00 00 00 72 ce eb 8f <41> f6 82 ab 07 00 00 04 0f 85 76 ff ff ff 0f b6 4d 08 88 0e 49 RIP [<ffffffff8132234a>] device_to_iommu+0x11a/0x1a0 RSP <ffffc90001457a78> CR2: 00000000000007ab ---[ end trace 16f974b6d58d0aad ]--- Add the missing pointer check. Fixes: 1c387188c60f53b338c20eee32db055dfe022a9b ("iommu/vt-d: Fix IOMMU lookup for SR-IOV Virtual Functions") Signed-off-by: Koos Vriezen <koos.vriezen@gmail.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/vt-d: Tylersburg isoch identity map check is done too late.Ashok Raj2017-03-121-1/+2
| | | | | | | | | | | | | | | | | | | | commit 21e722c4c8377b5bc82ad058fed12165af739c1b upstream. The check to set identity map for tylersburg is done too late. It needs to be done before the check for identity_map domain is done. To: Joerg Roedel <joro@8bytes.org> To: David Woodhouse <dwmw2@infradead.org> Cc: iommu@lists.linux-foundation.org Cc: linux-kernel@vger.kernel.org Cc: Ashok Raj <ashok.raj@intel.com> Fixes: 86080ccc22 ("iommu/vt-d: Allocate si_domain in init_dmars()") Signed-off-by: Ashok Raj <ashok.raj@intel.com> Reported-by: Yunhong Jiang <yunhong.jiang@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/vt-d: Flush old iommu caches for kdump when the device gets context mappedXunlei Pang2017-01-121-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit aec0e86172a79eb5e44aff1055bb953fe4d47c59 upstream. We met the DMAR fault both on hpsa P420i and P421 SmartArray controllers under kdump, it can be steadily reproduced on several different machines, the dmesg log is like: HP HPSA Driver (v 3.4.16-0) hpsa 0000:02:00.0: using doorbell to reset controller hpsa 0000:02:00.0: board ready after hard reset. hpsa 0000:02:00.0: Waiting for controller to respond to no-op DMAR: Setting identity map for device 0000:02:00.0 [0xe8000 - 0xe8fff] DMAR: Setting identity map for device 0000:02:00.0 [0xf4000 - 0xf4fff] DMAR: Setting identity map for device 0000:02:00.0 [0xbdf6e000 - 0xbdf6efff] DMAR: Setting identity map for device 0000:02:00.0 [0xbdf6f000 - 0xbdf7efff] DMAR: Setting identity map for device 0000:02:00.0 [0xbdf7f000 - 0xbdf82fff] DMAR: Setting identity map for device 0000:02:00.0 [0xbdf83000 - 0xbdf84fff] DMAR: DRHD: handling fault status reg 2 DMAR: [DMA Read] Request device [02:00.0] fault addr fffff000 [fault reason 06] PTE Read access is not set hpsa 0000:02:00.0: controller message 03:00 timed out hpsa 0000:02:00.0: no-op failed; re-trying After some debugging, we found that the fault addr is from DMA initiated at the driver probe stage after reset(not in-flight DMA), and the corresponding pte entry value is correct, the fault is likely due to the old iommu caches of the in-flight DMA before it. Thus we need to flush the old cache after context mapping is setup for the device, where the device is supposed to finish reset at its driver probe stage and no in-flight DMA exists hereafter. I'm not sure if the hardware is responsible for invalidating all the related caches allocated in the iommu hardware before, but seems not the case for hpsa, actually many device drivers have problems in properly resetting the hardware. Anyway flushing (again) by software in kdump kernel when the device gets context mapped which is a quite infrequent operation does little harm. With this patch, the problematic machine can survive the kdump tests. CC: Myron Stowe <myron.stowe@gmail.com> CC: Joseph Szczypek <jszczype@redhat.com> CC: Don Brace <don.brace@microsemi.com> CC: Baoquan He <bhe@redhat.com> CC: Dave Young <dyoung@redhat.com> Fixes: 091d42e43d21 ("iommu/vt-d: Copy translation tables from old kernel") Fixes: dbcd861f252d ("iommu/vt-d: Do not re-use domain-ids from the old kernel") Fixes: cf484d0e6939 ("iommu/vt-d: Mark copied context entries") Signed-off-by: Xunlei Pang <xlpang@redhat.com> Tested-by: Don Brace <don.brace@microsemi.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/vt-d: Fix pasid table size encodingJacob Pan2017-01-121-1/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | commit 65ca7f5f7d1cdde6c25172fe6107cd16902f826f upstream. Different encodings are used to represent supported PASID bits and number of PASID table entries. The current code assigns ecap_pss directly to extended context table entry PTS which is wrong and could result in writing non-zero bits to the reserved fields. IOMMU fault reason 11 will be reported when reserved bits are nonzero. This patch converts ecap_pss to extend context entry pts encoding based on VT-d spec. Chapter 9.4 as follows: - number of PASID bits = ecap_pss + 1 - number of PASID table entries = 2^(pts + 5) Software assigned limit of pasid_max value is also respected to match the allocation limitation of PASID table. cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> cc: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com> Tested-by: Mika Kuoppala <mika.kuoppala@intel.com> Fixes: 2f26e0a9c9860 ('iommu/vt-d: Add basic SVM PASID support') Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/amd: Fix the left value check of cmd bufferHuang Rui2017-01-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | commit 432abf68a79332282329286d190e21fe3ac02a31 upstream. The generic command buffer entry is 128 bits (16 bytes), so the offset of tail and head pointer should be 16 bytes aligned and increased with 0x10 per command. When cmd buf is full, head = (tail + 0x10) % CMD_BUFFER_SIZE. So when left space of cmd buf should be able to store only two command, we should be issued one COMPLETE_WAIT additionally to wait all older commands completed. Then the left space should be increased after IOMMU fetching from cmd buf. So left check value should be left <= 0x20 (two commands). Signed-off-by: Huang Rui <ray.huang@amd.com> Fixes: ac0ea6e92b222 ('x86/amd-iommu: Improve handling of full command buffer') Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu/amd: Missing error code in amd_iommu_init_device()Dan Carpenter2017-01-121-1/+3
| | | | | | | | | | | | commit 24c790fbf5d8f54c8c82979db11edea8855b74bf upstream. We should set "ret" to -EINVAL if iommu_group_get() fails. Fixes: 55c99a4dc50f ("iommu/amd: Use iommu_attach_group()") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* Merge git://git.infradead.org/intel-iommuLinus Torvalds2016-11-273-12/+33
|\ | | | | | | | | | | | | | | | | | | | | | | | | Pull IOMMU fixes from David Woodhouse: "Two minor fixes. The first fixes the assignment of SR-IOV virtual functions to the correct IOMMU unit, and the second fixes the excessively large (and physically contiguous) PASID tables used with SVM" * git://git.infradead.org/intel-iommu: iommu/vt-d: Fix PASID table allocation iommu/vt-d: Fix IOMMU lookup for SR-IOV Virtual Functions
| * iommu/vt-d: Fix PASID table allocationDavid Woodhouse2016-11-191-11/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Somehow I ended up with an off-by-three error in calculating the size of the PASID and PASID State tables, which triggers allocations failures as those tables unfortunately have to be physically contiguous. In fact, even the *correct* maximum size of 8MiB is problematic and is wont to lead to allocation failures. Since I have extracted a promise that this *will* be fixed in hardware, I'm happy to limit it on the current hardware to a maximum of 0x20000 PASIDs, which gives us 1MiB tables — still not ideal, but better than before. Reported by Mika Kuoppala <mika.kuoppala@linux.intel.com> and also by Xunlei Pang <xlpang@redhat.com> who submitted a simpler patch to fix only the allocation (and not the free) to the "correct" limit... which was still problematic. Signed-off-by: David Woodhouse <dwmw2@infradead.org> Cc: stable@vger.kernel.org
| * iommu/vt-d: Fix IOMMU lookup for SR-IOV Virtual FunctionsAshok Raj2016-10-302-1/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The VT-d specification (§8.3.3) says: ‘Virtual Functions’ of a ‘Physical Function’ are under the scope of the same remapping unit as the ‘Physical Function’. The BIOS is not required to list all the possible VFs in the scope tables, and arguably *shouldn't* make any attempt to do so, since there could be a huge number of them. This has been broken basically for ever — the VF is never going to match against a specific unit's scope, so it ends up being assigned to the INCLUDE_ALL IOMMU. Which was always actually correct by coincidence, but now we're looking at Root-Complex integrated devices with SR-IOV support it's going to start being wrong. Fix it to simply use pci_physfn() before doing the lookup for PCI devices. Cc: stable@vger.kernel.org Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: David Woodhouse <dwmw2@infradead.org>
* | iommu/vt-d: Fix dead-locks in disable_dmar_iommu() pathJoerg Roedel2016-11-081-2/+12
| | | | | | | | | | | | | | | | | | | | It turns out that the disable_dmar_iommu() code-path tried to get the device_domain_lock recursivly, which will dead-lock when this code runs on dmar removal. Fix both code-paths that could lead to the dead-lock. Fixes: 55d940430ab9 ('iommu/vt-d: Get rid of domain->iommu_lock') Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/arm-smmu: Fix out-of-bounds dereferenceRobin Murphy2016-11-081-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we iterate a master's config entries, what we generally care about is the entry's stream map index, rather than the entry index itself, so it's nice to have the iterator automatically assign the former from the latter. Unfortunately, booting with KASAN reveals the oversight that using a simple comma operator results in the entry index being dereferenced before being checked for validity, so we always access one element past the end of the fwspec array. Flip things around so that the check always happens before the index may be dereferenced. Fixes: adfec2e709d2 ("iommu/arm-smmu: Convert to iommu_fwspec") Reported-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/arm-smmu: Check that iommu_fwspecs are oursRobin Murphy2016-11-081-1/+1
| | | | | | | | | | | | | | | | | | We seem to have forgotten to check that iommu_fwspecs actually belong to us before we go ahead and dereference their private data. Oops. Fixes: 021bb8420d44 ("iommu/arm-smmu: Wire up generic configuration support") Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/arm-smmu: Don't inadvertently reject multiple SMMUv3sRobin Murphy2016-11-081-8/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We now delay installing our per-bus iommu_ops until we know an SMMU has successfully probed, as they don't serve much purpose beforehand, and doing so also avoids fights between multiple IOMMU drivers in a single kernel. However, the upshot of passing the return value of bus_set_iommu() back from our probe function is that if there happens to be more than one SMMUv3 device in a system, the second and subsequent probes will wind up returning -EBUSY to the driver core and getting torn down again. Avoid re-setting ops if ours are already installed, so that any genuine failures stand out. Fixes: 08d4ca2a672b ("iommu/arm-smmu: Support non-PCI devices with SMMUv3") CC: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> CC: Hanjun Guo <hanjun.guo@linaro.org> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/arm-smmu: Work around ARM DMA configurationRobin Murphy2016-11-081-0/+10
|/ | | | | | | | | | | | | | | | | | The 32-bit ARM DMA configuration code predates the IOMMU core's default domain functionality, and instead relies on allocating its own domains and attaching any devices using the generic IOMMU binding to them. Unfortunately, it does this relatively early on in the creation of the device, before we've seen our add_device callback, which leads us to attempt to operate on a half-configured master. To avoid a crash, check for this situation on attach, but refuse to play, as there's nothing we can do. This at least allows VFIO to keep working for people who update their 32-bit DTs to the generic binding, albeit with a few (innocuous) warnings from the DMA layer on boot. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* Merge tag 'iommu-updates-v4.9' of ↵Linus Torvalds2016-10-1113-848/+1118
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull IOMMU updates from Joerg Roedel: - support for interrupt virtualization in the AMD IOMMU driver. These patches were shared with the KVM tree and are already merged through that tree. - generic DT-binding support for the ARM-SMMU driver. With this the driver now makes use of the generic DMA-API code. This also required some changes outside of the IOMMU code, but these are acked by the respective maintainers. - more cleanups and fixes all over the place. * tag 'iommu-updates-v4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (40 commits) iommu/amd: No need to wait iommu completion if no dte irq entry change iommu/amd: Free domain id when free a domain of struct dma_ops_domain iommu/amd: Use standard bitmap operation to set bitmap iommu/amd: Clean up the cmpxchg64 invocation iommu/io-pgtable-arm: Check for v7s-incapable systems iommu/dma: Avoid PCI host bridge windows iommu/dma: Add support for mapping MSIs iommu/arm-smmu: Set domain geometry iommu/arm-smmu: Wire up generic configuration support Docs: dt: document ARM SMMU generic binding usage iommu/arm-smmu: Convert to iommu_fwspec iommu/arm-smmu: Intelligent SMR allocation iommu/arm-smmu: Add a stream map entry iterator iommu/arm-smmu: Streamline SMMU data lookups iommu/arm-smmu: Refactor mmu-masters handling iommu/arm-smmu: Keep track of S2CR state iommu/arm-smmu: Consolidate stream map entry state iommu/arm-smmu: Handle stream IDs more dynamically iommu/arm-smmu: Set PRIVCFG in stage 1 STEs iommu/arm-smmu: Support non-PCI devices with SMMUv3 ...
| *-------. Merge branches 'x86/amd', 'x86/vt-d', 'arm/exynos', 'arm/mediatek', ↵Joerg Roedel2016-09-2011-870/+1129
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | 'arm/renesas' and 'arm/smmu' into next
| | | | | | * iommu/io-pgtable-arm: Check for v7s-incapable systemsRobin Murphy2016-09-161-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On machines with no 32-bit addressable RAM whatsoever, we shouldn't even touch the v7s format as it's never going to work. Fixes: e5fc9753b1a8 ("iommu/io-pgtable: Add ARMv7 short descriptor support") Reported-by: Eric Auger <eric.auger@redhat.com> Tested-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/dma: Avoid PCI host bridge windowsRobin Murphy2016-09-161-1/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With our DMA ops enabled for PCI devices, we should avoid allocating IOVAs which a host bridge might misinterpret as peer-to-peer DMA and lead to faults, corruption or other badness. To be safe, punch out holes for all of the relevant host bridge's windows when initialising a DMA domain for a PCI device. CC: Marek Szyprowski <m.szyprowski@samsung.com> CC: Inki Dae <inki.dae@samsung.com> Reported-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/dma: Add support for mapping MSIsRobin Murphy2016-09-161-15/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When an MSI doorbell is located downstream of an IOMMU, attaching devices to a DMA ops domain and switching on translation leads to a rude shock when their attempt to write to the physical address returned by the irqchip driver faults (or worse, writes into some already-mapped buffer) and no interrupt is forthcoming. Address this by adding a hook for relevant irqchip drivers to call from their compose_msi_msg() callback, to swizzle the physical address with an appropriatly-mapped IOVA for any device attached to one of our DMA ops domains. Acked-by: Thomas Gleixner <tglx@linutronix.de> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Set domain geometryRobin Murphy2016-09-162-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For non-aperture-based IOMMUs, the domain geometry seems to have become the de-facto way of indicating the input address space size. That is quite a useful thing from the users' perspective, so let's do the same. Reviewed-by: Eric Auger <eric.auger@redhat.com> Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Wire up generic configuration supportRobin Murphy2016-09-161-60/+108
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With everything else now in place, fill in an of_xlate callback and the appropriate registration to plumb into the generic configuration machinery, and watch everything just work. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Convert to iommu_fwspecRobin Murphy2016-09-161-62/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the final step of preparation for full generic configuration support, swap our fixed-size master_cfg for the generic iommu_fwspec. For the legacy DT bindings, the driver simply gets to act as its own 'firmware'. Farewell, arbitrary MAX_MASTER_STREAMIDS! Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Intelligent SMR allocationRobin Murphy2016-09-161-78/+119
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Stream Match Registers are one of the more awkward parts of the SMMUv2 architecture; there are typically never enough to assign one to each stream ID in the system, and configuring them such that a single ID matches multiple entries is catastrophically bad - at best, every transaction raises a global fault; at worst, they go *somewhere*. To address the former issue, we can mask ID bits such that a single register may be used to match multiple IDs belonging to the same device or group, but doing so also heightens the risk of the latter problem (which can be nasty to debug). Tackle both problems at once by replacing the simple bitmap allocator with something much cleverer. Now that we have convenient in-memory representations of the stream mapping table, it becomes straightforward to properly validate new SMR entries against the current state, opening the door to arbitrary masking and SMR sharing. Another feature which falls out of this is that with IDs shared by separate devices being automatically accounted for, simply associating a group pointer with the S2CR offers appropriate group allocation almost for free, so hook that up in the process. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Add a stream map entry iteratorRobin Murphy2016-09-161-14/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We iterate over the SMEs associated with a master config quite a lot in various places, and are about to do so even more. Let's wrap the idiom in a handy iterator macro before the repetition gets out of hand. Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Streamline SMMU data lookupsRobin Murphy2016-09-161-33/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Simplify things somewhat by stashing our arm_smmu_device instance in drvdata, so that it's readily available to our driver model callbacks. Then we can excise the private list entirely, since the driver core already has a perfectly good list of SMMU devices we can use in the one instance we actually need to. Finally, make a further modest code saving with the relatively new of_device_get_match_data() helper. Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Refactor mmu-masters handlingRobin Murphy2016-09-161-269/+101
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To be able to support the generic bindings and handle of_xlate() calls, we need to be able to associate SMMUs and stream IDs directly with devices *before* allocating IOMMU groups. Furthermore, to support real default domains with multi-device groups we also have to handle domain attach on a per-device basis, as the "whole group at a time" assumption fails to properly handle subsequent devices added to a group after the first has already triggered default domain creation and attachment. To that end, use the now-vacant dev->archdata.iommu field for easy config and SMMU instance lookup, and unify config management by chopping down the platform-device-specific tree and probing the "mmu-masters" property on-demand instead. This may add a bit of one-off overhead to initially adding a new device, but we're about to deprecate that binding in favour of the inherently-more-efficient generic ones anyway. For the sake of simplicity, this patch does temporarily regress the case of aliasing PCI devices by losing the duplicate stream ID detection that the previous per-group config had. Stay tuned, because we'll be back to fix that in a better and more general way momentarily... Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Keep track of S2CR stateRobin Murphy2016-09-161-66/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Making S2CRs first-class citizens within the driver with a high-level representation of their state offers a neat solution to a few problems: Firstly, the information about which context a device's stream IDs are associated with is already present by necessity in the S2CR. With that state easily accessible we can refer directly to it and obviate the need to track an IOMMU domain in each device's archdata (its earlier purpose of enforcing correct attachment of multi-device groups now being handled by the IOMMU core itself). Secondly, the core API now deprecates explicit domain detach and expects domain attach to move devices smoothly from one domain to another; for SMMUv2, this notion maps directly to simply rewriting the S2CRs assigned to the device. By giving the driver a suitable abstraction of those S2CRs to work with, we can massively reduce the overhead of the current heavy-handed "detach, free resources, reallocate resources, attach" approach. Thirdly, making the software state hardware-shaped and attached to the SMMU instance once again makes suspend/resume of this register group that much simpler to implement in future. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Consolidate stream map entry stateRobin Murphy2016-09-161-61/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to consider SMR masking, we really want to be able to validate ID/mask pairs against existing SMR contents to prevent stream match conflicts, which at best would cause transactions to fault unexpectedly, and at worst lead to silent unpredictable behaviour. With our SMMU instance data holding only an allocator bitmap, and the SMR values themselves scattered across master configs hanging off devices which we may have no way of finding, there's essentially no way short of digging everything back out of the hardware. Similarly, the thought of power management ops to support suspend/resume faces the exact same problem. By massaging the software state into a closer shape to the underlying hardware, everything comes together quite nicely; the allocator and the high-level view of the data become a single centralised state which we can easily keep track of, and to which any updates can be validated in full before being synchronised to the hardware itself. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Handle stream IDs more dynamicallyRobin Murphy2016-09-161-21/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rather than assuming fixed worst-case values for stream IDs and SMR masks, keep track of whatever implemented bits the hardware actually reports. This also obviates the slightly questionable validation of SMR fields in isolation - rather than aborting the whole SMMU probe for a hardware configuration which is still architecturally valid, we can simply refuse masters later if they try to claim an unrepresentable ID or mask (which almost certainly implies a DT error anyway). Acked-by: Will Deacon <will.deacon@arm.com> Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Set PRIVCFG in stage 1 STEsRobin Murphy2016-09-161-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement the SMMUv3 equivalent of d346180e70b9 ("iommu/arm-smmu: Treat all device transactions as unprivileged"), so that once again those pesky DMA controllers with their privileged instruction fetches don't unexpectedly fault in stage 1 domains due to VMSAv8 rules. Acked-by: Will Deacon <will.deacon@arm.com> Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Support non-PCI devices with SMMUv3Robin Murphy2016-09-162-7/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With the device <-> stream ID relationship suitably abstracted and of_xlate() hooked up, the PCI dependency now looks, and is, entirely arbitrary. Any bus using the of_dma_configure() mechanism will work, so extend support to the platform and AMBA buses which do just that. Acked-by: Will Deacon <will.deacon@arm.com> Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Implement of_xlate() for SMMUv3Robin Murphy2016-09-161-170/+128
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that we can properly describe the mapping between PCI RIDs and stream IDs via "iommu-map", and have it fed it to the driver automatically via of_xlate(), rework the SMMUv3 driver to benefit from that, and get rid of the current misuse of the "iommus" binding. Since having of_xlate wired up means that masters will now be given the appropriate DMA ops, we also need to make sure that default domains work properly. This necessitates dispensing with the "whole group at a time" notion for attaching to a domain, as devices which share a group get attached to the group's default domain one by one as they are initially probed. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Fall back to global bypassRobin Murphy2016-09-161-4/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Unlike SMMUv2, SMMUv3 has no easy way to bypass unknown stream IDs, other than allocating and filling in the entire stream table with bypass entries, which for some configurations would waste *gigabytes* of RAM. Otherwise, all transactions on unknown stream IDs will simply be aborted with a C_BAD_STREAMID event. Rather than render the system unusable in the case of an invalid DT, avoid enabling the SMMU altogether such that everything bypasses (though letting the explicit disable_bypass option take precedence). Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu: Introduce iommu_fwspecRobin Murphy2016-09-162-2/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce a common structure to hold the per-device firmware data that most IOMMU drivers need to keep track of. This enables us to configure much of that data from common firmware code, and consolidate a lot of the equivalent implementations, device look-up tables, etc. which are currently strewn across IOMMU drivers. This will also be enable us to address the outstanding "multiple IOMMUs on the platform bus" problem by tweaking IOMMU API calls to prefer dev->fwspec->ops before falling back to dev->bus->iommu_ops, and thus gracefully handle those troublesome systems which we currently cannot. As the first user, hook up the OF IOMMU configuration mechanism. The driver-defined nature of DT cells means that we still need the drivers to translate and add the IDs themselves, but future users such as the much less free-form ACPI IORT will be much simpler and self-contained. CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Suggested-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/of: Handle iommu-map property for PCIRobin Murphy2016-09-161-5/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that we have a way to pick up the RID translation and target IOMMU, hook up of_iommu_configure() to bring PCI devices into the of_xlate mechanism and allow them IOMMU-backed DMA ops without the need for driver-specific handling. Reviewed-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Disable interrupts whilst holding the cmdq lockWill Deacon2016-09-161-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The cmdq lock is taken whenever we issue commands into the command queue, which can occur in IRQ context (as a result of unmap) or in process context (as a result of a threaded IRQ handler or device probe). This can lead to a theoretical deadlock if the interrupt handler performing the unmap hits whilst the lock is taken, so explicitly use the {irqsave,irqrestore} spin_lock accessors for the cmdq lock. Tested-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Fix polling of command queueJean-Philippe Brucker2016-09-161-19/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the SMMUv3 driver attempts to send a command, it adds an entry to the command queue. This is a circular buffer, where both the producer and consumer have a wrap bit. When producer.index == consumer.index and producer.wrap == consumer.wrap, the list is empty. When producer.index == consumer.index and producer.wrap != consumer.wrap, the list is full. If the list is full when the driver needs to add a command, it waits for the SMMU to consume one command, and advance the consumer pointer. The problem is that we currently rely on "X before Y" operation to know if entries have been consumed, which is a bit fiddly since it only makes sense when the distance between X and Y is less than or equal to the size of the queue. At the moment when the list is full, we use "Consumer before Producer + 1", which is out of range and returns a value opposite to what we expect: when the queue transitions to not full, we stay in the polling loop and time out, printing an error. Given that the actual bug was difficult to determine, simplify the polling logic by relying exclusively on queue_full and queue_empty, that don't have this range constraint. Polling the queue is now straightforward: * When we want to add a command and the list is full, wait until it isn't full and retry. * After adding a sync, wait for the list to be empty before returning. Suggested-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
| | | | | | * iommu/arm-smmu: Support v7s context formatRobin Murphy2016-09-161-22/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fill in the last bits of machinery required to drive a stage 1 context bank in v7 short descriptor format. By default we'll prefer to use it only when the CPUs are also using the same format, such that we're guaranteed that everything will be strictly 32-bit. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-23 16:43:27 +0000